2024-03-12T03:47:41.749 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-12T03:47:41.749 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348 --owner scheduled_teuthology@teuthology --name teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} -- /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/orig.config.yaml 2024-03-12T03:47:41.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:47:41.892 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7591348' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 94a539707733325322964c6a187c6a1b41be1daf subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 94a539707733325322964c6a187c6a1b41be1daf kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad owner: scheduled_teuthology@teuthology priority: 250 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8179 sha1: 94a539707733325322964c6a187c6a1b41be1daf sleep_before_teardown: 0 subset: 297/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJwPTkt6lpgayKwmzGh6G+CKpCR7nFE6xi/tjhd2VFmzYQm/EjQK+BoOEBczi14EEvC216yVNmz8BVh3Mj88a5k= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5KBhfoTIyfHd2ltYBKNkMkHxZM2TzqfsHQjC6DZhgyDMU2qf7iepU6kSc1Oz4Kn8oOeGNJR42yvHLG4aOjNXc= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3BJyFff1cqpwLxYCcbthQ1lcefv47aApahzvQRmI9yl7fjCBZWRzo0rZpHzbnIZSR78GU2uI5gkl8cbCEbAOc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-11_14:02:35 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-12T03:47:41.892 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa; will attempt to use it 2024-03-12T03:47:41.892 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks 2024-03-12T03:47:41.893 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-12T03:47:41.894 INFO:teuthology.task.internal:Checking packages... 2024-03-12T03:47:41.916 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '94a539707733325322964c6a187c6a1b41be1daf' 2024-03-12T03:47:41.917 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-12T03:47:41.917 INFO:teuthology.packaging:ref: None 2024-03-12T03:47:41.917 INFO:teuthology.packaging:tag: None 2024-03-12T03:47:41.917 INFO:teuthology.packaging:branch: squid 2024-03-12T03:47:41.917 INFO:teuthology.packaging:sha1: 94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T03:47:41.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-12T03:47:42.140 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1607.g94a53970 2024-03-12T03:47:42.141 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-12T03:47:42.152 INFO:teuthology.task.internal:no buildpackages task found 2024-03-12T03:47:42.152 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-12T03:47:42.176 INFO:teuthology.task.internal:Saving configuration 2024-03-12T03:47:42.196 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-12T03:47:42.204 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T03:47:42.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 03:42:37.212831', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJwPTkt6lpgayKwmzGh6G+CKpCR7nFE6xi/tjhd2VFmzYQm/EjQK+BoOEBczi14EEvC216yVNmz8BVh3Mj88a5k='} 2024-03-12T03:47:42.253 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 03:42:37.210894', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5KBhfoTIyfHd2ltYBKNkMkHxZM2TzqfsHQjC6DZhgyDMU2qf7iepU6kSc1Oz4Kn8oOeGNJR42yvHLG4aOjNXc='} 2024-03-12T03:47:42.278 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 03:42:37.214475', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3BJyFff1cqpwLxYCcbthQ1lcefv47aApahzvQRmI9yl7fjCBZWRzo0rZpHzbnIZSR78GU2uI5gkl8cbCEbAOc='} 2024-03-12T03:47:42.278 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-12T03:47:42.286 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-12T03:47:42.286 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-12T03:47:42.286 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-12T03:47:42.286 INFO:teuthology.run_tasks:Running task console_log... 2024-03-12T03:47:42.392 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7bcb512160>, signals=[15]) 2024-03-12T03:47:42.393 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-12T03:47:42.400 INFO:teuthology.task.internal:Opening connections... 2024-03-12T03:47:42.400 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T03:47:42.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:42.512 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-03-12T03:47:42.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:42.602 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-03-12T03:47:42.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:42.708 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-12T03:47:42.715 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-03-12T03:47:42.734 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-03-12T03:47:42.734 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:42.791 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:42.792 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:42.792 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:42.792 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:42.792 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:42.792 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:42.792 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-03-12T03:47:42.828 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-03-12T03:47:42.845 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-03-12T03:47:42.845 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-03-12T03:47:42.899 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:42.900 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:42.901 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-03-12T03:47:42.919 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-03-12T03:47:42.937 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-03-12T03:47:42.937 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-03-12T03:47:42.993 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:42.994 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:42.995 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:42.995 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-03-12T03:47:43.019 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-12T03:47:43.034 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-12T03:47:43.053 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-12T03:47:43.053 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-03-12T03:47:43.055 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-03-12T03:47:43.058 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-03-12T03:47:43.074 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-12T03:47:43.085 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-12T03:47:43.085 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-03-12T03:47:43.112 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-03-12T03:47:43.115 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-03-12T03:47:43.216 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-12T03:47:43.299 INFO:teuthology.run_tasks:Running task kernel... 2024-03-12T03:47:43.320 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-12T03:47:43.320 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-12T03:47:43.320 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T03:47:43.321 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T03:47:43.321 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T03:47:43.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:47:43.338 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-03-12T03:47:43.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:47:43.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:47:43.340 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-03-12T03:47:43.340 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-03-12T03:47:43.396 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-361.el9.x86_64 2024-03-12T03:47:43.396 INFO:teuthology.task.kernel:Running kernel on smithi119: 5.14.0-361.el9.x86_64 2024-03-12T03:47:43.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum install -y kernel 2024-03-12T03:47:43.397 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-361.el9.x86_64 2024-03-12T03:47:43.397 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.14.0-361.el9.x86_64 2024-03-12T03:47:43.397 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum install -y kernel 2024-03-12T03:47:43.399 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-361.el9.x86_64 2024-03-12T03:47:43.399 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.14.0-361.el9.x86_64 2024-03-12T03:47:43.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2024-03-12T03:47:45.529 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-12T03:47:45.537 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-12T03:47:45.552 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-12T03:47:46.954 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-12T03:47:46.995 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-12T03:47:47.202 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-12T03:47:48.219 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-03-12T03:47:48.293 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-12T03:47:48.378 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-03-12T03:47:50.563 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-12T03:48:02.926 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 703 kB/s | 6.1 MB 00:08 2024-03-12T03:48:04.585 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-03-12T03:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-12T03:48:09.193 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-12T03:48:10.719 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-03-12T03:48:12.535 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 832 kB/s | 19 MB 00:23 2024-03-12T03:48:14.125 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-03-12T03:48:16.736 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-12T03:48:19.044 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-12T03:48:19.295 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2024-03-12T03:48:21.251 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-12T03:48:21.251 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-12T03:48:21.355 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:48:21.360 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-03-12T03:48:21.361 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-03-12T03:48:21.361 INFO:teuthology.orchestra.run.smithi136.stdout:Install 4 Packages 2024-03-12T03:48:21.361 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:48:21.361 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 100 M 2024-03-12T03:48:21.361 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 123 M 2024-03-12T03:48:21.361 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-03-12T03:48:23.178 INFO:teuthology.orchestra.run.smithi136.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 3.5 MB/s | 6.0 MB 00:01 2024-03-12T03:48:23.654 INFO:teuthology.orchestra.run.smithi136.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 9.6 MB/s | 21 MB 00:02 2024-03-12T03:48:25.670 INFO:teuthology.orchestra.run.smithi136.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 9.4 MB/s | 39 MB 00:04 2024-03-12T03:48:28.495 INFO:teuthology.orchestra.run.smithi136.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 6.3 MB/s | 33 MB 00:05 2024-03-12T03:48:28.496 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-03-12T03:48:28.496 INFO:teuthology.orchestra.run.smithi136.stdout:Total 14 MB/s | 100 MB 00:07 2024-03-12T03:48:29.376 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-03-12T03:48:29.769 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-03-12T03:48:29.769 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-03-12T03:48:31.088 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-03-12T03:48:31.088 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-03-12T03:48:33.767 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-03-12T03:48:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-12T03:48:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:48:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:48:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:48:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:48:42.437 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:48:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:49:16.279 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:49:16.279 INFO:teuthology.orchestra.run.smithi136.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-12T03:49:16.280 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:49:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:49:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:49:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-12T03:49:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:49:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:49:17.236 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-03-12T03:49:17.770 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T03:49:17.933 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T03:49:17.933 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T03:49:17.934 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-12T03:49:17.934 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-12T03:49:17.934 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-12T03:49:17.934 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-03-12T03:49:17.934 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-03-12T03:49:17.934 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T03:49:18.054 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T03:49:18.055 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T03:49:18.056 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -qi grub2-tools 2024-03-12T03:49:18.090 INFO:teuthology.orchestra.run.smithi136.stdout:Name : grub2-tools 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Epoch : 1 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Version : 2.06 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Release : 68.el9 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: x86_64 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Group : Unspecified 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Size : 8264254 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:License : GPLv3+ 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Packager : builder@centos.org 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:Vendor : CentOS 2024-03-12T03:49:18.091 INFO:teuthology.orchestra.run.smithi136.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-12T03:49:18.092 INFO:teuthology.orchestra.run.smithi136.stdout:Summary : Support tools for GRUB. 2024-03-12T03:49:18.092 INFO:teuthology.orchestra.run.smithi136.stdout:Description : 2024-03-12T03:49:18.092 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:49:18.092 INFO:teuthology.orchestra.run.smithi136.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-12T03:49:18.092 INFO:teuthology.orchestra.run.smithi136.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-12T03:49:18.092 INFO:teuthology.orchestra.run.smithi136.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-12T03:49:18.093 INFO:teuthology.orchestra.run.smithi136.stdout:hardware devices. 2024-03-12T03:49:18.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:49:18.093 INFO:teuthology.orchestra.run.smithi136.stdout:This subpackage provides tools for support of all platforms. 2024-03-12T03:49:18.094 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-12T03:49:18.094 INFO:teuthology.task.kernel:Updating grub on smithi136 to boot 5.14.0-427.el9.x86_64 2024-03-12T03:49:18.094 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-12T03:49:18.718 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-03-12T03:49:20.144 INFO:teuthology.orchestra.run.smithi136.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-12T03:49:20.184 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-03-12T03:49:20.186 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-03-12T03:49:20.202 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.B2DeKACoYD 2024-03-12T03:49:20.203 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.B2DeKACoYD 2024-03-12T03:49:20.270 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.B2DeKACoYD 2024-03-12T03:49:20.411 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.B2DeKACoYD is 6KB 2024-03-12T03:49:20.460 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.B2DeKACoYD 2024-03-12T03:49:20.473 DEBUG:teuthology.orchestra.run.smithi136:> sudo /bin/ls /boot/loader/entries || true 2024-03-12T03:49:20.538 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-12T03:49:20.538 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-12T03:49:20.538 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-12T03:49:20.539 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-12T03:49:20.540 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-12T03:49:20.665 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-03-12T03:49:50.668 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-12T03:49:50.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-12T03:49:50.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:50:09.183 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-12T03:50:18.183 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-12T03:50:18.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:50:21.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-12T03:50:33.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-12T03:50:33.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:50:51.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-12T03:51:04.604 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 117 kB/s | 21 MB 02:59 2024-03-12T03:51:06.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-12T03:51:06.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:51:07.042 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-03-12T03:51:07.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-12T03:51:07.482 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-427.el9.x86_64"... 2024-03-12T03:51:07.482 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-03-12T03:51:07.495 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-427.el9.x86_64 2024-03-12T03:51:07.496 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-12T03:51:07.496 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T03:51:07.496 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-12T03:51:08.496 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-12T03:51:08.497 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-12T03:51:08.586 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-03-12T03:51:08.621 DEBUG:teuthology.parallel:result is None 2024-03-12T03:51:09.495 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-12T03:51:11.619 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-12T03:51:11.619 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-12T03:51:11.707 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-03-12T03:51:11.717 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-03-12T03:51:11.718 INFO:teuthology.orchestra.run.smithi119.stdout:Install 4 Packages 2024-03-12T03:51:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:51:11.718 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 100 M 2024-03-12T03:51:11.718 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 123 M 2024-03-12T03:51:11.718 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-03-12T03:51:12.429 INFO:teuthology.orchestra.run.smithi119.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 14 MB/s | 6.0 MB 00:00 2024-03-12T03:51:12.971 INFO:teuthology.orchestra.run.smithi119.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 21 MB/s | 21 MB 00:00 2024-03-12T03:51:13.647 INFO:teuthology.orchestra.run.smithi119.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 27 MB/s | 33 MB 00:01 2024-03-12T03:51:14.180 INFO:teuthology.orchestra.run.smithi119.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-03-12T03:51:14.183 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-03-12T03:51:14.183 INFO:teuthology.orchestra.run.smithi119.stdout:Total 40 MB/s | 100 MB 00:02 2024-03-12T03:51:15.071 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-03-12T03:51:15.453 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-03-12T03:51:15.454 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-03-12T03:51:16.808 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-03-12T03:51:16.808 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-03-12T03:51:19.756 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-03-12T03:51:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-12T03:51:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:51:23.339 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:51:23.344 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 115 kB/s | 21 MB 03:03 2024-03-12T03:51:23.519 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:51:28.248 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:51:28.453 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-12T03:51:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:51:30.744 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-12T03:51:30.744 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-12T03:51:30.847 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-03-12T03:51:30.864 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-03-12T03:51:30.864 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-03-12T03:51:30.865 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T03:51:30.866 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 100 M 2024-03-12T03:51:30.866 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 123 M 2024-03-12T03:51:30.866 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-03-12T03:51:33.201 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:51:35.296 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 1.4 MB/s | 6.0 MB 00:04 2024-03-12T03:51:38.579 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): kernel-modules-core-5.14.0-427.el9.x86_6 10 MB/s | 33 MB 00:03 2024-03-12T03:51:45.387 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): kernel-core-5.14.0-427.el9.x86_64.rpm 1.4 MB/s | 21 MB 00:14 2024-03-12T03:51:46.137 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 2.6 MB/s | 39 MB 00:15 2024-03-12T03:51:46.138 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-03-12T03:51:46.138 INFO:teuthology.orchestra.run.smithi042.stdout:Total 6.5 MB/s | 100 MB 00:15 2024-03-12T03:51:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-03-12T03:51:47.496 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-03-12T03:51:47.496 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-03-12T03:51:48.896 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-03-12T03:51:48.897 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-03-12T03:51:51.764 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-03-12T03:51:52.444 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-12T03:51:52.580 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:51:55.506 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:51:55.686 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:52:00.790 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:52:00.915 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:02.555 INFO:teuthology.orchestra.run.smithi119.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-12T03:52:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:52:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-12T03:52:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:52:03.131 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:52:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:52:03.543 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-03-12T03:52:04.049 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T03:52:04.198 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T03:52:04.198 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T03:52:04.198 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-12T03:52:04.198 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-12T03:52:04.198 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-12T03:52:04.198 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-12T03:52:04.199 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=None, version=distro) 2024-03-12T03:52:04.199 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T03:52:04.318 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T03:52:04.319 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T03:52:04.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2024-03-12T03:52:04.353 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2024-03-12T03:52:04.353 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2024-03-12T03:52:04.353 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2024-03-12T03:52:04.353 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 68.el9 2024-03-12T03:52:04.353 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2024-03-12T03:52:04.353 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-12T03:52:04.353 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8264254 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-12T03:52:04.354 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-12T03:52:04.355 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2024-03-12T03:52:04.355 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:52:04.355 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2024-03-12T03:52:04.355 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-12T03:52:04.356 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 5.14.0-427.el9.x86_64 2024-03-12T03:52:04.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-12T03:52:04.943 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-03-12T03:52:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:06.223 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-12T03:52:06.261 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-03-12T03:52:06.264 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-03-12T03:52:06.289 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.i7TNYpbyoM 2024-03-12T03:52:06.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.i7TNYpbyoM 2024-03-12T03:52:06.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.i7TNYpbyoM 2024-03-12T03:52:06.497 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.i7TNYpbyoM is 6KB 2024-03-12T03:52:06.545 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.i7TNYpbyoM 2024-03-12T03:52:06.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2024-03-12T03:52:06.624 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-12T03:52:06.624 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-12T03:52:06.625 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-12T03:52:06.625 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-12T03:52:06.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-12T03:52:06.774 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-03-12T03:52:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:36.753 INFO:teuthology.orchestra.run.smithi042.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-12T03:52:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T03:52:36.777 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-12T03:52:36.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-12T03:52:36.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:52:36.840 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-12T03:52:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T03:52:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T03:52:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T03:52:37.699 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-03-12T03:52:38.216 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T03:52:38.410 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T03:52:38.410 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T03:52:38.410 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-12T03:52:38.410 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-12T03:52:38.411 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-12T03:52:38.411 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-12T03:52:38.411 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=None, version=distro) 2024-03-12T03:52:38.411 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T03:52:38.537 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T03:52:38.538 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T03:52:38.538 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 68.el9 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8264254 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-03-12T03:52:38.576 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-12T03:52:38.577 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-12T03:52:38.578 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-03-12T03:52:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T03:52:38.578 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-03-12T03:52:38.579 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-12T03:52:38.579 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 5.14.0-427.el9.x86_64 2024-03-12T03:52:38.579 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-12T03:52:39.176 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-03-12T03:52:40.659 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-12T03:52:40.700 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-03-12T03:52:40.702 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-03-12T03:52:40.719 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.N08uxqxONs 2024-03-12T03:52:40.719 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.N08uxqxONs 2024-03-12T03:52:40.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.N08uxqxONs 2024-03-12T03:52:40.940 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.N08uxqxONs is 6KB 2024-03-12T03:52:40.990 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.N08uxqxONs 2024-03-12T03:52:41.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-03-12T03:52:41.079 INFO:teuthology.orchestra.run.smithi042.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-12T03:52:41.079 INFO:teuthology.orchestra.run.smithi042.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-12T03:52:41.080 INFO:teuthology.orchestra.run.smithi042.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-12T03:52:41.080 INFO:teuthology.orchestra.run.smithi042.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-12T03:52:41.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-12T03:52:41.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-03-12T03:52:55.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-12T03:53:04.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-12T03:53:04.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:53:07.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-12T03:53:11.248 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-12T03:53:11.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-03-12T03:53:11.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:53:19.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-12T03:53:19.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:53:29.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-03-12T03:53:37.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-12T03:53:38.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-03-12T03:53:38.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:53:52.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-12T03:53:52.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:53:53.243 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-03-12T03:53:53.684 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-12T03:53:53.684 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-427.el9.x86_64"... 2024-03-12T03:53:53.684 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-03-12T03:53:53.698 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-427.el9.x86_64 2024-03-12T03:53:53.698 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-12T03:53:53.699 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T03:53:53.699 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-12T03:53:54.700 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-12T03:53:54.700 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-12T03:53:54.786 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-03-12T03:53:54.828 DEBUG:teuthology.parallel:result is None 2024-03-12T03:54:38.635 DEBUG:teuthology.orchestra.remote:timed out 2024-03-12T03:54:50.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-03-12T03:54:50.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:54:50.918 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-03-12T03:54:51.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-03-12T03:54:51.276 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-427.el9.x86_64"... 2024-03-12T03:54:51.277 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-03-12T03:54:51.291 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-427.el9.x86_64 2024-03-12T03:54:51.291 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-12T03:54:51.292 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T03:54:51.292 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-12T03:54:52.292 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-12T03:54:52.292 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-12T03:54:52.351 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-03-12T03:54:52.391 DEBUG:teuthology.parallel:result is None 2024-03-12T03:54:52.391 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-12T03:54:52.398 INFO:teuthology.task.internal:Creating test directory... 2024-03-12T03:54:52.399 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T03:54:52.401 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T03:54:52.405 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T03:54:52.423 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-12T03:54:52.430 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-12T03:54:52.441 INFO:teuthology.task.internal:Creating archive directory... 2024-03-12T03:54:52.441 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T03:54:52.459 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T03:54:52.464 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T03:54:52.499 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-12T03:54:52.506 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-12T03:54:52.506 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T03:54:52.524 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T03:54:52.527 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T03:54:52.559 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T03:54:52.566 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T03:54:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T03:54:52.573 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T03:54:52.580 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T03:54:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T03:54:52.584 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-12T03:54:52.592 INFO:teuthology.task.internal:Configuring sudo... 2024-03-12T03:54:52.592 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T03:54:52.618 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T03:54:52.626 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T03:54:52.656 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-12T03:54:52.666 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-12T03:54:52.666 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T03:54:52.688 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T03:54:52.696 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T03:54:52.713 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T03:54:52.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T03:54:52.839 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T03:54:52.923 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T03:54:52.968 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T03:54:52.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T03:54:53.035 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T03:54:53.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T03:54:53.142 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T03:54:53.188 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T03:54:53.230 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T03:54:53.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T03:54:53.299 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T03:54:53.344 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T03:54:53.400 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T03:54:53.486 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T03:54:53.529 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T03:54:53.529 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T03:54:53.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-03-12T03:54:53.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-03-12T03:54:53.603 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-03-12T03:54:53.654 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T03:54:53.656 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T03:54:53.705 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T03:54:54.103 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-12T03:54:54.111 INFO:teuthology.task.internal:Starting timer... 2024-03-12T03:54:54.111 INFO:teuthology.run_tasks:Running task pcp... 2024-03-12T03:54:54.121 INFO:teuthology.run_tasks:Running task selinux... 2024-03-12T03:54:54.131 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-12T03:54:54.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-03-12T03:54:54.192 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-03-12T03:54:54.194 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-03-12T03:54:54.256 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-03-12T03:54:54.258 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-03-12T03:54:54.315 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-03-12T03:54:54.317 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-12T03:54:54.317 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-03-12T03:54:54.338 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-03-12T03:54:54.339 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-03-12T03:54:54.361 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-03-12T03:54:54.361 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-03-12T03:54:54.382 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-03-12T03:54:54.383 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-03-12T03:54:54.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T03:54:54.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:54:54.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T03:54:54.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:54:54.444 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T03:54:54.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:54:54.472 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-12T03:54:54.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-03-12T03:54:54.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-03-12T03:54:54.547 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-03-12T03:54:54.583 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-12T03:54:54.593 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-12T03:54:54.599 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-12T03:54:54.727 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-12T03:54:54.744 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-12T03:54:54.745 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi119.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-12T03:59:09.235 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-03-12T03:59:09.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-03-12T03:59:09.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:59:09.332 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-03-12T03:59:09.413 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-03-12T03:59:09.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-12T03:59:09.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:59:09.500 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-03-12T03:59:09.584 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-12T03:59:09.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-12T03:59:09.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:59:09.714 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-03-12T03:59:09.792 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-12T03:59:09.792 INFO:teuthology.run_tasks:Running task clock... 2024-03-12T03:59:09.803 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-12T03:59:09.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T03:59:09.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T03:59:09.806 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T03:59:09.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T03:59:09.809 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T03:59:09.809 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T03:59:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T03:59:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T03:59:09.853 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T03:59:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T03:59:09.862 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T03:59:09.878 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T03:59:09.882 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-03-12T03:59:09.888 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-03-12T03:59:09.895 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-03-12T03:59:09.902 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-03-12T03:59:09.907 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-03-12T03:59:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T03:59:09.919 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T03:59:09.920 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-03-12T03:59:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T03:59:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T03:59:09.937 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T03:59:09.953 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T03:59:09.980 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-03-12T03:59:09.983 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T03:59:09.983 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-03-12T03:59:09.983 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:09.983 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:09.983 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:09.983 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:09.987 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-03-12T03:59:09.991 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T03:59:09.991 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-03-12T03:59:09.991 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:09.991 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:09.991 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:09.992 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:10.009 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-03-12T03:59:10.012 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T03:59:10.012 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-03-12T03:59:10.012 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:10.012 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:10.012 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:10.012 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T03:59:10.013 INFO:teuthology.run_tasks:Running task install... 2024-03-12T03:59:10.021 DEBUG:teuthology.task.install:project ceph 2024-03-12T03:59:10.021 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}} 2024-03-12T03:59:10.021 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'} 2024-03-12T03:59:10.021 INFO:teuthology.task.install:Using flavor: default 2024-03-12T03:59:10.027 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-12T03:59:10.027 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-12T03:59:10.028 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-12T03:59:10.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T03:59:10.030 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-12T03:59:10.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T03:59:10.031 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-12T03:59:10.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T03:59:10.176 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-12T03:59:10.177 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-12T03:59:10.180 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-12T03:59:10.180 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-12T03:59:10.248 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-12T03:59:10.248 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-12T03:59:10.298 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T03:59:10.299 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T03:59:10.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T03:59:10.299 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T03:59:10.299 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T03:59:10.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T03:59:10.327 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T03:59:10.327 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T03:59:10.328 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T03:59:10.328 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T03:59:10.395 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T03:59:10.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T03:59:10.421 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T03:59:10.421 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T03:59:10.421 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T03:59:10.449 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T03:59:10.449 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T03:59:10.475 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T03:59:10.476 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T03:59:10.518 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T03:59:10.541 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-03-12T03:59:10.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-03-12T03:59:10.543 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-03-12T03:59:10.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-03-12T03:59:10.598 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T03:59:10.665 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-03-12T03:59:10.666 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-03-12T03:59:10.781 INFO:teuthology.orchestra.run.smithi119.stdout:74 files removed 2024-03-12T03:59:10.806 INFO:teuthology.orchestra.run.smithi042.stdout:74 files removed 2024-03-12T03:59:10.809 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T03:59:10.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T03:59:10.910 INFO:teuthology.orchestra.run.smithi136.stdout:74 files removed 2024-03-12T03:59:10.940 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T03:59:11.537 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 199 kB/s | 82 kB 00:00 2024-03-12T03:59:11.592 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 247 kB/s | 82 kB 00:00 2024-03-12T03:59:11.632 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 172 kB/s | 82 kB 00:00 2024-03-12T03:59:11.921 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-03-12T03:59:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-03-12T03:59:12.042 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-12T03:59:12.297 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-12T03:59:12.401 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-12T03:59:12.408 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-03-12T03:59:12.749 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-12T03:59:12.828 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-12T03:59:12.861 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-12T03:59:14.183 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-12T03:59:14.270 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-12T03:59:14.320 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2024-03-12T03:59:15.549 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-03-12T03:59:15.730 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-03-12T03:59:16.211 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-03-12T03:59:19.395 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-03-12T03:59:19.812 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-03-12T03:59:23.387 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-12T03:59:24.298 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.1 MB 00:00 2024-03-12T03:59:24.863 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 15 kB 00:00 2024-03-12T03:59:25.930 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-03-12T03:59:26.056 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-12T03:59:31.014 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-03-12T03:59:31.031 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-03-12T03:59:33.502 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T03:59:33.503 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T03:59:33.517 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T03:59:33.518 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T03:59:33.519 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T03:59:33.520 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T03:59:33.522 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T03:59:33.525 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T03:59:33.526 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T03:59:33.605 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-03-12T03:59:33.623 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================================ 2024-03-12T03:59:33.623 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-03-12T03:59:33.623 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================================ 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-12T03:59:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T03:59:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T03:59:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T03:59:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-12T03:59:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T03:59:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-12T03:59:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-12T03:59:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T03:59:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T03:59:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T03:59:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T03:59:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================================ 2024-03-12T03:59:33.636 INFO:teuthology.orchestra.run.smithi136.stdout:Install 150 Packages 2024-03-12T03:59:33.637 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 24 Packages 2024-03-12T03:59:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T03:59:33.637 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 223 M 2024-03-12T03:59:33.637 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-03-12T03:59:34.340 INFO:teuthology.orchestra.run.smithi136.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-03-12T03:59:34.657 INFO:teuthology.orchestra.run.smithi136.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 2.7 MB/s | 886 kB 00:00 2024-03-12T03:59:34.740 INFO:teuthology.orchestra.run.smithi136.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 1.7 MB/s | 145 kB 00:00 2024-03-12T03:59:35.157 INFO:teuthology.orchestra.run.smithi136.stdout:(4/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 5.3 MB/s | 5.3 MB 00:01 2024-03-12T03:59:35.199 INFO:teuthology.orchestra.run.smithi136.stdout:(5/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 4.6 MB/s | 2.1 MB 00:00 2024-03-12T03:59:35.416 INFO:teuthology.orchestra.run.smithi136.stdout:(6/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-03-12T03:59:35.691 INFO:teuthology.orchestra.run.smithi136.stdout:(7/174): ceph-common-19.0.0-1607.g94a53970.el9. 13 MB/s | 20 MB 00:01 2024-03-12T03:59:35.749 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.1 MB 00:01 2024-03-12T03:59:35.950 INFO:teuthology.orchestra.run.smithi136.stdout:(8/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 6.2 MB/s | 4.7 MB 00:00 2024-03-12T03:59:36.050 INFO:teuthology.orchestra.run.smithi136.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 252 kB/s | 25 kB 00:00 2024-03-12T03:59:36.600 INFO:teuthology.orchestra.run.smithi136.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 11 MB/s | 9.8 MB 00:00 2024-03-12T03:59:36.667 INFO:teuthology.orchestra.run.smithi136.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 3.2 MB/s | 217 kB 00:00 2024-03-12T03:59:36.726 INFO:teuthology.orchestra.run.smithi136.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 463 kB/s | 27 kB 00:00 2024-03-12T03:59:36.784 INFO:teuthology.orchestra.run.smithi136.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 536 kB/s | 31 kB 00:00 2024-03-12T03:59:36.951 INFO:teuthology.orchestra.run.smithi136.stdout:(14/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 11 MB/s | 17 MB 00:01 2024-03-12T03:59:36.993 INFO:teuthology.orchestra.run.smithi136.stdout:(15/174): libcephfs2-19.0.0-1607.g94a53970.el9. 3.5 MB/s | 742 kB 00:00 2024-03-12T03:59:37.018 INFO:teuthology.orchestra.run.smithi136.stdout:(16/174): libcephsqlite-19.0.0-1607.g94a53970.e 2.4 MB/s | 163 kB 00:00 2024-03-12T03:59:37.052 INFO:teuthology.orchestra.run.smithi136.stdout:(17/174): librados-devel-19.0.0-1607.g94a53970. 2.1 MB/s | 127 kB 00:00 2024-03-12T03:59:37.085 INFO:teuthology.orchestra.run.smithi136.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 7.3 MB/s | 498 kB 00:00 2024-03-12T03:59:37.152 INFO:teuthology.orchestra.run.smithi136.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 678 kB/s | 45 kB 00:00 2024-03-12T03:59:37.244 INFO:teuthology.orchestra.run.smithi136.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.4 MB/s | 131 kB 00:00 2024-03-12T03:59:37.328 INFO:teuthology.orchestra.run.smithi136.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 1.9 MB/s | 161 kB 00:00 2024-03-12T03:59:37.375 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-03-12T03:59:37.428 INFO:teuthology.orchestra.run.smithi136.stdout:(22/174): librgw2-19.0.0-1607.g94a53970.el9.x86 14 MB/s | 5.1 MB 00:00 2024-03-12T03:59:37.453 INFO:teuthology.orchestra.run.smithi136.stdout:(23/174): python3-rados-19.0.0-1607.g94a53970.e 2.5 MB/s | 320 kB 00:00 2024-03-12T03:59:37.487 INFO:teuthology.orchestra.run.smithi136.stdout:(24/174): python3-rbd-19.0.0-1607.g94a53970.el9 5.0 MB/s | 298 kB 00:00 2024-03-12T03:59:37.513 INFO:teuthology.orchestra.run.smithi136.stdout:(25/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.7 MB/s | 99 kB 00:00 2024-03-12T03:59:37.537 INFO:teuthology.orchestra.run.smithi136.stdout:(26/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-12T03:59:37.604 INFO:teuthology.orchestra.run.smithi136.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-12T03:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 539 kB/s | 31 kB 00:00 2024-03-12T03:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 2.0 MB/s | 138 kB 00:00 2024-03-12T03:59:37.789 INFO:teuthology.orchestra.run.smithi136.stdout:(30/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 11 MB/s | 3.1 MB 00:00 2024-03-12T03:59:37.939 INFO:teuthology.orchestra.run.smithi136.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 8.5 MB/s | 1.8 MB 00:00 2024-03-12T03:59:38.014 INFO:teuthology.orchestra.run.smithi136.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 3.2 MB/s | 247 kB 00:00 2024-03-12T03:59:38.089 INFO:teuthology.orchestra.run.smithi136.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 658 kB/s | 49 kB 00:00 2024-03-12T03:59:38.165 INFO:teuthology.orchestra.run.smithi136.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 204 kB/s | 15 kB 00:00 2024-03-12T03:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout:(35/174): ceph-volume-19.0.0-1607.g94a53970.el9 3.4 MB/s | 264 kB 00:00 2024-03-12T03:59:38.382 INFO:teuthology.orchestra.run.smithi136.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-03-12T03:59:38.416 INFO:teuthology.orchestra.run.smithi136.stdout:(37/174): cephadm-19.0.0-1607.g94a53970.el9.noa 4.2 MB/s | 748 kB 00:00 2024-03-12T03:59:38.458 INFO:teuthology.orchestra.run.smithi136.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 242 kB/s | 18 kB 00:00 2024-03-12T03:59:38.633 INFO:teuthology.orchestra.run.smithi136.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 662 kB 00:00 2024-03-12T03:59:38.658 INFO:teuthology.orchestra.run.smithi136.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.1 MB/s | 920 kB 00:00 2024-03-12T03:59:38.683 INFO:teuthology.orchestra.run.smithi136.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-12T03:59:38.708 INFO:teuthology.orchestra.run.smithi136.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-12T03:59:38.725 INFO:teuthology.orchestra.run.smithi136.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 761 kB/s | 32 kB 00:00 2024-03-12T03:59:38.809 INFO:teuthology.orchestra.run.smithi136.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-12T03:59:38.825 INFO:teuthology.orchestra.run.smithi136.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 172 kB/s | 17 kB 00:00 2024-03-12T03:59:38.859 INFO:teuthology.orchestra.run.smithi136.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 303 kB/s | 10 kB 00:00 2024-03-12T03:59:38.885 INFO:teuthology.orchestra.run.smithi136.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-12T03:59:38.901 INFO:teuthology.orchestra.run.smithi136.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-12T03:59:38.935 INFO:teuthology.orchestra.run.smithi136.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-12T03:59:38.969 INFO:teuthology.orchestra.run.smithi136.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2024-03-12T03:59:38.986 INFO:teuthology.orchestra.run.smithi136.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 939 kB/s | 46 kB 00:00 2024-03-12T03:59:39.011 INFO:teuthology.orchestra.run.smithi136.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-12T03:59:39.036 INFO:teuthology.orchestra.run.smithi136.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-12T03:59:39.061 INFO:teuthology.orchestra.run.smithi136.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-12T03:59:39.076 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-12T03:59:39.078 INFO:teuthology.orchestra.run.smithi136.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 822 kB/s | 34 kB 00:00 2024-03-12T03:59:39.162 INFO:teuthology.orchestra.run.smithi136.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 47 MB/s | 4.7 MB 00:00 2024-03-12T03:59:39.196 INFO:teuthology.orchestra.run.smithi136.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 13 MB/s | 1.5 MB 00:00 2024-03-12T03:59:39.221 INFO:teuthology.orchestra.run.smithi136.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2024-03-12T03:59:39.246 INFO:teuthology.orchestra.run.smithi136.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-12T03:59:39.346 INFO:teuthology.orchestra.run.smithi136.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 46 MB/s | 5.7 MB 00:00 2024-03-12T03:59:39.388 INFO:teuthology.orchestra.run.smithi136.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-03-12T03:59:39.422 INFO:teuthology.orchestra.run.smithi136.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 78 kB 00:00 2024-03-12T03:59:39.523 INFO:teuthology.orchestra.run.smithi136.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 43 MB/s | 5.8 MB 00:00 2024-03-12T03:59:39.550 INFO:teuthology.orchestra.run.smithi136.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 199 kB/s | 24 kB 00:00 2024-03-12T03:59:39.581 INFO:teuthology.orchestra.run.smithi136.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 547 kB/s | 18 kB 00:00 2024-03-12T03:59:39.606 INFO:teuthology.orchestra.run.smithi136.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-12T03:59:39.631 INFO:teuthology.orchestra.run.smithi136.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-03-12T03:59:39.715 INFO:teuthology.orchestra.run.smithi136.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 4.8 MB 00:00 2024-03-12T03:59:39.740 INFO:teuthology.orchestra.run.smithi136.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 217 kB 00:00 2024-03-12T03:59:39.766 INFO:teuthology.orchestra.run.smithi136.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-12T03:59:39.791 INFO:teuthology.orchestra.run.smithi136.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-12T03:59:39.816 INFO:teuthology.orchestra.run.smithi136.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-12T03:59:39.842 INFO:teuthology.orchestra.run.smithi136.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-12T03:59:39.867 INFO:teuthology.orchestra.run.smithi136.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-12T03:59:39.892 INFO:teuthology.orchestra.run.smithi136.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-12T03:59:39.917 INFO:teuthology.orchestra.run.smithi136.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-03-12T03:59:39.942 INFO:teuthology.orchestra.run.smithi136.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-03-12T03:59:40.176 INFO:teuthology.orchestra.run.smithi136.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 61 MB/s | 16 MB 00:00 2024-03-12T03:59:40.193 INFO:teuthology.orchestra.run.smithi136.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 39 kB 00:00 2024-03-12T03:59:40.227 INFO:teuthology.orchestra.run.smithi136.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 821 kB/s | 41 kB 00:00 2024-03-12T03:59:40.260 INFO:teuthology.orchestra.run.smithi136.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-12T03:59:40.277 INFO:teuthology.orchestra.run.smithi136.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-12T03:59:40.294 INFO:teuthology.orchestra.run.smithi136.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-12T03:59:40.311 INFO:teuthology.orchestra.run.smithi136.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-12T03:59:40.328 INFO:teuthology.orchestra.run.smithi136.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-03-12T03:59:40.353 INFO:teuthology.orchestra.run.smithi136.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-12T03:59:40.395 INFO:teuthology.orchestra.run.smithi136.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-12T03:59:40.420 INFO:teuthology.orchestra.run.smithi136.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-12T03:59:40.446 INFO:teuthology.orchestra.run.smithi136.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-12T03:59:40.463 INFO:teuthology.orchestra.run.smithi136.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-03-12T03:59:40.588 INFO:teuthology.orchestra.run.smithi136.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-12T03:59:40.647 INFO:teuthology.orchestra.run.smithi136.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-03-12T03:59:40.689 INFO:teuthology.orchestra.run.smithi136.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-03-12T03:59:40.714 INFO:teuthology.orchestra.run.smithi136.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-12T03:59:41.215 INFO:teuthology.orchestra.run.smithi136.stdout:(95/174): ceph-test-19.0.0-1607.g94a53970.el9.x 9.2 MB/s | 47 MB 00:05 2024-03-12T03:59:41.715 INFO:teuthology.orchestra.run.smithi136.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 21 kB/s | 21 kB 00:01 2024-03-12T03:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 24 kB/s | 25 kB 00:01 2024-03-12T03:59:42.698 INFO:teuthology.orchestra.run.smithi136.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 250 kB/s | 245 kB 00:00 2024-03-12T03:59:42.807 INFO:teuthology.orchestra.run.smithi136.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 240 kB/s | 248 kB 00:01 2024-03-12T03:59:43.474 INFO:teuthology.orchestra.run.smithi136.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 52 kB/s | 35 kB 00:00 2024-03-12T03:59:43.657 INFO:teuthology.orchestra.run.smithi136.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 226 kB/s | 216 kB 00:00 2024-03-12T03:59:44.026 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-12T03:59:44.182 INFO:teuthology.orchestra.run.smithi136.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 63 kB/s | 44 kB 00:00 2024-03-12T03:59:44.224 INFO:teuthology.orchestra.run.smithi136.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 541 kB/s | 22 kB 00:00 2024-03-12T03:59:44.358 INFO:teuthology.orchestra.run.smithi136.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-03-12T03:59:44.466 INFO:teuthology.orchestra.run.smithi136.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 116 kB/s | 94 kB 00:00 2024-03-12T03:59:44.958 INFO:teuthology.orchestra.run.smithi136.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 914 kB/s | 548 kB 00:00 2024-03-12T03:59:44.983 INFO:teuthology.orchestra.run.smithi136.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-03-12T03:59:45.225 INFO:teuthology.orchestra.run.smithi136.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-12T03:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-03-12T03:59:45.842 INFO:teuthology.orchestra.run.smithi136.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-03-12T03:59:46.167 INFO:teuthology.orchestra.run.smithi136.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-03-12T03:59:46.193 INFO:teuthology.orchestra.run.smithi136.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 192 kB/s | 67 kB 00:00 2024-03-12T03:59:46.582 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T03:59:46.583 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T03:59:46.593 INFO:teuthology.orchestra.run.smithi136.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-03-12T03:59:46.597 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T03:59:46.598 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T03:59:46.599 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T03:59:46.600 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T03:59:46.602 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T03:59:46.605 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T03:59:46.606 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T03:59:46.684 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-03-12T03:59:46.702 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-12T03:59:46.703 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-12T03:59:46.704 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T03:59:46.705 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T03:59:46.706 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T03:59:46.707 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T03:59:46.708 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T03:59:46.709 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-12T03:59:46.710 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T03:59:46.711 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T03:59:46.712 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T03:59:46.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T03:59:46.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-03-12T03:59:46.715 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-03-12T03:59:46.716 INFO:teuthology.orchestra.run.smithi119.stdout:Install 150 Packages 2024-03-12T03:59:46.716 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 24 Packages 2024-03-12T03:59:46.716 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T03:59:46.716 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 223 M 2024-03-12T03:59:46.716 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-03-12T03:59:46.860 INFO:teuthology.orchestra.run.smithi136.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-03-12T03:59:47.210 INFO:teuthology.orchestra.run.smithi136.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-03-12T03:59:47.260 INFO:teuthology.orchestra.run.smithi136.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 821 kB/s | 548 kB 00:00 2024-03-12T03:59:47.486 INFO:teuthology.orchestra.run.smithi119.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-03-12T03:59:47.502 INFO:teuthology.orchestra.run.smithi136.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 110 kB/s | 32 kB 00:00 2024-03-12T03:59:47.527 INFO:teuthology.orchestra.run.smithi136.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-03-12T03:59:47.761 INFO:teuthology.orchestra.run.smithi136.stdout:(119/174): openssl-devel-3.0.7-27.el9.x86_64.rp 646 kB/s | 4.1 MB 00:06 2024-03-12T03:59:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 338 kB/s | 172 kB 00:00 2024-03-12T03:59:48.028 INFO:teuthology.orchestra.run.smithi119.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 1.6 MB/s | 886 kB 00:00 2024-03-12T03:59:48.036 INFO:teuthology.orchestra.run.smithi136.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-03-12T03:59:48.128 INFO:teuthology.orchestra.run.smithi119.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 1.4 MB/s | 145 kB 00:00 2024-03-12T03:59:48.303 INFO:teuthology.orchestra.run.smithi136.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 121 kB/s | 35 kB 00:00 2024-03-12T03:59:48.354 INFO:teuthology.orchestra.run.smithi136.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 512 kB/s | 303 kB 00:00 2024-03-12T03:59:48.471 INFO:teuthology.orchestra.run.smithi136.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 467 kB/s | 203 kB 00:00 2024-03-12T03:59:48.646 INFO:teuthology.orchestra.run.smithi136.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 61 kB/s | 11 kB 00:00 2024-03-12T03:59:48.770 INFO:teuthology.orchestra.run.smithi119.stdout:(4/174): ceph-common-19.0.0-1607.g94a53970.el9. 13 MB/s | 20 MB 00:01 2024-03-12T03:59:48.796 INFO:teuthology.orchestra.run.smithi136.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 327 kB/s | 144 kB 00:00 2024-03-12T03:59:48.871 INFO:teuthology.orchestra.run.smithi119.stdout:(5/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 3.3 MB/s | 5.3 MB 00:01 2024-03-12T03:59:48.913 INFO:teuthology.orchestra.run.smithi136.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-03-12T03:59:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:(6/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-03-12T03:59:48.996 INFO:teuthology.orchestra.run.smithi119.stdout:(7/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 2.4 MB/s | 2.1 MB 00:00 2024-03-12T03:59:49.064 INFO:teuthology.orchestra.run.smithi136.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2024-03-12T03:59:49.155 INFO:teuthology.orchestra.run.smithi136.stdout:(129/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-03-12T03:59:49.181 INFO:teuthology.orchestra.run.smithi136.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-03-12T03:59:49.339 INFO:teuthology.orchestra.run.smithi136.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 71 kB/s | 20 kB 00:00 2024-03-12T03:59:49.523 INFO:teuthology.orchestra.run.smithi136.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-03-12T03:59:49.690 INFO:teuthology.orchestra.run.smithi136.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-03-12T03:59:49.755 INFO:teuthology.orchestra.run.smithi119.stdout:(8/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-03-12T03:59:49.840 INFO:teuthology.orchestra.run.smithi136.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-03-12T03:59:49.854 INFO:teuthology.orchestra.run.smithi119.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 252 kB/s | 25 kB 00:00 2024-03-12T03:59:49.857 INFO:teuthology.orchestra.run.smithi136.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 173 kB/s | 58 kB 00:00 2024-03-12T03:59:50.099 INFO:teuthology.orchestra.run.smithi136.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 63 kB/s | 16 kB 00:00 2024-03-12T03:59:50.257 INFO:teuthology.orchestra.run.smithi136.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 480 kB/s | 272 kB 00:00 2024-03-12T03:59:50.332 INFO:teuthology.orchestra.run.smithi136.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 189 kB/s | 90 kB 00:00 2024-03-12T03:59:50.441 INFO:teuthology.orchestra.run.smithi136.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 82 kB/s | 28 kB 00:00 2024-03-12T03:59:50.516 INFO:teuthology.orchestra.run.smithi136.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-03-12T03:59:50.766 INFO:teuthology.orchestra.run.smithi136.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 435 kB/s | 188 kB 00:00 2024-03-12T03:59:50.783 INFO:teuthology.orchestra.run.smithi136.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-03-12T03:59:50.800 INFO:teuthology.orchestra.run.smithi136.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 126 kB/s | 36 kB 00:00 2024-03-12T03:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 153 kB/s | 51 kB 00:00 2024-03-12T03:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 256 kB/s | 90 kB 00:00 2024-03-12T03:59:51.272 INFO:teuthology.orchestra.run.smithi119.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 4.3 MB/s | 9.8 MB 00:02 2024-03-12T03:59:51.317 INFO:teuthology.orchestra.run.smithi136.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 444 kB/s | 230 kB 00:00 2024-03-12T03:59:51.418 INFO:teuthology.orchestra.run.smithi136.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-03-12T03:59:51.438 INFO:teuthology.orchestra.run.smithi119.stdout:(11/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 6.7 MB/s | 17 MB 00:02 2024-03-12T03:59:51.464 INFO:teuthology.orchestra.run.smithi119.stdout:(12/174): cephfs-mirror-19.0.0-1607.g94a53970.e 1.1 MB/s | 217 kB 00:00 2024-03-12T03:59:51.506 INFO:teuthology.orchestra.run.smithi119.stdout:(13/174): cephfs-shell-19.0.0-1607.g94a53970.el 406 kB/s | 27 kB 00:00 2024-03-12T03:59:51.539 INFO:teuthology.orchestra.run.smithi119.stdout:(14/174): libcephfs-devel-19.0.0-1607.g94a53970 414 kB/s | 31 kB 00:00 2024-03-12T03:59:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.6 MB/s | 163 kB 00:00 2024-03-12T03:59:51.685 INFO:teuthology.orchestra.run.smithi136.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-03-12T03:59:51.723 INFO:teuthology.orchestra.run.smithi119.stdout:(16/174): libcephfs2-19.0.0-1607.g94a53970.el9. 3.3 MB/s | 742 kB 00:00 2024-03-12T03:59:51.750 INFO:teuthology.orchestra.run.smithi119.stdout:(17/174): librados-devel-19.0.0-1607.g94a53970. 1.1 MB/s | 127 kB 00:00 2024-03-12T03:59:51.810 INFO:teuthology.orchestra.run.smithi136.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 389 kB/s | 192 kB 00:00 2024-03-12T03:59:51.865 INFO:teuthology.orchestra.run.smithi119.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 3.4 MB/s | 498 kB 00:00 2024-03-12T03:59:51.949 INFO:teuthology.orchestra.run.smithi119.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 540 kB/s | 45 kB 00:00 2024-03-12T03:59:52.074 INFO:teuthology.orchestra.run.smithi119.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.0 MB/s | 131 kB 00:00 2024-03-12T03:59:52.168 INFO:teuthology.orchestra.run.smithi136.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-03-12T03:59:52.210 INFO:teuthology.orchestra.run.smithi136.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-12T03:59:52.735 INFO:teuthology.orchestra.run.smithi136.stdout:(152/174): librados2-19.0.0-1607.g94a53970.el9. 3.2 MB/s | 3.4 MB 00:01 2024-03-12T03:59:52.785 INFO:teuthology.orchestra.run.smithi136.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 4.0 MB/s | 2.3 MB 00:00 2024-03-12T03:59:52.827 INFO:teuthology.orchestra.run.smithi136.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-12T03:59:52.849 INFO:teuthology.orchestra.run.smithi119.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 208 kB/s | 161 kB 00:00 2024-03-12T03:59:52.903 INFO:teuthology.orchestra.run.smithi136.stdout:(155/174): librbd1-19.0.0-1607.g94a53970.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-03-12T03:59:52.936 INFO:teuthology.orchestra.run.smithi136.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-03-12T03:59:52.958 INFO:teuthology.orchestra.run.smithi119.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 2.9 MB/s | 320 kB 00:00 2024-03-12T03:59:52.978 INFO:teuthology.orchestra.run.smithi136.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 878 kB/s | 36 kB 00:00 2024-03-12T03:59:53.020 INFO:teuthology.orchestra.run.smithi136.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-12T03:59:53.046 INFO:teuthology.orchestra.run.smithi136.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 500 kB/s | 71 kB 00:00 2024-03-12T03:59:53.075 INFO:teuthology.orchestra.run.smithi119.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 2.5 MB/s | 298 kB 00:00 2024-03-12T03:59:53.087 INFO:teuthology.orchestra.run.smithi136.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-03-12T03:59:53.113 INFO:teuthology.orchestra.run.smithi136.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 345 kB/s | 23 kB 00:00 2024-03-12T03:59:53.138 INFO:teuthology.orchestra.run.smithi136.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-12T03:59:53.142 INFO:teuthology.orchestra.run.smithi119.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.4 MB/s | 99 kB 00:00 2024-03-12T03:59:53.180 INFO:teuthology.orchestra.run.smithi136.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-03-12T03:59:53.205 INFO:teuthology.orchestra.run.smithi136.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 155 kB 00:00 2024-03-12T03:59:53.217 INFO:teuthology.orchestra.run.smithi119.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-12T03:59:53.231 INFO:teuthology.orchestra.run.smithi136.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-12T03:59:53.256 INFO:teuthology.orchestra.run.smithi136.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-03-12T03:59:53.298 INFO:teuthology.orchestra.run.smithi136.stdout:(167/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.2 MB/s | 1.8 MB 00:00 2024-03-12T03:59:53.448 INFO:teuthology.orchestra.run.smithi136.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 1.2 MB 00:00 2024-03-12T03:59:53.524 INFO:teuthology.orchestra.run.smithi136.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.1 MB/s | 465 kB 00:00 2024-03-12T03:59:53.565 INFO:teuthology.orchestra.run.smithi136.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.0 MB/s | 2.2 MB 00:00 2024-03-12T03:59:53.591 INFO:teuthology.orchestra.run.smithi136.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 466 kB/s | 31 kB 00:00 2024-03-12T03:59:53.593 INFO:teuthology.orchestra.run.smithi119.stdout:(26/174): librgw2-19.0.0-1607.g94a53970.el9.x86 2.8 MB/s | 5.1 MB 00:01 2024-03-12T03:59:53.684 INFO:teuthology.orchestra.run.smithi119.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 1.8 MB/s | 171 kB 00:00 2024-03-12T03:59:53.785 INFO:teuthology.orchestra.run.smithi119.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 314 kB/s | 31 kB 00:00 2024-03-12T03:59:53.841 INFO:teuthology.orchestra.run.smithi136.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.9 MB/s | 1.1 MB 00:00 2024-03-12T03:59:53.885 INFO:teuthology.orchestra.run.smithi119.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 1.4 MB/s | 138 kB 00:00 2024-03-12T03:59:53.958 INFO:teuthology.orchestra.run.smithi136.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.5 MB/s | 2.3 MB 00:00 2024-03-12T03:59:54.002 INFO:teuthology.orchestra.run.smithi119.stdout:(30/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 3.9 MB/s | 3.1 MB 00:00 2024-03-12T03:59:54.349 INFO:teuthology.orchestra.run.smithi136.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 98 kB/s | 74 kB 00:00 2024-03-12T03:59:54.354 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-03-12T03:59:54.354 INFO:teuthology.orchestra.run.smithi136.stdout:Total 11 MB/s | 223 MB 00:20 2024-03-12T03:59:54.669 INFO:teuthology.orchestra.run.smithi119.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 2.3 MB/s | 1.8 MB 00:00 2024-03-12T03:59:54.815 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-03-12T03:59:54.852 INFO:teuthology.orchestra.run.smithi119.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 1.3 MB/s | 247 kB 00:00 2024-03-12T03:59:54.953 INFO:teuthology.orchestra.run.smithi119.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 493 kB/s | 49 kB 00:00 2024-03-12T03:59:55.036 INFO:teuthology.orchestra.run.smithi119.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 183 kB/s | 15 kB 00:00 2024-03-12T03:59:55.130 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-03-12T03:59:55.130 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-03-12T03:59:55.195 INFO:teuthology.orchestra.run.smithi119.stdout:(35/174): ceph-volume-19.0.0-1607.g94a53970.el9 1.6 MB/s | 264 kB 00:00 2024-03-12T03:59:55.545 INFO:teuthology.orchestra.run.smithi119.stdout:(36/174): cephadm-19.0.0-1607.g94a53970.el9.noa 2.1 MB/s | 748 kB 00:00 2024-03-12T03:59:55.646 INFO:teuthology.orchestra.run.smithi119.stdout:(37/174): cephfs-top-19.0.0-1607.g94a53970.el9. 182 kB/s | 18 kB 00:00 2024-03-12T03:59:55.821 INFO:teuthology.orchestra.run.smithi119.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-03-12T03:59:56.038 INFO:teuthology.orchestra.run.smithi119.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2024-03-12T03:59:56.130 INFO:teuthology.orchestra.run.smithi119.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 920 kB 00:00 2024-03-12T03:59:56.155 INFO:teuthology.orchestra.run.smithi119.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 908 kB/s | 106 kB 00:00 2024-03-12T03:59:56.189 INFO:teuthology.orchestra.run.smithi119.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-12T03:59:56.214 INFO:teuthology.orchestra.run.smithi119.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 542 kB/s | 32 kB 00:00 2024-03-12T03:59:56.381 INFO:teuthology.orchestra.run.smithi119.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 103 kB/s | 17 kB 00:00 2024-03-12T03:59:56.440 INFO:teuthology.orchestra.run.smithi119.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-12T03:59:56.465 INFO:teuthology.orchestra.run.smithi119.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 10 kB 00:00 2024-03-12T03:59:56.490 INFO:teuthology.orchestra.run.smithi119.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-12T03:59:56.515 INFO:teuthology.orchestra.run.smithi119.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-12T03:59:56.566 INFO:teuthology.orchestra.run.smithi119.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-12T03:59:56.591 INFO:teuthology.orchestra.run.smithi119.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-12T03:59:56.616 INFO:teuthology.orchestra.run.smithi119.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 46 kB 00:00 2024-03-12T03:59:56.642 INFO:teuthology.orchestra.run.smithi119.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-12T03:59:56.667 INFO:teuthology.orchestra.run.smithi119.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2024-03-12T03:59:56.692 INFO:teuthology.orchestra.run.smithi119.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-12T03:59:56.709 INFO:teuthology.orchestra.run.smithi119.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 825 kB/s | 34 kB 00:00 2024-03-12T03:59:56.885 INFO:teuthology.orchestra.run.smithi119.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-12T03:59:56.926 INFO:teuthology.orchestra.run.smithi119.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 7.0 MB/s | 1.5 MB 00:00 2024-03-12T03:59:56.968 INFO:teuthology.orchestra.run.smithi119.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 9.9 MB/s | 811 kB 00:00 2024-03-12T03:59:56.985 INFO:teuthology.orchestra.run.smithi119.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-12T03:59:57.119 INFO:teuthology.orchestra.run.smithi119.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-03-12T03:59:57.152 INFO:teuthology.orchestra.run.smithi119.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-03-12T03:59:57.178 INFO:teuthology.orchestra.run.smithi119.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-12T03:59:57.287 INFO:teuthology.orchestra.run.smithi119.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 43 MB/s | 5.8 MB 00:00 2024-03-12T03:59:57.312 INFO:teuthology.orchestra.run.smithi119.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 187 kB/s | 24 kB 00:00 2024-03-12T03:59:57.337 INFO:teuthology.orchestra.run.smithi119.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-03-12T03:59:57.363 INFO:teuthology.orchestra.run.smithi119.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-12T03:59:57.388 INFO:teuthology.orchestra.run.smithi119.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-12T03:59:57.412 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-03-12T03:59:57.412 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-03-12T03:59:57.488 INFO:teuthology.orchestra.run.smithi119.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-12T03:59:57.513 INFO:teuthology.orchestra.run.smithi119.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-12T03:59:57.539 INFO:teuthology.orchestra.run.smithi119.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-12T03:59:57.564 INFO:teuthology.orchestra.run.smithi119.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-12T03:59:57.589 INFO:teuthology.orchestra.run.smithi119.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-03-12T03:59:57.614 INFO:teuthology.orchestra.run.smithi119.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-12T03:59:57.640 INFO:teuthology.orchestra.run.smithi119.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-12T03:59:57.665 INFO:teuthology.orchestra.run.smithi119.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-12T03:59:57.690 INFO:teuthology.orchestra.run.smithi119.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 953 kB/s | 47 kB 00:00 2024-03-12T03:59:57.715 INFO:teuthology.orchestra.run.smithi119.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-03-12T03:59:57.999 INFO:teuthology.orchestra.run.smithi119.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2024-03-12T03:59:58.024 INFO:teuthology.orchestra.run.smithi119.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 126 kB/s | 39 kB 00:00 2024-03-12T03:59:58.050 INFO:teuthology.orchestra.run.smithi119.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 819 kB/s | 41 kB 00:00 2024-03-12T03:59:58.075 INFO:teuthology.orchestra.run.smithi119.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-03-12T03:59:58.100 INFO:teuthology.orchestra.run.smithi119.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-12T03:59:58.134 INFO:teuthology.orchestra.run.smithi119.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-03-12T03:59:58.160 INFO:teuthology.orchestra.run.smithi119.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-03-12T03:59:58.185 INFO:teuthology.orchestra.run.smithi119.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-12T03:59:58.210 INFO:teuthology.orchestra.run.smithi119.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-12T03:59:58.244 INFO:teuthology.orchestra.run.smithi119.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-12T03:59:58.269 INFO:teuthology.orchestra.run.smithi119.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-12T03:59:58.294 INFO:teuthology.orchestra.run.smithi119.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-12T03:59:58.320 INFO:teuthology.orchestra.run.smithi119.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-12T03:59:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 222 kB/s | 187 kB 00:00 2024-03-12T03:59:59.770 INFO:teuthology.orchestra.run.smithi119.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 308 kB/s | 195 kB 00:00 2024-03-12T04:00:00.012 INFO:teuthology.orchestra.run.smithi119.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 516 kB/s | 125 kB 00:00 2024-03-12T04:00:00.146 INFO:teuthology.orchestra.run.smithi119.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-03-12T04:00:00.346 INFO:teuthology.orchestra.run.smithi119.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 127 kB/s | 25 kB 00:00 2024-03-12T04:00:02.079 INFO:teuthology.orchestra.run.smithi119.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 214 kB/s | 803 kB 00:03 2024-03-12T04:00:02.471 INFO:teuthology.orchestra.run.smithi119.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 627 kB/s | 245 kB 00:00 2024-03-12T04:00:02.771 INFO:teuthology.orchestra.run.smithi119.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 826 kB/s | 248 kB 00:00 2024-03-12T04:00:02.972 INFO:teuthology.orchestra.run.smithi119.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.1 MB/s | 216 kB 00:00 2024-03-12T04:00:03.039 INFO:teuthology.orchestra.run.smithi119.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 524 kB/s | 35 kB 00:00 2024-03-12T04:00:03.106 INFO:teuthology.orchestra.run.smithi119.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 668 kB/s | 44 kB 00:00 2024-03-12T04:00:03.198 INFO:teuthology.orchestra.run.smithi119.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-12T04:00:03.473 INFO:teuthology.orchestra.run.smithi119.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-03-12T04:00:04.082 INFO:teuthology.orchestra.run.smithi119.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-03-12T04:00:04.265 INFO:teuthology.orchestra.run.smithi119.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 1.1 MB/s | 4.1 MB 00:03 2024-03-12T04:00:04.409 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-03-12T04:00:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-12T04:00:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-12T04:00:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:00:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:00:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-12T04:00:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-12T04:00:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-12T04:00:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-12T04:00:04.932 INFO:teuthology.orchestra.run.smithi119.stdout:(106/174): ceph-test-19.0.0-1607.g94a53970.el9. 3.1 MB/s | 47 MB 00:15 2024-03-12T04:00:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-12T04:00:04.957 INFO:teuthology.orchestra.run.smithi119.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 626 kB/s | 548 kB 00:00 2024-03-12T04:00:05.199 INFO:teuthology.orchestra.run.smithi119.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-12T04:00:05.225 INFO:teuthology.orchestra.run.smithi119.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 322 kB/s | 309 kB 00:00 2024-03-12T04:00:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-12T04:00:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:00:05.467 INFO:teuthology.orchestra.run.smithi119.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-03-12T04:00:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:00:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:00:05.567 INFO:teuthology.orchestra.run.smithi119.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-03-12T04:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T04:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-12T04:00:05.842 INFO:teuthology.orchestra.run.smithi119.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 179 kB/s | 67 kB 00:00 2024-03-12T04:00:05.882 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-12T04:00:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-12T04:00:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:00:06.067 INFO:teuthology.orchestra.run.smithi119.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-03-12T04:00:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:00:06.092 INFO:teuthology.orchestra.run.smithi119.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 287 kB/s | 151 kB 00:00 2024-03-12T04:00:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:00:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:00:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-12T04:00:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:00:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:00:06.435 INFO:teuthology.orchestra.run.smithi119.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-03-12T04:00:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:00:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:00:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-12T04:00:06.510 INFO:teuthology.orchestra.run.smithi119.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-03-12T04:00:06.660 INFO:teuthology.orchestra.run.smithi119.stdout:(117/174): python3-asyncssh-2.13.2-4.el9.noarch 924 kB/s | 548 kB 00:00 2024-03-12T04:00:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-12T04:00:06.719 INFO:teuthology.orchestra.run.smithi119.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 113 kB/s | 32 kB 00:00 2024-03-12T04:00:06.777 INFO:teuthology.orchestra.run.smithi119.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-03-12T04:00:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:00:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:00:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:00:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:00:07.102 INFO:teuthology.orchestra.run.smithi119.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 389 kB/s | 172 kB 00:00 2024-03-12T04:00:07.228 INFO:teuthology.orchestra.run.smithi119.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 704 kB/s | 358 kB 00:00 2024-03-12T04:00:07.250 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:00:07.295 INFO:teuthology.orchestra.run.smithi119.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 586 kB/s | 303 kB 00:00 2024-03-12T04:00:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-12T04:00:07.361 INFO:teuthology.orchestra.run.smithi119.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 136 kB/s | 35 kB 00:00 2024-03-12T04:00:07.432 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-12T04:00:07.463 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-12T04:00:07.490 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-12T04:00:07.525 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-12T04:00:07.598 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-12T04:00:07.652 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-12T04:00:07.691 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-12T04:00:07.729 INFO:teuthology.orchestra.run.smithi119.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 405 kB/s | 203 kB 00:00 2024-03-12T04:00:07.746 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-12T04:00:07.782 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-12T04:00:07.795 INFO:teuthology.orchestra.run.smithi119.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-03-12T04:00:07.825 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-12T04:00:07.859 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-12T04:00:07.887 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-12T04:00:07.912 INFO:teuthology.orchestra.run.smithi119.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-03-12T04:00:07.963 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-12T04:00:07.999 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-12T04:00:08.042 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-12T04:00:08.063 INFO:teuthology.orchestra.run.smithi119.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-03-12T04:00:08.071 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-12T04:00:08.131 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-12T04:00:08.146 INFO:teuthology.orchestra.run.smithi119.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-03-12T04:00:08.172 INFO:teuthology.orchestra.run.smithi119.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 89 kB/s | 23 kB 00:00 2024-03-12T04:00:08.178 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-12T04:00:08.250 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-12T04:00:08.330 INFO:teuthology.orchestra.run.smithi119.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-03-12T04:00:08.382 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-12T04:00:08.422 INFO:teuthology.orchestra.run.smithi119.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 71 kB/s | 20 kB 00:00 2024-03-12T04:00:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-12T04:00:08.689 INFO:teuthology.orchestra.run.smithi119.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 129 kB/s | 46 kB 00:00 2024-03-12T04:00:08.839 INFO:teuthology.orchestra.run.smithi119.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-03-12T04:00:08.865 INFO:teuthology.orchestra.run.smithi119.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 178 kB/s | 79 kB 00:00 2024-03-12T04:00:08.945 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-12T04:00:09.025 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-12T04:00:09.031 INFO:teuthology.orchestra.run.smithi119.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 168 kB/s | 58 kB 00:00 2024-03-12T04:00:09.113 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-12T04:00:09.149 INFO:teuthology.orchestra.run.smithi119.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 58 kB/s | 16 kB 00:00 2024-03-12T04:00:09.177 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:00:09.185 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:00:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-12T04:00:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-12T04:00:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-12T04:00:09.345 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-12T04:00:09.357 INFO:teuthology.orchestra.run.smithi119.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 526 kB/s | 272 kB 00:00 2024-03-12T04:00:09.416 INFO:teuthology.orchestra.run.smithi119.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-03-12T04:00:09.441 INFO:teuthology.orchestra.run.smithi119.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 220 kB/s | 90 kB 00:00 2024-03-12T04:00:09.643 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-12T04:00:09.675 INFO:teuthology.orchestra.run.smithi119.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 97 kB/s | 31 kB 00:00 2024-03-12T04:00:09.784 INFO:teuthology.orchestra.run.smithi119.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-03-12T04:00:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-12T04:00:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-12T04:00:09.917 INFO:teuthology.orchestra.run.smithi119.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 376 kB/s | 188 kB 00:00 2024-03-12T04:00:09.942 INFO:teuthology.orchestra.run.smithi119.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-03-12T04:00:09.958 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-12T04:00:10.104 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-12T04:00:10.134 INFO:teuthology.orchestra.run.smithi119.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-03-12T04:00:10.153 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-12T04:00:10.207 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-12T04:00:10.280 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-12T04:00:10.284 INFO:teuthology.orchestra.run.smithi119.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-03-12T04:00:10.312 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-12T04:00:10.349 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-12T04:00:10.395 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-12T04:00:10.426 INFO:teuthology.orchestra.run.smithi119.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-03-12T04:00:10.431 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-12T04:00:10.474 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-12T04:00:10.517 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-12T04:00:10.535 INFO:teuthology.orchestra.run.smithi119.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 80 kB/s | 20 kB 00:00 2024-03-12T04:00:10.649 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-12T04:00:10.706 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-12T04:00:10.727 INFO:teuthology.orchestra.run.smithi119.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 721 kB/s | 427 kB 00:00 2024-03-12T04:00:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-12T04:00:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-12T04:00:10.852 INFO:teuthology.orchestra.run.smithi119.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-03-12T04:00:10.872 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-12T04:00:10.912 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-12T04:00:10.952 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-12T04:00:10.978 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-12T04:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-12T04:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-12T04:00:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-12T04:00:11.286 INFO:teuthology.orchestra.run.smithi119.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-03-12T04:00:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-12T04:00:11.328 INFO:teuthology.orchestra.run.smithi119.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-12T04:00:11.344 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-12T04:00:11.393 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-12T04:00:11.449 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-12T04:00:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-12T04:00:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-12T04:00:11.791 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-12T04:00:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-12T04:00:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-12T04:00:11.893 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-12T04:00:11.930 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-12T04:00:11.964 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-12T04:00:12.003 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-12T04:00:12.035 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-12T04:00:12.065 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-12T04:00:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-12T04:00:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-12T04:00:12.200 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-12T04:00:12.241 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-12T04:00:12.277 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-12T04:00:12.365 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-12T04:00:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-12T04:00:12.503 INFO:teuthology.orchestra.run.smithi119.stdout:(152/174): librados2-19.0.0-1607.g94a53970.el9. 1.9 MB/s | 3.4 MB 00:01 2024-03-12T04:00:12.515 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-12T04:00:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-12T04:00:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-12T04:00:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-12T04:00:12.757 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-12T04:00:13.245 INFO:teuthology.orchestra.run.smithi119.stdout:(153/174): librbd1-19.0.0-1607.g94a53970.el9.x8 1.3 MB/s | 3.1 MB 00:02 2024-03-12T04:00:13.473 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-12T04:00:13.521 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-12T04:00:13.555 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-12T04:00:13.603 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-12T04:00:13.682 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-12T04:00:13.714 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-12T04:00:13.753 INFO:teuthology.orchestra.run.smithi119.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 83 kB/s | 42 kB 00:00 2024-03-12T04:00:13.896 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-12T04:00:13.926 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-12T04:00:14.003 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-12T04:00:14.087 INFO:teuthology.orchestra.run.smithi119.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 745 kB/s | 248 kB 00:00 2024-03-12T04:00:14.295 INFO:teuthology.orchestra.run.smithi119.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 342 kB/s | 71 kB 00:00 2024-03-12T04:00:14.362 INFO:teuthology.orchestra.run.smithi119.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 546 kB/s | 36 kB 00:00 2024-03-12T04:00:14.538 INFO:teuthology.orchestra.run.smithi119.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.1 MB/s | 192 kB 00:00 2024-03-12T04:00:14.628 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-12T04:00:14.731 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-12T04:00:15.254 INFO:teuthology.orchestra.run.smithi119.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 357 kB/s | 256 kB 00:00 2024-03-12T04:00:15.397 INFO:teuthology.orchestra.run.smithi119.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 163 kB/s | 23 kB 00:00 2024-03-12T04:00:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 260 kB/s | 108 kB 00:00 2024-03-12T04:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-12T04:00:16.053 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-12T04:00:16.084 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-12T04:00:16.107 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-12T04:00:16.134 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-12T04:00:16.205 INFO:teuthology.orchestra.run.smithi119.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 395 kB/s | 155 kB 00:00 2024-03-12T04:00:16.430 INFO:teuthology.orchestra.run.smithi119.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 603 kB/s | 136 kB 00:00 2024-03-12T04:00:16.443 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-12T04:00:16.473 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:00:16.554 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:00:16.581 INFO:teuthology.orchestra.run.smithi119.stdout:(164/174): NetworkManager-1.46.0-1.el9.x86_64.r 446 kB/s | 2.3 MB 00:05 2024-03-12T04:00:16.586 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-12T04:00:16.606 INFO:teuthology.orchestra.run.smithi119.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 364 kB/s | 64 kB 00:00 2024-03-12T04:00:16.623 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-12T04:00:16.673 INFO:teuthology.orchestra.run.smithi119.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 312 kB/s | 29 kB 00:00 2024-03-12T04:00:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-12T04:00:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:00:17.132 INFO:teuthology.orchestra.run.smithi119.stdout:(167/174): NetworkManager-libnm-1.46.0-1.el9.x8 402 kB/s | 1.8 MB 00:04 2024-03-12T04:00:17.180 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:00:17.203 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-12T04:00:19.369 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:00:19.408 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:00:19.582 INFO:teuthology.orchestra.run.smithi119.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 416 kB/s | 1.2 MB 00:02 2024-03-12T04:00:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:00:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:00:20.088 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:00:20.088 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T04:00:20.089 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T04:00:20.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:20.137 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:00:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 627 kB/s | 465 kB 00:00 2024-03-12T04:00:20.424 INFO:teuthology.orchestra.run.smithi119.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 312 kB/s | 31 kB 00:00 2024-03-12T04:00:21.116 INFO:teuthology.orchestra.run.smithi119.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 497 kB/s | 2.2 MB 00:04 2024-03-12T04:00:21.308 INFO:teuthology.orchestra.run.smithi119.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 388 kB/s | 74 kB 00:00 2024-03-12T04:00:22.707 INFO:teuthology.orchestra.run.smithi119.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 481 kB/s | 1.1 MB 00:02 2024-03-12T04:00:23.883 INFO:teuthology.orchestra.run.smithi119.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 346 kB/s | 2.3 MB 00:06 2024-03-12T04:00:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-03-12T04:00:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:Total 6.0 MB/s | 223 MB 00:37 2024-03-12T04:00:24.347 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-03-12T04:00:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-03-12T04:00:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-03-12T04:00:26.648 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-03-12T04:00:26.648 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-03-12T04:00:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:00:32.166 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-03-12T04:00:32.166 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-03-12T04:00:32.167 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-03-12T04:00:32.167 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-03-12T04:00:32.167 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-03-12T04:00:32.167 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-03-12T04:00:32.167 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-03-12T04:00:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:00:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:00:32.350 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:32.350 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T04:00:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:33.579 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-03-12T04:00:33.662 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-12T04:00:33.696 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-12T04:00:33.734 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:00:33.743 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:00:33.788 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-12T04:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:00:33.846 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-12T04:00:33.865 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:00:33.865 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:33.865 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T04:00:33.865 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:00:33.865 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:00:33.865 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:33.879 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-12T04:00:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-12T04:00:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:00:34.095 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:00:34.096 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:34.096 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T04:00:34.096 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:00:34.096 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:00:34.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:34.292 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-12T04:00:34.348 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:00:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:00:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:00:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T04:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:34.637 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:00:34.643 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-12T04:00:34.674 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T04:00:34.719 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-12T04:00:34.907 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-12T04:00:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-12T04:00:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:00:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:00:35.050 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:00:35.081 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-12T04:00:35.081 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:00:35.105 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:00:35.141 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-12T04:00:35.164 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:00:35.170 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:00:35.210 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-12T04:00:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:00:35.247 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:00:35.394 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:00:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T04:00:35.475 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:00:35.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:00:35.545 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-12T04:00:35.819 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-12T04:00:36.085 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:00:36.145 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:00:36.275 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:00:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:00:36.390 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T04:00:36.416 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T04:00:36.515 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T04:00:36.536 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:00:36.643 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-12T04:00:36.669 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-12T04:00:36.702 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T04:00:36.724 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T04:00:36.724 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:36.724 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T04:00:36.724 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T04:00:36.724 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T04:00:36.724 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:36.746 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-12T04:00:36.769 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T04:00:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-12T04:00:36.780 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T04:00:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-12T04:00:36.831 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-12T04:00:36.855 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-12T04:00:36.929 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-12T04:00:36.982 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-12T04:00:37.020 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-12T04:00:37.084 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-12T04:00:37.119 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-12T04:00:37.171 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-12T04:00:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-12T04:00:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-12T04:00:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-12T04:00:37.337 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-12T04:00:37.380 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-12T04:00:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-12T04:00:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-12T04:00:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-12T04:00:37.597 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-12T04:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-12T04:00:37.728 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-12T04:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T04:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T04:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T04:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T04:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T04:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:37.784 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-12T04:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T04:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T04:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T04:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T04:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T04:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T04:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T04:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T04:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T04:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T04:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T04:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:38.307 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-12T04:00:38.388 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-12T04:00:38.465 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-12T04:00:38.499 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:00:38.507 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:00:38.549 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-12T04:00:38.597 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-12T04:00:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-12T04:00:38.665 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-12T04:00:38.931 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-12T04:00:39.096 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-12T04:00:39.183 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-12T04:00:39.235 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-12T04:00:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-12T04:00:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-12T04:00:39.493 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-12T04:00:39.565 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-12T04:00:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-12T04:00:39.642 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-12T04:00:39.688 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-12T04:00:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-12T04:00:39.768 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-12T04:00:39.810 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-12T04:00:39.942 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-12T04:00:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-12T04:00:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-12T04:00:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-12T04:00:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-12T04:00:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-12T04:00:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-12T04:00:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-12T04:00:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-12T04:00:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-12T04:00:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-12T04:00:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-12T04:00:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-12T04:00:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-12T04:00:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-12T04:00:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-12T04:00:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-12T04:00:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-12T04:00:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-12T04:00:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-12T04:00:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-12T04:00:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-12T04:00:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-12T04:00:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-12T04:00:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-12T04:00:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-12T04:00:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-12T04:00:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-12T04:00:41.515 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-12T04:00:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-12T04:00:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-12T04:00:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-12T04:00:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-12T04:00:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-12T04:00:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-12T04:00:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-12T04:00:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-12T04:00:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-12T04:00:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T04:00:42.719 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-12T04:00:42.750 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-12T04:00:42.776 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-12T04:00:42.812 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-12T04:00:42.860 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-12T04:00:42.894 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-12T04:00:42.942 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-12T04:00:42.962 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-12T04:00:43.030 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-12T04:00:43.061 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-12T04:00:43.087 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-12T04:00:43.127 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-12T04:00:43.183 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-12T04:00:43.231 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-12T04:00:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-12T04:00:43.259 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-12T04:00:43.287 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-12T04:00:43.320 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-12T04:00:43.337 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-12T04:00:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-12T04:00:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-12T04:00:43.451 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-12T04:00:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-12T04:00:43.572 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-12T04:00:43.976 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-12T04:00:44.089 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-12T04:00:44.212 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:00:45.103 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-12T04:00:45.192 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-12T04:00:45.233 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-12T04:00:45.324 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-12T04:00:45.340 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-12T04:00:45.354 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-12T04:00:45.379 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-12T04:00:45.404 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-12T04:00:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-12T04:00:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-12T04:00:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T04:00:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-12T04:00:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-12T04:00:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-12T04:00:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T04:00:45.557 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-12T04:00:45.557 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T04:00:45.642 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T04:00:45.668 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-12T04:00:45.701 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-12T04:00:45.727 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-12T04:00:45.751 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-12T04:00:45.790 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-12T04:00:45.831 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-12T04:00:45.833 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-12T04:00:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:00:45.865 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-12T04:00:45.890 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-12T04:00:45.924 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-12T04:00:45.948 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:00:45.949 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-12T04:00:45.975 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-12T04:00:45.976 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-12T04:00:45.982 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:00:45.982 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:00:46.012 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-12T04:00:46.169 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:00:46.170 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T04:00:46.261 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T04:00:46.285 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-12T04:00:46.311 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-12T04:00:46.344 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-12T04:00:46.370 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-12T04:00:46.436 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T04:00:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-12T04:00:46.499 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:00:46.570 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:00:46.592 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-12T04:00:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:00:48.881 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:00:49.433 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:00:49.476 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:00:49.559 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:00:49.559 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T04:00:49.559 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T04:00:49.559 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:00:49.594 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:00:51.339 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-12T04:00:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-12T04:00:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-12T04:00:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-12T04:00:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-12T04:00:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-12T04:00:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-12T04:00:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-12T04:00:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-12T04:00:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-12T04:00:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-12T04:00:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-12T04:00:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-12T04:00:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-12T04:00:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-12T04:00:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-12T04:00:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-12T04:00:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T04:00:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: jq-1.6-16.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T04:00:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T04:00:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T04:00:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T04:00:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T04:00:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T04:00:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T04:00:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T04:00:54.384 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T04:00:54.384 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:00:54.384 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-03-12T04:00:54.856 DEBUG:teuthology.parallel:result is None 2024-03-12T04:01:01.855 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:01:01.855 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-03-12T04:01:01.855 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-03-12T04:01:01.855 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-03-12T04:01:01.856 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-03-12T04:01:01.856 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-03-12T04:01:01.856 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-03-12T04:01:01.856 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-03-12T04:01:01.856 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:01:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:01:02.039 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:02.039 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T04:01:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:03.517 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T04:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:01:03.544 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:03.759 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:01:03.781 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:01:03.781 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:03.781 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T04:01:03.781 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:01:03.781 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:01:03.781 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:04.226 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:01:04.249 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:01:04.249 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:04.249 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T04:01:04.249 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:01:04.249 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:01:04.249 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:04.336 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-12T04:01:04.762 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:01:04.793 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:01:04.825 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-12T04:01:04.885 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-12T04:01:04.908 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:01:05.139 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:01:05.167 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T04:01:06.128 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T04:01:06.151 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T04:01:06.250 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T04:01:06.404 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-12T04:01:06.427 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T04:01:06.450 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T04:01:06.451 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:06.451 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T04:01:06.451 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T04:01:06.451 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T04:01:06.451 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:06.495 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T04:01:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T04:01:06.559 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-12T04:01:07.418 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-12T04:01:07.455 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T04:01:07.478 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T04:01:07.478 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:07.478 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T04:01:07.478 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T04:01:07.478 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T04:01:07.478 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:07.523 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T04:01:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T04:01:07.545 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:07.545 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T04:01:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:07.590 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:01:07.612 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:01:07.612 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:07.613 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T04:01:07.613 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T04:01:07.613 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T04:01:07.613 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:07.916 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T04:01:07.938 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T04:01:07.938 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:01:07.938 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T04:01:07.938 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T04:01:07.939 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T04:01:07.939 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:12.381 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T04:01:12.422 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-12T04:01:12.452 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-12T04:01:12.479 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-12T04:01:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-12T04:01:12.770 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-12T04:01:12.810 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-12T04:01:12.862 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-12T04:01:12.932 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-12T04:01:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-12T04:01:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-12T04:01:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-12T04:01:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-12T04:01:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-12T04:01:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-12T04:01:13.221 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-12T04:01:13.785 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:01:13.828 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:01:13.847 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:01:14.656 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-12T04:01:14.731 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-12T04:01:14.772 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-12T04:01:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-12T04:01:14.901 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-12T04:01:14.926 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-12T04:01:14.952 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-12T04:01:14.978 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-12T04:01:14.978 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T04:01:15.066 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T04:01:15.104 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-12T04:01:15.104 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T04:01:15.188 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T04:01:15.214 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-12T04:01:15.247 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-12T04:01:15.273 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-12T04:01:15.298 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-12T04:01:15.331 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-12T04:01:15.371 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-12T04:01:15.403 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-12T04:01:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-12T04:01:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-12T04:01:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-12T04:01:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-12T04:01:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:01:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:01:15.690 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:01:15.690 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T04:01:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T04:01:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-12T04:01:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-12T04:01:15.856 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-12T04:01:15.882 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-12T04:01:15.948 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T04:01:20.784 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T04:01:20.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-12T04:01:20.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-12T04:01:20.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-12T04:01:20.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-12T04:01:20.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-12T04:01:20.785 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-12T04:01:20.786 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-12T04:01:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-12T04:01:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-12T04:01:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-12T04:01:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-12T04:01:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-12T04:01:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-12T04:01:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-12T04:01:20.803 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-12T04:01:20.804 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-12T04:01:20.805 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-12T04:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-12T04:01:20.807 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-12T04:01:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-12T04:01:20.809 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-12T04:01:20.810 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-12T04:01:20.811 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-12T04:01:20.812 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-12T04:01:20.813 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-12T04:01:23.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T04:01:23.878 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T04:01:23.879 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.880 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.881 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-16.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.882 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T04:01:23.883 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-12T04:01:23.884 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-12T04:01:23.885 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T04:01:23.886 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T04:01:23.887 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T04:01:23.888 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T04:01:23.889 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T04:01:23.890 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T04:01:23.890 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T04:01:23.890 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T04:01:23.890 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T04:01:23.890 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-12T04:01:23.890 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-03-12T04:01:24.393 DEBUG:teuthology.parallel:result is None 2024-03-12T04:02:08.184 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 130 kB/s | 21 MB 02:42 2024-03-12T04:02:13.159 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-12T04:02:15.729 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:02:15.730 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:02:15.743 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T04:02:15.744 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T04:02:15.745 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T04:02:15.745 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T04:02:15.747 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T04:02:15.750 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T04:02:15.751 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T04:02:15.836 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-03-12T04:02:15.854 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================ 2024-03-12T04:02:15.854 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-03-12T04:02:15.854 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================ 2024-03-12T04:02:15.854 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-12T04:02:15.855 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T04:02:15.856 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T04:02:15.857 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T04:02:15.858 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-12T04:02:15.859 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T04:02:15.860 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T04:02:15.861 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T04:02:15.862 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T04:02:15.863 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T04:02:15.864 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T04:02:15.865 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T04:02:15.866 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================ 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout:Install 150 Packages 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 24 Packages 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:02:15.867 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 223 M 2024-03-12T04:02:15.868 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-03-12T04:02:16.660 INFO:teuthology.orchestra.run.smithi042.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-03-12T04:02:16.844 INFO:teuthology.orchestra.run.smithi042.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 4.7 MB/s | 886 kB 00:00 2024-03-12T04:02:16.894 INFO:teuthology.orchestra.run.smithi042.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 2.8 MB/s | 145 kB 00:00 2024-03-12T04:02:17.027 INFO:teuthology.orchestra.run.smithi042.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 16 MB/s | 2.1 MB 00:00 2024-03-12T04:02:17.136 INFO:teuthology.orchestra.run.smithi042.stdout:(5/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 14 MB/s | 1.5 MB 00:00 2024-03-12T04:02:17.438 INFO:teuthology.orchestra.run.smithi042.stdout:(6/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 16 MB/s | 4.7 MB 00:00 2024-03-12T04:02:18.253 INFO:teuthology.orchestra.run.smithi042.stdout:(7/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 3.0 MB/s | 5.3 MB 00:01 2024-03-12T04:02:18.462 INFO:teuthology.orchestra.run.smithi042.stdout:(8/174): ceph-osd-19.0.0-1607.g94a53970.el9.x86 16 MB/s | 17 MB 00:01 2024-03-12T04:02:18.520 INFO:teuthology.orchestra.run.smithi042.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 435 kB/s | 25 kB 00:00 2024-03-12T04:02:19.979 INFO:teuthology.orchestra.run.smithi042.stdout:(10/174): ceph-common-19.0.0-1607.g94a53970.el9 5.7 MB/s | 20 MB 00:03 2024-03-12T04:02:20.046 INFO:teuthology.orchestra.run.smithi042.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 3.2 MB/s | 217 kB 00:00 2024-03-12T04:02:20.138 INFO:teuthology.orchestra.run.smithi042.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 296 kB/s | 27 kB 00:00 2024-03-12T04:02:20.221 INFO:teuthology.orchestra.run.smithi042.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 375 kB/s | 31 kB 00:00 2024-03-12T04:02:20.347 INFO:teuthology.orchestra.run.smithi042.stdout:(14/174): ceph-radosgw-19.0.0-1607.g94a53970.el 4.7 MB/s | 9.8 MB 00:02 2024-03-12T04:02:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:(15/174): libcephfs2-19.0.0-1607.g94a53970.el9. 4.1 MB/s | 742 kB 00:00 2024-03-12T04:02:20.430 INFO:teuthology.orchestra.run.smithi042.stdout:(16/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.9 MB/s | 163 kB 00:00 2024-03-12T04:02:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:(17/174): librados-devel-19.0.0-1607.g94a53970. 1.5 MB/s | 127 kB 00:00 2024-03-12T04:02:20.581 INFO:teuthology.orchestra.run.smithi042.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 3.2 MB/s | 498 kB 00:00 2024-03-12T04:02:20.665 INFO:teuthology.orchestra.run.smithi042.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 542 kB/s | 45 kB 00:00 2024-03-12T04:02:20.756 INFO:teuthology.orchestra.run.smithi042.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.4 MB/s | 131 kB 00:00 2024-03-12T04:02:20.865 INFO:teuthology.orchestra.run.smithi042.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 1.5 MB/s | 161 kB 00:00 2024-03-12T04:02:20.974 INFO:teuthology.orchestra.run.smithi042.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 2.9 MB/s | 320 kB 00:00 2024-03-12T04:02:21.074 INFO:teuthology.orchestra.run.smithi042.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 2.9 MB/s | 298 kB 00:00 2024-03-12T04:02:21.149 INFO:teuthology.orchestra.run.smithi042.stdout:(24/174): librgw2-19.0.0-1607.g94a53970.el9.x86 7.6 MB/s | 5.1 MB 00:00 2024-03-12T04:02:21.166 INFO:teuthology.orchestra.run.smithi042.stdout:(25/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.1 MB/s | 99 kB 00:00 2024-03-12T04:02:21.592 INFO:teuthology.orchestra.run.smithi042.stdout:(26/174): ceph-test-19.0.0-1607.g94a53970.el9.x 15 MB/s | 47 MB 00:03 2024-03-12T04:02:21.617 INFO:teuthology.orchestra.run.smithi042.stdout:(27/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 182 kB/s | 85 kB 00:00 2024-03-12T04:02:21.642 INFO:teuthology.orchestra.run.smithi042.stdout:(28/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 3.3 MB/s | 171 kB 00:00 2024-03-12T04:02:21.667 INFO:teuthology.orchestra.run.smithi042.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1607.g 626 kB/s | 31 kB 00:00 2024-03-12T04:02:21.701 INFO:teuthology.orchestra.run.smithi042.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 2.3 MB/s | 138 kB 00:00 2024-03-12T04:02:21.761 INFO:teuthology.orchestra.run.smithi042.stdout:(31/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 5.2 MB/s | 3.1 MB 00:00 2024-03-12T04:02:21.843 INFO:teuthology.orchestra.run.smithi042.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.9 MB/s | 247 kB 00:00 2024-03-12T04:02:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:(33/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 6.9 MB/s | 1.8 MB 00:00 2024-03-12T04:02:21.943 INFO:teuthology.orchestra.run.smithi042.stdout:(34/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 493 kB/s | 49 kB 00:00 2024-03-12T04:02:21.994 INFO:teuthology.orchestra.run.smithi042.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1607.g9 230 kB/s | 15 kB 00:00 2024-03-12T04:02:22.036 INFO:teuthology.orchestra.run.smithi042.stdout:(36/174): ceph-volume-19.0.0-1607.g94a53970.el9 2.8 MB/s | 264 kB 00:00 2024-03-12T04:02:22.186 INFO:teuthology.orchestra.run.smithi042.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-03-12T04:02:22.211 INFO:teuthology.orchestra.run.smithi042.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 104 kB/s | 18 kB 00:00 2024-03-12T04:02:22.245 INFO:teuthology.orchestra.run.smithi042.stdout:(39/174): cephadm-19.0.0-1607.g94a53970.el9.noa 2.9 MB/s | 748 kB 00:00 2024-03-12T04:02:22.412 INFO:teuthology.orchestra.run.smithi042.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 636 kB/s | 106 kB 00:00 2024-03-12T04:02:22.462 INFO:teuthology.orchestra.run.smithi042.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 662 kB 00:00 2024-03-12T04:02:22.496 INFO:teuthology.orchestra.run.smithi042.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2024-03-12T04:02:22.521 INFO:teuthology.orchestra.run.smithi042.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 307 kB 00:00 2024-03-12T04:02:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 383 kB/s | 32 kB 00:00 2024-03-12T04:02:22.571 INFO:teuthology.orchestra.run.smithi042.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 342 kB/s | 17 kB 00:00 2024-03-12T04:02:22.639 INFO:teuthology.orchestra.run.smithi042.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-12T04:02:22.664 INFO:teuthology.orchestra.run.smithi042.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 86 kB/s | 10 kB 00:00 2024-03-12T04:02:22.680 INFO:teuthology.orchestra.run.smithi042.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 159 kB 00:00 2024-03-12T04:02:22.706 INFO:teuthology.orchestra.run.smithi042.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-12T04:02:22.731 INFO:teuthology.orchestra.run.smithi042.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-12T04:02:22.765 INFO:teuthology.orchestra.run.smithi042.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 578 kB 00:00 2024-03-12T04:02:22.790 INFO:teuthology.orchestra.run.smithi042.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 560 kB/s | 46 kB 00:00 2024-03-12T04:02:22.815 INFO:teuthology.orchestra.run.smithi042.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-12T04:02:22.840 INFO:teuthology.orchestra.run.smithi042.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-12T04:02:22.865 INFO:teuthology.orchestra.run.smithi042.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-12T04:02:22.891 INFO:teuthology.orchestra.run.smithi042.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-03-12T04:02:22.958 INFO:teuthology.orchestra.run.smithi042.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-12T04:02:23.000 INFO:teuthology.orchestra.run.smithi042.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-12T04:02:23.025 INFO:teuthology.orchestra.run.smithi042.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 6.0 MB/s | 811 kB 00:00 2024-03-12T04:02:23.042 INFO:teuthology.orchestra.run.smithi042.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 134 kB 00:00 2024-03-12T04:02:23.075 INFO:teuthology.orchestra.run.smithi042.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-03-12T04:02:23.167 INFO:teuthology.orchestra.run.smithi042.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-03-12T04:02:23.184 INFO:teuthology.orchestra.run.smithi042.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 547 kB/s | 78 kB 00:00 2024-03-12T04:02:23.285 INFO:teuthology.orchestra.run.smithi042.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-12T04:02:23.301 INFO:teuthology.orchestra.run.smithi042.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 159 kB/s | 18 kB 00:00 2024-03-12T04:02:23.318 INFO:teuthology.orchestra.run.smithi042.stdout:(66/174): python3-jsonpatch-1.21-16.el9.noarch. 187 kB/s | 24 kB 00:00 2024-03-12T04:02:23.335 INFO:teuthology.orchestra.run.smithi042.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 959 kB/s | 32 kB 00:00 2024-03-12T04:02:23.360 INFO:teuthology.orchestra.run.smithi042.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-12T04:02:23.444 INFO:teuthology.orchestra.run.smithi042.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-12T04:02:23.469 INFO:teuthology.orchestra.run.smithi042.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-12T04:02:23.494 INFO:teuthology.orchestra.run.smithi042.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-12T04:02:23.520 INFO:teuthology.orchestra.run.smithi042.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 918 kB/s | 69 kB 00:00 2024-03-12T04:02:23.537 INFO:teuthology.orchestra.run.smithi042.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 989 kB/s | 41 kB 00:00 2024-03-12T04:02:23.562 INFO:teuthology.orchestra.run.smithi042.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-12T04:02:23.587 INFO:teuthology.orchestra.run.smithi042.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-12T04:02:23.604 INFO:teuthology.orchestra.run.smithi042.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-12T04:02:23.638 INFO:teuthology.orchestra.run.smithi042.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-12T04:02:23.655 INFO:teuthology.orchestra.run.smithi042.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 43 kB 00:00 2024-03-12T04:02:23.905 INFO:teuthology.orchestra.run.smithi042.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 52 MB/s | 16 MB 00:00 2024-03-12T04:02:23.930 INFO:teuthology.orchestra.run.smithi042.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 142 kB/s | 39 kB 00:00 2024-03-12T04:02:23.955 INFO:teuthology.orchestra.run.smithi042.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 138 kB/s | 41 kB 00:00 2024-03-12T04:02:23.980 INFO:teuthology.orchestra.run.smithi042.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-12T04:02:24.006 INFO:teuthology.orchestra.run.smithi042.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-12T04:02:24.031 INFO:teuthology.orchestra.run.smithi042.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 838 kB/s | 63 kB 00:00 2024-03-12T04:02:24.056 INFO:teuthology.orchestra.run.smithi042.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-12T04:02:24.081 INFO:teuthology.orchestra.run.smithi042.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-12T04:02:24.107 INFO:teuthology.orchestra.run.smithi042.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-12T04:02:24.148 INFO:teuthology.orchestra.run.smithi042.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-12T04:02:24.174 INFO:teuthology.orchestra.run.smithi042.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-03-12T04:02:24.199 INFO:teuthology.orchestra.run.smithi042.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-12T04:02:24.224 INFO:teuthology.orchestra.run.smithi042.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-12T04:02:24.358 INFO:teuthology.orchestra.run.smithi042.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-12T04:02:24.383 INFO:teuthology.orchestra.run.smithi042.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-12T04:02:24.408 INFO:teuthology.orchestra.run.smithi042.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-12T04:02:24.500 INFO:teuthology.orchestra.run.smithi042.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-03-12T04:02:24.526 INFO:teuthology.orchestra.run.smithi042.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 151 kB/s | 21 kB 00:00 2024-03-12T04:02:24.551 INFO:teuthology.orchestra.run.smithi042.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-03-12T04:02:24.718 INFO:teuthology.orchestra.run.smithi042.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-12T04:02:24.777 INFO:teuthology.orchestra.run.smithi042.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-03-12T04:02:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 600 kB/s | 35 kB 00:00 2024-03-12T04:02:24.902 INFO:teuthology.orchestra.run.smithi042.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 667 kB/s | 44 kB 00:00 2024-03-12T04:02:24.935 INFO:teuthology.orchestra.run.smithi042.stdout:(102/174): qatlib-23.11.0-1.el9.x86_64.rpm 996 kB/s | 216 kB 00:00 2024-03-12T04:02:24.961 INFO:teuthology.orchestra.run.smithi042.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-12T04:02:25.011 INFO:teuthology.orchestra.run.smithi042.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-03-12T04:02:25.153 INFO:teuthology.orchestra.run.smithi042.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 6.3 MB/s | 4.1 MB 00:00 2024-03-12T04:02:25.220 INFO:teuthology.orchestra.run.smithi042.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-03-12T04:02:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-03-12T04:02:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-03-12T04:02:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-03-12T04:02:25.405 INFO:teuthology.orchestra.run.smithi042.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-03-12T04:02:25.421 INFO:teuthology.orchestra.run.smithi042.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 500 kB/s | 49 kB 00:00 2024-03-12T04:02:25.438 INFO:teuthology.orchestra.run.smithi042.stdout:(112/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 227 kB/s | 26 kB 00:00 2024-03-12T04:02:25.463 INFO:teuthology.orchestra.run.smithi042.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-12T04:02:25.489 INFO:teuthology.orchestra.run.smithi042.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-12T04:02:25.514 INFO:teuthology.orchestra.run.smithi042.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 11 MB/s | 548 kB 00:00 2024-03-12T04:02:25.548 INFO:teuthology.orchestra.run.smithi042.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-03-12T04:02:25.564 INFO:teuthology.orchestra.run.smithi042.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 586 kB/s | 43 kB 00:00 2024-03-12T04:02:25.581 INFO:teuthology.orchestra.run.smithi042.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 492 kB/s | 32 kB 00:00 2024-03-12T04:02:25.599 INFO:teuthology.orchestra.run.smithi042.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 449 kB/s | 15 kB 00:00 2024-03-12T04:02:25.623 INFO:teuthology.orchestra.run.smithi042.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-12T04:02:25.649 INFO:teuthology.orchestra.run.smithi042.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-03-12T04:02:25.674 INFO:teuthology.orchestra.run.smithi042.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 717 kB/s | 35 kB 00:00 2024-03-12T04:02:25.699 INFO:teuthology.orchestra.run.smithi042.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-03-12T04:02:25.724 INFO:teuthology.orchestra.run.smithi042.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.7 MB/s | 203 kB 00:00 2024-03-12T04:02:25.783 INFO:teuthology.orchestra.run.smithi042.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-03-12T04:02:25.808 INFO:teuthology.orchestra.run.smithi042.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.8 MB/s | 144 kB 00:00 2024-03-12T04:02:25.833 INFO:teuthology.orchestra.run.smithi042.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 101 kB/s | 11 kB 00:00 2024-03-12T04:02:25.859 INFO:teuthology.orchestra.run.smithi042.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-03-12T04:02:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-03-12T04:02:25.909 INFO:teuthology.orchestra.run.smithi042.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-03-12T04:02:25.934 INFO:teuthology.orchestra.run.smithi042.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-03-12T04:02:25.960 INFO:teuthology.orchestra.run.smithi042.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 927 kB/s | 46 kB 00:00 2024-03-12T04:02:25.993 INFO:teuthology.orchestra.run.smithi042.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-03-12T04:02:26.027 INFO:teuthology.orchestra.run.smithi042.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 860 kB/s | 79 kB 00:00 2024-03-12T04:02:26.052 INFO:teuthology.orchestra.run.smithi042.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 643 kB/s | 58 kB 00:00 2024-03-12T04:02:26.077 INFO:teuthology.orchestra.run.smithi042.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-03-12T04:02:26.103 INFO:teuthology.orchestra.run.smithi042.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-12T04:02:26.128 INFO:teuthology.orchestra.run.smithi042.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-12T04:02:26.153 INFO:teuthology.orchestra.run.smithi042.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-03-12T04:02:26.178 INFO:teuthology.orchestra.run.smithi042.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 618 kB/s | 31 kB 00:00 2024-03-12T04:02:26.204 INFO:teuthology.orchestra.run.smithi042.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-12T04:02:26.246 INFO:teuthology.orchestra.run.smithi042.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 645 kB/s | 59 kB 00:00 2024-03-12T04:02:26.271 INFO:teuthology.orchestra.run.smithi042.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 537 kB/s | 36 kB 00:00 2024-03-12T04:02:26.296 INFO:teuthology.orchestra.run.smithi042.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 556 kB/s | 51 kB 00:00 2024-03-12T04:02:26.321 INFO:teuthology.orchestra.run.smithi042.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-12T04:02:26.346 INFO:teuthology.orchestra.run.smithi042.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-03-12T04:02:26.372 INFO:teuthology.orchestra.run.smithi042.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-03-12T04:02:26.397 INFO:teuthology.orchestra.run.smithi042.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-03-12T04:02:26.422 INFO:teuthology.orchestra.run.smithi042.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-03-12T04:02:26.464 INFO:teuthology.orchestra.run.smithi042.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-12T04:02:26.564 INFO:teuthology.orchestra.run.smithi042.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.8 MB/s | 321 kB 00:00 2024-03-12T04:02:26.690 INFO:teuthology.orchestra.run.smithi042.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 19 MB/s | 2.3 MB 00:00 2024-03-12T04:02:26.765 INFO:teuthology.orchestra.run.smithi042.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-12T04:02:26.807 INFO:teuthology.orchestra.run.smithi042.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-12T04:02:26.849 INFO:teuthology.orchestra.run.smithi042.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-12T04:02:26.882 INFO:teuthology.orchestra.run.smithi042.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 71 kB 00:00 2024-03-12T04:02:26.925 INFO:teuthology.orchestra.run.smithi042.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 874 kB/s | 36 kB 00:00 2024-03-12T04:02:26.966 INFO:teuthology.orchestra.run.smithi042.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-12T04:02:27.008 INFO:teuthology.orchestra.run.smithi042.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-12T04:02:27.075 INFO:teuthology.orchestra.run.smithi042.stdout:(160/174): librados2-19.0.0-1607.g94a53970.el9. 5.0 MB/s | 3.4 MB 00:00 2024-03-12T04:02:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 276 kB/s | 23 kB 00:00 2024-03-12T04:02:27.126 INFO:teuthology.orchestra.run.smithi042.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-12T04:02:27.151 INFO:teuthology.orchestra.run.smithi042.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-12T04:02:27.201 INFO:teuthology.orchestra.run.smithi042.stdout:(164/174): librbd1-19.0.0-1607.g94a53970.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-03-12T04:02:27.226 INFO:teuthology.orchestra.run.smithi042.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-12T04:02:27.252 INFO:teuthology.orchestra.run.smithi042.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 635 kB/s | 64 kB 00:00 2024-03-12T04:02:27.278 INFO:teuthology.orchestra.run.smithi042.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 381 kB/s | 29 kB 00:00 2024-03-12T04:02:27.369 INFO:teuthology.orchestra.run.smithi042.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 24 MB/s | 2.2 MB 00:00 2024-03-12T04:02:27.411 INFO:teuthology.orchestra.run.smithi042.stdout:(169/174): openssl-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 1.2 MB 00:00 2024-03-12T04:02:27.445 INFO:teuthology.orchestra.run.smithi042.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-12T04:02:27.470 INFO:teuthology.orchestra.run.smithi042.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 532 kB/s | 31 kB 00:00 2024-03-12T04:02:27.512 INFO:teuthology.orchestra.run.smithi042.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-12T04:02:27.570 INFO:teuthology.orchestra.run.smithi042.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.8 MB/s | 2.3 MB 00:00 2024-03-12T04:02:27.629 INFO:teuthology.orchestra.run.smithi042.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 468 kB/s | 74 kB 00:00 2024-03-12T04:02:27.634 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-03-12T04:02:27.635 INFO:teuthology.orchestra.run.smithi042.stdout:Total 19 MB/s | 223 MB 00:11 2024-03-12T04:02:28.100 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-03-12T04:02:28.432 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-03-12T04:02:28.432 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-03-12T04:02:30.506 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-03-12T04:02:30.507 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-03-12T04:02:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-03-12T04:02:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-12T04:02:37.743 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-12T04:02:37.773 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:02:37.782 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:02:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-12T04:02:37.874 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-12T04:02:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-12T04:02:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-12T04:02:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-12T04:02:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-12T04:02:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:02:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:02:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:02:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:02:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T04:02:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-12T04:02:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-12T04:02:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-12T04:02:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:02:39.244 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:02:39.274 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:02:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:02:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-12T04:02:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:02:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:02:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:02:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:02:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-12T04:02:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-12T04:02:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:02:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:02:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:02:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:02:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:02:40.452 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-12T04:02:40.565 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-12T04:02:40.600 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-12T04:02:40.627 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-12T04:02:40.662 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-12T04:02:40.738 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-12T04:02:40.793 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-12T04:02:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-12T04:02:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-12T04:02:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-12T04:02:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-12T04:02:41.006 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-12T04:02:41.033 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-12T04:02:41.105 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-12T04:02:41.138 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-12T04:02:41.182 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-12T04:02:41.210 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-12T04:02:41.281 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-12T04:02:41.325 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-12T04:02:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-12T04:02:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-12T04:02:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-12T04:02:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-12T04:02:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-12T04:02:42.320 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-12T04:02:42.367 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:02:42.374 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:02:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-12T04:02:42.456 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-12T04:02:42.494 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-12T04:02:42.524 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-12T04:02:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-12T04:02:43.107 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-12T04:02:43.168 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-12T04:02:43.212 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-12T04:02:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-12T04:02:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-12T04:02:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-12T04:02:43.543 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-12T04:02:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-12T04:02:43.610 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-12T04:02:43.657 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-12T04:02:43.693 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-12T04:02:43.736 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-12T04:02:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-12T04:02:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-12T04:02:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-12T04:02:44.014 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-12T04:02:44.055 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-12T04:02:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-12T04:02:44.188 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-12T04:02:44.229 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-12T04:02:44.254 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-12T04:02:44.279 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-12T04:02:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-12T04:02:44.518 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-12T04:02:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-12T04:02:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-12T04:02:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-12T04:02:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-12T04:02:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-12T04:02:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-12T04:02:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-12T04:02:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-12T04:02:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-12T04:02:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-12T04:02:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-12T04:02:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-12T04:02:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-12T04:02:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-12T04:02:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-12T04:02:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-12T04:02:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-12T04:02:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-12T04:02:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-12T04:02:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-12T04:02:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-12T04:02:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-12T04:02:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-12T04:02:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-12T04:02:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-12T04:02:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-12T04:02:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-12T04:02:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-12T04:02:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-12T04:02:47.005 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-12T04:02:47.062 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-12T04:02:47.154 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-12T04:02:47.189 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-12T04:02:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-12T04:02:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-12T04:02:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-12T04:02:48.152 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-12T04:02:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-12T04:02:49.567 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-12T04:02:49.646 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-12T04:02:49.677 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-12T04:02:49.708 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-12T04:02:49.736 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-12T04:02:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-12T04:02:50.207 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:02:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:02:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-12T04:02:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-12T04:02:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-12T04:02:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:02:50.948 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:02:50.970 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-12T04:02:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:02:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:02:53.820 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:02:53.870 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:02:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:02:53.959 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T04:02:53.959 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T04:02:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:02:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:03:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:03:06.032 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-03-12T04:03:06.033 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-03-12T04:03:06.033 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-03-12T04:03:06.033 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-03-12T04:03:06.033 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-03-12T04:03:06.033 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-03-12T04:03:06.033 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-03-12T04:03:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:06.200 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:03:06.215 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:03:06.215 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:06.215 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T04:03:06.215 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:03:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:03:07.779 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:07.779 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T04:03:07.779 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:03:07.779 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:03:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:07.997 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:03:08.019 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:03:08.019 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:08.019 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T04:03:08.019 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:03:08.019 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:03:08.019 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:08.480 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:03:08.503 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:03:08.503 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:08.503 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T04:03:08.503 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:03:08.503 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:03:08.503 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:08.607 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-12T04:03:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:03:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:03:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-12T04:03:09.057 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-12T04:03:09.087 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:03:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:03:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T04:03:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T04:03:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T04:03:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T04:03:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-12T04:03:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T04:03:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T04:03:10.738 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:10.738 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T04:03:10.738 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T04:03:10.738 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T04:03:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T04:03:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T04:03:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-12T04:03:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-12T04:03:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T04:03:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T04:03:11.825 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:11.825 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T04:03:11.825 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T04:03:11.825 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T04:03:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T04:03:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T04:03:11.890 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:11.891 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T04:03:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:03:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:03:11.958 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:11.958 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T04:03:11.958 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T04:03:11.959 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T04:03:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T04:03:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T04:03:12.319 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:03:12.319 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T04:03:12.319 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T04:03:12.319 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T04:03:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T04:03:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-12T04:03:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-12T04:03:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-12T04:03:17.286 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-12T04:03:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-12T04:03:17.456 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-12T04:03:17.506 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-12T04:03:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-12T04:03:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-12T04:03:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-12T04:03:17.671 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-12T04:03:17.701 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-12T04:03:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-12T04:03:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-12T04:03:17.882 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-12T04:03:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:03:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:03:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T04:03:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-12T04:03:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-12T04:03:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-12T04:03:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-12T04:03:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-12T04:03:19.689 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-12T04:03:19.714 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-12T04:03:19.741 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-12T04:03:19.741 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T04:03:19.829 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T04:03:19.858 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-12T04:03:19.858 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T04:03:19.935 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T04:03:19.969 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-12T04:03:20.002 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-12T04:03:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-12T04:03:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-12T04:03:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-12T04:03:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-12T04:03:20.174 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-12T04:03:20.199 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-12T04:03:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-12T04:03:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-12T04:03:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-12T04:03:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:03:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:03:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T04:03:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T04:03:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T04:03:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-12T04:03:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-12T04:03:20.628 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-12T04:03:20.653 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T04:03:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T04:03:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-12T04:03:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-12T04:03:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-12T04:03:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-12T04:03:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:03:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-12T04:03:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-12T04:03:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-12T04:03:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-12T04:03:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-12T04:03:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-12T04:03:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-12T04:03:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-12T04:03:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-12T04:03:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-12T04:03:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-12T04:03:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-12T04:03:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-12T04:03:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T04:03:28.561 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.562 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: jq-1.6-16.el9.x86_64 2024-03-12T04:03:28.565 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T04:03:28.566 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T04:03:28.567 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T04:03:28.568 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T04:03:28.569 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T04:03:28.570 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T04:03:28.571 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-12T04:03:28.572 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:03:28.573 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-03-12T04:03:29.045 DEBUG:teuthology.parallel:result is None 2024-03-12T04:03:29.046 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:03:29.189 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T04:03:29.210 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-1607.g94a53970.el9 2024-03-12T04:03:29.211 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-12T04:03:29.211 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-12T04:03:29.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:03:29.355 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T04:03:29.382 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-1607.g94a53970.el9 2024-03-12T04:03:29.382 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-12T04:03:29.382 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-12T04:03:29.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:03:29.603 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T04:03:29.629 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-1607.g94a53970.el9 2024-03-12T04:03:29.629 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-12T04:03:29.629 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-12T04:03:29.631 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-12T04:03:29.632 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:03:29.632 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T04:03:29.663 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:03:29.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T04:03:29.697 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:03:29.697 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T04:03:29.731 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-12T04:03:29.731 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:03:29.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T04:03:29.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T04:03:29.826 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:03:29.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T04:03:29.857 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T04:03:29.926 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:03:29.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T04:03:29.956 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T04:03:30.026 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-12T04:03:30.029 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:03:30.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T04:03:30.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T04:03:30.122 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:03:30.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T04:03:30.152 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T04:03:30.222 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:03:30.222 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T04:03:30.249 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T04:03:30.317 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-12T04:03:30.317 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:03:30.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T04:03:30.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T04:03:30.410 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:03:30.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T04:03:30.442 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T04:03:30.511 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:03:30.511 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T04:03:30.538 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T04:03:30.606 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-12T04:03:30.708 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-12T04:03:30.708 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:03:30.709 INFO:tasks.cephadm:Cluster fsid is 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:03:30.709 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-12T04:03:30.709 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.42', 'mon.b': '172.21.15.119', 'mon.c': '172.21.15.136'} 2024-03-12T04:03:30.709 INFO:tasks.cephadm:First mon is mon.a on smithi042 2024-03-12T04:03:30.709 INFO:tasks.cephadm:First mgr is x 2024-03-12T04:03:30.709 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-12T04:03:30.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2024-03-12T04:03:30.737 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2024-03-12T04:03:30.766 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-03-12T04:03:30.793 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-12T04:03:30.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:03:31.012 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77431/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:08:19.449750', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77429/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:06:59.696700', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-12T04:03:31.232 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 94a539707733325322964c6a187c6a1b41be1daf from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:03:31.233 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm 2024-03-12T04:03:31.233 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:32.013 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 12 04:03 /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:32.013 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:33.185 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 12 04:03 /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:33.185 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:34.098 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 12 04:03 /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:34.098 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:34.117 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:34.135 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T04:03:34.169 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf on all hosts... 2024-03-12T04:03:34.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-12T04:03:34.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-12T04:03:34.178 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-12T04:03:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T04:03:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T04:03:35.120 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T04:04:32.563 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-12T04:04:32.563 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-12T04:04:32.563 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-12T04:04:32.563 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2024-03-12T04:04:32.563 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-12T04:04:32.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-12T04:04:32.563 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-12T04:04:35.432 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-03-12T04:04:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-12T04:04:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-12T04:04:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-03-12T04:04:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-12T04:04:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-03-12T04:04:35.432 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-03-12T04:04:36.106 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:04:36.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-12T04:04:36.106 INFO:teuthology.orchestra.run.smithi042.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-12T04:04:36.106 INFO:teuthology.orchestra.run.smithi042.stdout: "repo_digests": [ 2024-03-12T04:04:36.106 INFO:teuthology.orchestra.run.smithi042.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-12T04:04:36.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:04:36.106 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:04:36.137 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2024-03-12T04:04:36.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2024-03-12T04:04:36.220 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-03-12T04:04:36.250 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2024-03-12T04:04:36.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2024-03-12T04:04:36.310 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-03-12T04:04:36.338 INFO:tasks.cephadm:Writing seed config... 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-12T04:04:36.339 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-12T04:04:36.340 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-12T04:04:36.341 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-12T04:04:36.342 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-12T04:04:36.342 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-12T04:04:36.342 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-12T04:04:36.342 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-12T04:04:36.342 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:04:36.343 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-12T04:04:36.358 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7cca013a-e025-11ee-95c8-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-12T04:04:36.359 DEBUG:teuthology.orchestra.run.smithi042:mon.a> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service 2024-03-12T04:04:36.401 DEBUG:teuthology.orchestra.run.smithi042:mgr.x> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.x.service 2024-03-12T04:04:36.443 INFO:tasks.cephadm:Bootstrapping... 2024-03-12T04:04:36.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf -v bootstrap --fsid 7cca013a-e025-11ee-95c8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.42 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:04:36.641 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-03-12T04:04:36.641 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf', '-v', 'bootstrap', '--fsid', '7cca013a-e025-11ee-95c8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.42', '--skip-admin-label'] 2024-03-12T04:04:36.664 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-12T04:04:36.665 INFO:teuthology.orchestra.run.smithi042.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-12T04:04:36.665 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying podman|docker is present... 2024-03-12T04:04:36.688 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-12T04:04:36.688 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying lvm2 is present... 2024-03-12T04:04:36.688 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying time synchronization is in place... 2024-03-12T04:04:36.695 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-12T04:04:36.696 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-12T04:04:36.701 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-12T04:04:36.701 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-03-12T04:04:36.707 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-03-12T04:04:36.713 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-03-12T04:04:36.713 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-03-12T04:04:36.713 INFO:teuthology.orchestra.run.smithi042.stdout:Repeating the final host check... 2024-03-12T04:04:36.736 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-12T04:04:36.736 INFO:teuthology.orchestra.run.smithi042.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-12T04:04:36.736 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl is present 2024-03-12T04:04:36.736 INFO:teuthology.orchestra.run.smithi042.stdout:lvcreate is present 2024-03-12T04:04:36.742 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-12T04:04:36.742 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-12T04:04:36.747 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-12T04:04:36.748 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-03-12T04:04:36.754 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-03-12T04:04:36.760 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-03-12T04:04:36.760 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-03-12T04:04:36.760 INFO:teuthology.orchestra.run.smithi042.stdout:Host looks OK 2024-03-12T04:04:36.760 INFO:teuthology.orchestra.run.smithi042.stdout:Cluster fsid: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:04:36.760 INFO:teuthology.orchestra.run.smithi042.stdout:Acquiring lock 139999206780880 on /run/cephadm/7cca013a-e025-11ee-95c8-87774f69a715.lock 2024-03-12T04:04:36.760 INFO:teuthology.orchestra.run.smithi042.stdout:Lock 139999206780880 acquired on /run/cephadm/7cca013a-e025-11ee-95c8-87774f69a715.lock 2024-03-12T04:04:36.760 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 3300 ... 2024-03-12T04:04:36.761 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 6789 ... 2024-03-12T04:04:36.761 INFO:teuthology.orchestra.run.smithi042.stdout:Base mon IP(s) is [172.21.15.42:3300, 172.21.15.42:6789], mon addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-03-12T04:04:36.764 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.42 metric 100 2024-03-12T04:04:36.764 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2024-03-12T04:04:36.766 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-12T04:04:36.766 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-12T04:04:36.766 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-12T04:04:36.769 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-12T04:04:36.769 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-12T04:04:36.769 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-12T04:04:36.769 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-12T04:04:36.769 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-03-12T04:04:36.769 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-12T04:04:36.770 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-03-12T04:04:36.770 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-03-12T04:04:36.770 INFO:teuthology.orchestra.run.smithi042.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-12T04:04:36.770 INFO:teuthology.orchestra.run.smithi042.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-12T04:04:36.771 INFO:teuthology.orchestra.run.smithi042.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T04:04:37.258 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-12T04:04:37.258 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T04:04:37.258 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Getting image source signatures 2024-03-12T04:04:37.258 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:0b3c7bee4e66dab4b5376aaf0baf8ba5fc1c3e66c27dc601157782d617bd9dc6 2024-03-12T04:04:37.258 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-12T04:04:37.258 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying config sha256:a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-12T04:04:37.258 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-12T04:04:38.128 INFO:teuthology.orchestra.run.smithi042.stdout:ceph: stdout ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-12T04:04:38.129 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph version: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-12T04:04:38.129 INFO:teuthology.orchestra.run.smithi042.stdout:Extracting ceph user uid/gid from container image... 2024-03-12T04:04:39.068 INFO:teuthology.orchestra.run.smithi042.stdout:stat: stdout 167 167 2024-03-12T04:04:39.068 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial keys... 2024-03-12T04:04:39.915 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQBX1O9lC2tcExAA/Fj0eQLyFXZji4AlJzr7pw== 2024-03-12T04:04:40.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQBY1O9lHNnEDBAArU+O7SGZq7KJvfxpfot6wg== 2024-03-12T04:04:41.785 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQBZ1O9llAYjCxAAKqgEMZSh30VKQRkjC32SnQ== 2024-03-12T04:04:41.785 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial monmap... 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool for a [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: set fsid to 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:04:42.673 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mon... 2024-03-12T04:04:43.555 INFO:teuthology.orchestra.run.smithi042.stdout:create mon.a on 2024-03-12T04:04:43.772 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-12T04:04:43.915 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-12T04:04:44.090 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7cca013a-e025-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-7cca013a-e025-11ee-95c8-87774f69a715.target. 2024-03-12T04:04:44.090 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7cca013a-e025-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-7cca013a-e025-11ee-95c8-87774f69a715.target. 2024-03-12T04:04:44.334 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a 2024-03-12T04:04:44.334 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service: Unit ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service not loaded. 2024-03-12T04:04:44.507 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7cca013a-e025-11ee-95c8-87774f69a715.target.wants/ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-7cca013a-e025-11ee-95c8-87774f69a715@.service. 2024-03-12T04:04:45.053 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T04:04:45.053 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-03-12T04:04:45.060 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T04:04:45.060 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-03-12T04:04:45.060 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-03-12T04:04:45.060 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-03-12T04:04:45.060 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon to start... 2024-03-12T04:04:45.061 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon... 2024-03-12T04:04:46.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout cluster: 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout id: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout services: 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.571632s) 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout data: 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-12T04:04:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pgs: 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e5d9a3700 1 Processor -- start 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e5d9a3700 1 -- start start 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e5d9a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5810a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e5d9a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5807ae50 con 0x7f8e58107f70 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5810a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5810a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34122/0 (socket says 172.21.15.42:34122) 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e56ffd700 1 -- 172.21.15.42:0/2544141600 learned_addr learned my addr 172.21.15.42:0/2544141600 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.499+0000 7f8e56ffd700 1 -- 172.21.15.42:0/2544141600 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5810a8a0 con 0x7f8e58107f70 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e56ffd700 1 --2- 172.21.15.42:0/2544141600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5810a360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009e80 tx=0x7f8e4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b74115c546fb80d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e55ffb700 1 -- 172.21.15.42:0/2544141600 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4800e070 con 0x7f8e58107f70 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e55ffb700 1 -- 172.21.15.42:0/2544141600 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48003a20 con 0x7f8e58107f70 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e55ffb700 1 -- 172.21.15.42:0/2544141600 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e48003ce0 con 0x7f8e58107f70 2024-03-12T04:04:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/2544141600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 msgr2=0x7f8e5810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e5d9a3700 1 --2- 172.21.15.42:0/2544141600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5810a360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009e80 tx=0x7f8e4800b330 comp rx=0 tx=0).stop 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/2544141600 shutdown_connections 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e5d9a3700 1 --2- 172.21.15.42:0/2544141600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5810a360 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/2544141600 >> 172.21.15.42:0/2544141600 conn(0x7f8e581019a0 msgr2=0x7f8e58103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/2544141600 shutdown_connections 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.500+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/2544141600 wait complete. 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.501+0000 7f8e5d9a3700 1 Processor -- start 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.501+0000 7f8e5d9a3700 1 -- start start 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.501+0000 7f8e5d9a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.501+0000 7f8e5d9a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5819afd0 con 0x7f8e58107f70 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.501+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.501+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34134/0 (socket says 172.21.15.42:34134) 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.501+0000 7f8e56ffd700 1 -- 172.21.15.42:0/714993460 learned_addr learned my addr 172.21.15.42:0/714993460 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e56ffd700 1 -- 172.21.15.42:0/714993460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e48009b30 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e56ffd700 1 --2- 172.21.15.42:0/714993460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5819aa90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e5807a290 tx=0x7f8e48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e5c9a1700 1 -- 172.21.15.42:0/714993460 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e48004370 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e5c9a1700 1 -- 172.21.15.42:0/714993460 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48004510 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e5c9a1700 1 -- 172.21.15.42:0/714993460 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e480047f0 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5819b1d0 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5819b670 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e5c9a1700 1 -- 172.21.15.42:0/714993460 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8e480127c0 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.502+0000 7f8e5c9a1700 1 -- 172.21.15.42:0/714993460 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e4801dbf0 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.503+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e581947c0 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.505+0000 7f8e5c9a1700 1 -- 172.21.15.42:0/714993460 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8e4802b060 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.577+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8e5810ad80 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5c9a1700 1 -- 172.21.15.42:0/714993460 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f8e48021030 con 0x7f8e58107f70 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 msgr2=0x7f8e5819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:46.200 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5d9a3700 1 --2- 172.21.15.42:0/714993460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5819aa90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e5807a290 tx=0x7f8e48004ad0 comp rx=0 tx=0).stop 2024-03-12T04:04:46.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 shutdown_connections 2024-03-12T04:04:46.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5d9a3700 1 --2- 172.21.15.42:0/714993460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e58107f70 0x7f8e5819aa90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:46.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 >> 172.21.15.42:0/714993460 conn(0x7f8e581019a0 msgr2=0x7f8e58103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:46.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 shutdown_connections 2024-03-12T04:04:46.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:45.578+0000 7f8e5d9a3700 1 -- 172.21.15.42:0/714993460 wait complete. 2024-03-12T04:04:46.201 INFO:teuthology.orchestra.run.smithi042.stdout:mon is available 2024-03-12T04:04:46.201 INFO:teuthology.orchestra.run.smithi042.stdout:Assimilating anything we can from ceph.conf... 2024-03-12T04:04:47.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:47 smithi042 ceph-mon[31888]: from='client.? 172.21.15.42:0/3341829878' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-12T04:04:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.613+0000 7fa74309c700 1 Processor -- start 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.613+0000 7fa74309c700 1 -- start start 2024-03-12T04:04:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.613+0000 7fa74309c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c1078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.613+0000 7fa74309c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c107eb0 con 0x7fa73c1074c0 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.613+0000 7fa740e38700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c1078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.614+0000 7fa740e38700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c1078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34146/0 (socket says 172.21.15.42:34146) 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.614+0000 7fa740e38700 1 -- 172.21.15.42:0/2695749646 learned_addr learned my addr 172.21.15.42:0/2695749646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.614+0000 7fa740e38700 1 -- 172.21.15.42:0/2695749646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c108670 con 0x7fa73c1074c0 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.614+0000 7fa740e38700 1 --2- 172.21.15.42:0/2695749646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c1078e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa7300098a0 tx=0x7fa730009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767a7c5fed0ed586 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.614+0000 7fa73b7fe700 1 -- 172.21.15.42:0/2695749646 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa730004160 con 0x7fa73c1074c0 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.614+0000 7fa73b7fe700 1 -- 172.21.15.42:0/2695749646 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7300042c0 con 0x7fa73c1074c0 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.614+0000 7fa73b7fe700 1 -- 172.21.15.42:0/2695749646 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7300045a0 con 0x7fa73c1074c0 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 -- 172.21.15.42:0/2695749646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 msgr2=0x7fa73c1078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 --2- 172.21.15.42:0/2695749646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c1078e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa7300098a0 tx=0x7fa730009bb0 comp rx=0 tx=0).stop 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 -- 172.21.15.42:0/2695749646 shutdown_connections 2024-03-12T04:04:47.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 --2- 172.21.15.42:0/2695749646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c1078e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 -- 172.21.15.42:0/2695749646 >> 172.21.15.42:0/2695749646 conn(0x7fa73c102b00 msgr2=0x7fa73c104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 -- 172.21.15.42:0/2695749646 shutdown_connections 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 -- 172.21.15.42:0/2695749646 wait complete. 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 Processor -- start 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.615+0000 7fa74309c700 1 -- start start 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa74309c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa74309c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c19ae70 con 0x7fa73c1074c0 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa740e38700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa740e38700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34156/0 (socket says 172.21.15.42:34156) 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa740e38700 1 -- 172.21.15.42:0/3341829878 learned_addr learned my addr 172.21.15.42:0/3341829878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa740e38700 1 -- 172.21.15.42:0/3341829878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730009550 con 0x7fa73c1074c0 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa740e38700 1 --2- 172.21.15.42:0/3341829878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c19a930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa730004000 tx=0x7fa730004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa730011630 con 0x7fa73c1074c0 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa730011790 con 0x7fa73c1074c0 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c19b070 con 0x7fa73c1074c0 2024-03-12T04:04:47.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.616+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa730011a30 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.617+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c19b510 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.617+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa730022070 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.617+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa73001c450 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.618+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c1945f0 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.620+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa730011b90 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.690+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa73c108c50 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.693+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fa730011de0 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.694+0000 7fa739ffb700 1 -- 172.21.15.42:0/3341829878 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fa73001c6b0 con 0x7fa73c1074c0 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.694+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 msgr2=0x7fa73c19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.694+0000 7fa74309c700 1 --2- 172.21.15.42:0/3341829878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c19a930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa730004000 tx=0x7fa730004750 comp rx=0 tx=0).stop 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.695+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 shutdown_connections 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.695+0000 7fa74309c700 1 --2- 172.21.15.42:0/3341829878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73c1074c0 0x7fa73c19a930 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.695+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 >> 172.21.15.42:0/3341829878 conn(0x7fa73c102b00 msgr2=0x7fa73c191730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.695+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 shutdown_connections 2024-03-12T04:04:47.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:46.695+0000 7fa74309c700 1 -- 172.21.15.42:0/3341829878 wait complete. 2024-03-12T04:04:47.283 INFO:teuthology.orchestra.run.smithi042.stdout:Generating new minimal ceph.conf... 2024-03-12T04:04:48.309 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:48 smithi042 ceph-mon[31888]: from='client.? 172.21.15.42:0/2869078188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:04:48.316 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.680+0000 7f7260fb7700 1 Processor -- start 2024-03-12T04:04:48.316 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.680+0000 7f7260fb7700 1 -- start start 2024-03-12T04:04:48.316 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.680+0000 7f7260fb7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c10a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:48.316 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.680+0000 7f7260fb7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c07adb0 con 0x7f725c108010 2024-03-12T04:04:48.316 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.680+0000 7f725a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c10a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:48.316 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.680+0000 7f725a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c10a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34172/0 (socket says 172.21.15.42:34172) 2024-03-12T04:04:48.316 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.680+0000 7f725a59c700 1 -- 172.21.15.42:0/3134218794 learned_addr learned my addr 172.21.15.42:0/3134218794 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.681+0000 7f725a59c700 1 -- 172.21.15.42:0/3134218794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c07aef0 con 0x7f725c108010 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.681+0000 7f725a59c700 1 --2- 172.21.15.42:0/3134218794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c10a400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f724c0098a0 tx=0x7f724c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65142b4709fb8773 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.681+0000 7f725959a700 1 -- 172.21.15.42:0/3134218794 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f724c004160 con 0x7f725c108010 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.681+0000 7f725959a700 1 -- 172.21.15.42:0/3134218794 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f724c0042c0 con 0x7f725c108010 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.681+0000 7f725959a700 1 -- 172.21.15.42:0/3134218794 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f724c011a50 con 0x7f725c108010 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.681+0000 7f7260fb7700 1 -- 172.21.15.42:0/3134218794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 msgr2=0x7f725c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.681+0000 7f7260fb7700 1 --2- 172.21.15.42:0/3134218794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c10a400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f724c0098a0 tx=0x7f724c009bb0 comp rx=0 tx=0).stop 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.682+0000 7f7260fb7700 1 -- 172.21.15.42:0/3134218794 shutdown_connections 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.682+0000 7f7260fb7700 1 --2- 172.21.15.42:0/3134218794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c10a400 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.682+0000 7f7260fb7700 1 -- 172.21.15.42:0/3134218794 >> 172.21.15.42:0/3134218794 conn(0x7f725c1019a0 msgr2=0x7f725c103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.682+0000 7f7260fb7700 1 -- 172.21.15.42:0/3134218794 shutdown_connections 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.682+0000 7f7260fb7700 1 -- 172.21.15.42:0/3134218794 wait complete. 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.682+0000 7f7260fb7700 1 Processor -- start 2024-03-12T04:04:48.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.682+0000 7f7260fb7700 1 -- start start 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f7260fb7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f7260fb7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c19b780 con 0x7f725c108010 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f725a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f725a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c19b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34176/0 (socket says 172.21.15.42:34176) 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f725a59c700 1 -- 172.21.15.42:0/2869078188 learned_addr learned my addr 172.21.15.42:0/2869078188 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f725a59c700 1 -- 172.21.15.42:0/2869078188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c009550 con 0x7f725c108010 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f725a59c700 1 --2- 172.21.15.42:0/2869078188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c19b240 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f724c003f40 tx=0x7f724c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f724b7fe700 1 -- 172.21.15.42:0/2869078188 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f724c003a40 con 0x7f725c108010 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f724b7fe700 1 -- 172.21.15.42:0/2869078188 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f724c003ba0 con 0x7f725c108010 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f724b7fe700 1 -- 172.21.15.42:0/2869078188 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f724c019710 con 0x7f725c108010 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c19b980 con 0x7f725c108010 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.683+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c19be20 con 0x7f725c108010 2024-03-12T04:04:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.684+0000 7f724b7fe700 1 -- 172.21.15.42:0/2869078188 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f724c02d430 con 0x7f725c108010 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.684+0000 7f724b7fe700 1 -- 172.21.15.42:0/2869078188 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f724c024a30 con 0x7f725c108010 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.684+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c10b670 con 0x7f725c108010 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.687+0000 7f724b7fe700 1 -- 172.21.15.42:0/2869078188 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f724c042dd0 con 0x7f725c108010 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.757+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f725c19e990 con 0x7f725c108010 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.757+0000 7f724b7fe700 1 -- 172.21.15.42:0/2869078188 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f724c031020 con 0x7f725c108010 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.758+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 msgr2=0x7f725c19b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.758+0000 7f7260fb7700 1 --2- 172.21.15.42:0/2869078188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c19b240 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f724c003f40 tx=0x7f724c011ab0 comp rx=0 tx=0).stop 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.758+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 shutdown_connections 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.758+0000 7f7260fb7700 1 --2- 172.21.15.42:0/2869078188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f725c108010 0x7f725c19b240 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.758+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 >> 172.21.15.42:0/2869078188 conn(0x7f725c1019a0 msgr2=0x7f725c102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.758+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 shutdown_connections 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:47.758+0000 7f7260fb7700 1 -- 172.21.15.42:0/2869078188 wait complete. 2024-03-12T04:04:48.319 INFO:teuthology.orchestra.run.smithi042.stdout:Restarting the monitor... 2024-03-12T04:04:48.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:48 smithi042 systemd[1]: Stopping Ceph mon.a for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:04:48.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:48 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a[31884]: 2024-03-12T04:04:48.576+0000 7fba747ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:04:48.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:48 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a[31884]: 2024-03-12T04:04:48.576+0000 7fba747ca700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-12T04:04:48.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:48 smithi042 podman[32133]: 2024-03-12 04:04:48.760925371 +0000 UTC m=+0.280229704 container died 1a1815f2ea6cd554d727582c47373fd99c4cfda68b0ce25040f3861cc480bd79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-12T04:04:49.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 podman[32133]: 2024-03-12 04:04:49.084769629 +0000 UTC m=+0.604073960 container cleanup 1a1815f2ea6cd554d727582c47373fd99c4cfda68b0ce25040f3861cc480bd79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-12T04:04:49.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 bash[32133]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a 2024-03-12T04:04:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 podman[32147]: 2024-03-12 04:04:49.25230724 +0000 UTC m=+0.493065886 container remove 1a1815f2ea6cd554d727582c47373fd99c4cfda68b0ce25040f3861cc480bd79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-12T04:04:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-12T04:04:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 systemd[1]: Stopped Ceph mon.a for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:04:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 systemd[1]: Starting Ceph mon.a for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 podman[32241]: 2024-03-12 04:04:49.580562249 +0000 UTC m=+0.017685973 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 podman[32241]: 2024-03-12 04:04:49.677927788 +0000 UTC m=+0.115051494 container create 67e30e1d77b0d260bf52b00c42780cb2061fc24f46d859b5542315ff0428cd3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 podman[32241]: 2024-03-12 04:04:49.897284814 +0000 UTC m=+0.334408527 container init 67e30e1d77b0d260bf52b00c42780cb2061fc24f46d859b5542315ff0428cd3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 podman[32241]: 2024-03-12 04:04:49.900084755 +0000 UTC m=+0.337208461 container start 67e30e1d77b0d260bf52b00c42780cb2061fc24f46d859b5542315ff0428cd3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, ceph=True) 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: set uid:gid to 167:167 (ceph:ceph) 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev), process ceph-mon, pid 2 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: pidfile_write: ignore empty --pid-file 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: load: jerasure load: lrc 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: RocksDB version: 7.9.2 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Git sha 0 2024-03-12T04:04:49.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Compile date 2024-03-08 02:48:09 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: DB SUMMARY 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: DB Session ID: A4ZR7RK0FCFEICX3ECYS 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: CURRENT file: CURRENT 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: IDENTITY file: IDENTITY 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99752 ; 2024-03-12T04:04:49.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.error_if_exists: 0 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.create_if_missing: 0 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.paranoid_checks: 1 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.env: 0x55671ff55ae0 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.fs: PosixFileSystem 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.info_log: 0x5567214798c0 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_file_opening_threads: 16 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.statistics: (nil) 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.use_fsync: 0 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_log_file_size: 0 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-12T04:04:49.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.keep_log_file_num: 1000 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.recycle_log_file_num: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.allow_fallocate: 1 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.allow_mmap_reads: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.allow_mmap_writes: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.use_direct_reads: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.create_missing_column_families: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.db_log_dir: 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.wal_dir: 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-12T04:04:49.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.advise_random_on_open: 1 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.db_write_buffer_size: 0 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.write_buffer_manager: 0x55672144d400 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.rate_limiter: (nil) 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.wal_recovery_mode: 2 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.enable_thread_tracking: 0 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.enable_pipelined_write: 0 2024-03-12T04:04:49.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.unordered_write: 0 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.row_cache: None 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.wal_filter: None 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.allow_ingest_behind: 0 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.two_write_queues: 0 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.manual_wal_flush: 0 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.wal_compression: 0 2024-03-12T04:04:49.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.atomic_flush: 0 2024-03-12T04:04:50.042 INFO:teuthology.orchestra.run.smithi042.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-12T04:04:50.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-12T04:04:50.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.log_readahead_size: 0 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.best_efforts_recovery: 0 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.allow_data_in_errors: 0 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.db_host_id: __hostname__ 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_background_jobs: 2 2024-03-12T04:04:50.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_background_compactions: -1 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_subcompactions: 1 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_total_wal_size: 0 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_open_files: -1 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bytes_per_sync: 0 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-12T04:04:50.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_readahead_size: 0 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_background_flushes: -1 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Compression algorithms supported: 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kZSTD supported: 0 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kXpressCompression supported: 0 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kLZ4HCCompression supported: 1 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kZlibCompression supported: 1 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kSnappyCompression supported: 1 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kLZ4Compression supported: 1 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: kBZip2Compression supported: 0 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-12T04:04:50.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.merge_operator: 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_filter: None 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_filter_factory: None 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.sst_partitioner_factory: None 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5567214799c0) 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks: 1 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_top_level_index_and_filter: 1 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_type: 0 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_index_type: 0 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_shortening: 1 2024-03-12T04:04:50.241 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: checksum: 4 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: no_block_cache: 0 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache: 0x55672147d090 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_name: BinnedLRUCache 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_options: 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: capacity : 536870912 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_shard_bits : 4 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: strict_capacity_limit : 0 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: high_pri_pool_ratio: 0.000 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_compressed: (nil) 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: persistent_cache: (nil) 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size: 4096 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size_deviation: 10 2024-03-12T04:04:50.242 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_restart_interval: 16 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_block_restart_interval: 1 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: metadata_block_size: 4096 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: partition_filters: 0 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: use_delta_encoding: 1 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: filter_policy: bloomfilter 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: whole_key_filtering: 1 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: verify_compression: 0 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: read_amp_bytes_per_bit: 0 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: format_version: 5 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_index_compression: 1 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_align: 0 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: max_auto_readahead_size: 262144 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: prepopulate_block_cache: 0 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: initial_auto_readahead_size: 8192 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.write_buffer_size: 33554432 2024-03-12T04:04:50.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_write_buffer_number: 2 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression: NoCompression 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression: Disabled 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.prefix_extractor: nullptr 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.num_levels: 7 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-12T04:04:50.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.level: 32767 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.strategy: 0 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.enabled: false 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-12T04:04:50.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.target_file_size_base: 67108864 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-12T04:04:50.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-12T04:04:50.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-12T04:04:50.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-12T04:04:50.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-12T04:04:50.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.arena_block_size: 1048576 2024-03-12T04:04:50.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-12T04:04:50.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-12T04:04:50.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.disable_auto_compactions: 0 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.table_properties_collectors: 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.inplace_update_support: 0 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-12T04:04:50.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.bloom_locality: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.max_successive_merges: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.paranoid_file_checks: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.force_consistency_checks: 1 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.report_bg_io_stats: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.ttl: 2592000 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.enable_blob_files: false 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.min_blob_size: 0 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.blob_file_size: 268435456 2024-03-12T04:04:50.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.blob_file_starting_level: 0 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0a27e2ee-7d5e-4515-8f3f-307f414d69d0 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710216289928735, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710216289929758, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94931, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92929, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13769, "raw_average_key_size": 50, "raw_value_size": 85444, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710216289, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0a27e2ee-7d5e-4515-8f3f-307f414d69d0", "db_session_id": "A4ZR7RK0FCFEICX3ECYS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-12T04:04:50.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710216289929841, "job": 1, "event": "recovery_finished"} 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55672156e000 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: DB pointer 0x556721558000 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** DB Stats ** 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-03-12T04:04:50.251 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: L0 2/0 94.54 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: Sum 2/0 94.54 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T04:04:50.252 INFO:journalctl@ceph.mon.a.smithi042.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative compaction: 0.00 GB write, 29.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval compaction: 0.00 GB write, 29.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache BinnedLRUCache@0x55672147d090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: starting mon.a rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???) e1 preinit fsid 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???).mds e1 new map 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???).mds e1 print_map 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: e1 2024-03-12T04:04:50.253 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout: legacy client fscid: -1 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout: No filesystems configured 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: monmap e1: 1 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: fsmap 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: osdmap e1: 0 total, 0 up, 0 in 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:49 smithi042 ceph-mon[32260]: mgrmap e1: no daemons active 2024-03-12T04:04:50.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:50 smithi042 bash[32241]: 67e30e1d77b0d260bf52b00c42780cb2061fc24f46d859b5542315ff0428cd3c 2024-03-12T04:04:50.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:50 smithi042 systemd[1]: Started Ceph mon.a for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:04:51.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:51 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2992911087' entity='client.admin' 2024-03-12T04:04:52.599 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.420+0000 7f8c6a763700 1 Processor -- start 2024-03-12T04:04:52.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.420+0000 7f8c6a763700 1 -- start start 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.420+0000 7f8c6a763700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6410a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.420+0000 7f8c6a763700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6407ae50 con 0x7f8c64107f90 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.421+0000 7f8c63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6410a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.421+0000 7f8c63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6410a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34192/0 (socket says 172.21.15.42:34192) 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.421+0000 7f8c63fff700 1 -- 172.21.15.42:0/2479045665 learned_addr learned my addr 172.21.15.42:0/2479045665 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.421+0000 7f8c63fff700 1 -- 172.21.15.42:0/2479045665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6410a8c0 con 0x7f8c64107f90 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.421+0000 7f8c63fff700 1 --2- 172.21.15.42:0/2479045665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6410a380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c540098a0 tx=0x7f8c54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4fbabad99b136b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c62ffd700 1 -- 172.21.15.42:0/2479045665 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c54004160 con 0x7f8c64107f90 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c62ffd700 1 -- 172.21.15.42:0/2479045665 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8c540042c0 con 0x7f8c64107f90 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c62ffd700 1 -- 172.21.15.42:0/2479045665 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c54011a50 con 0x7f8c64107f90 2024-03-12T04:04:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c6a763700 1 -- 172.21.15.42:0/2479045665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 msgr2=0x7f8c6410a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c6a763700 1 --2- 172.21.15.42:0/2479045665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6410a380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c540098a0 tx=0x7f8c54009bb0 comp rx=0 tx=0).stop 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c6a763700 1 -- 172.21.15.42:0/2479045665 shutdown_connections 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c6a763700 1 --2- 172.21.15.42:0/2479045665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6410a380 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c6a763700 1 -- 172.21.15.42:0/2479045665 >> 172.21.15.42:0/2479045665 conn(0x7f8c64101940 msgr2=0x7f8c64103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c6a763700 1 -- 172.21.15.42:0/2479045665 shutdown_connections 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.422+0000 7f8c6a763700 1 -- 172.21.15.42:0/2479045665 wait complete. 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.423+0000 7f8c6a763700 1 Processor -- start 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.423+0000 7f8c6a763700 1 -- start start 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.423+0000 7f8c6a763700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.423+0000 7f8c6a763700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419b790 con 0x7f8c64107f90 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.423+0000 7f8c63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.423+0000 7f8c63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6419b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34196/0 (socket says 172.21.15.42:34196) 2024-03-12T04:04:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.423+0000 7f8c63fff700 1 -- 172.21.15.42:0/2992911087 learned_addr learned my addr 172.21.15.42:0/2992911087 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c63fff700 1 -- 172.21.15.42:0/2992911087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c54009550 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c63fff700 1 --2- 172.21.15.42:0/2992911087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6419b250 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c54003f40 tx=0x7f8c54011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c617fa700 1 -- 172.21.15.42:0/2992911087 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c54003a40 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c617fa700 1 -- 172.21.15.42:0/2992911087 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8c54003ba0 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6419b990 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c617fa700 1 -- 172.21.15.42:0/2992911087 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c54019710 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6419be30 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c617fa700 1 -- 172.21.15.42:0/2992911087 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8c5402d430 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.424+0000 7f8c617fa700 1 -- 172.21.15.42:0/2992911087 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8c54024a40 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.425+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6410b5f0 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.427+0000 7f8c617fa700 1 -- 172.21.15.42:0/2992911087 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8c54019d60 con 0x7f8c64107f90 2024-03-12T04:04:52.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.498+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8c6410b7d0 con 0x7f8c64107f90 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.500+0000 7f8c617fa700 1 -- 172.21.15.42:0/2992911087 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8c5401abc0 con 0x7f8c64107f90 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.500+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 msgr2=0x7f8c6419b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.501+0000 7f8c6a763700 1 --2- 172.21.15.42:0/2992911087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6419b250 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c54003f40 tx=0x7f8c54011ab0 comp rx=0 tx=0).stop 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.501+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 shutdown_connections 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.501+0000 7f8c6a763700 1 --2- 172.21.15.42:0/2992911087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c64107f90 0x7f8c6419b250 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.501+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 >> 172.21.15.42:0/2992911087 conn(0x7f8c64101940 msgr2=0x7f8c64102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.501+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 shutdown_connections 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:50.501+0000 7f8c6a763700 1 -- 172.21.15.42:0/2992911087 wait complete. 2024-03-12T04:04:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-12T04:04:52.964 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:04:52.964 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mgr... 2024-03-12T04:04:52.965 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-12T04:04:52.966 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-12T04:04:53.887 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.x 2024-03-12T04:04:53.887 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.x.service: Unit ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.x.service not loaded. 2024-03-12T04:04:54.090 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7cca013a-e025-11ee-95c8-87774f69a715.target.wants/ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-7cca013a-e025-11ee-95c8-87774f69a715@.service. 2024-03-12T04:04:54.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:54 smithi042 podman[32477]: 2024-03-12 04:04:54.224851957 +0000 UTC m=+0.018464633 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:04:55.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:55 smithi042 podman[32477]: 2024-03-12 04:04:55.01946968 +0000 UTC m=+0.813082382 container create 58dfb605d35e894405fd599cdf53d3a84a4db3ff0c250d94dd5c611ecbceb75b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-12T04:04:57.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:56 smithi042 podman[32477]: 2024-03-12 04:04:56.826641997 +0000 UTC m=+2.620254660 container init 58dfb605d35e894405fd599cdf53d3a84a4db3ff0c250d94dd5c611ecbceb75b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-03-12T04:04:57.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:56 smithi042 podman[32477]: 2024-03-12 04:04:56.830449134 +0000 UTC m=+2.624061803 container start 58dfb605d35e894405fd599cdf53d3a84a4db3ff0c250d94dd5c611ecbceb75b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:04:57.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:57 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:57.424+0000 7f2a3451c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:04:57.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:57 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:57.519+0000 7f2a3451c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:04:57.819 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T04:04:57.820 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-03-12T04:04:57.828 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T04:04:57.828 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-03-12T04:04:57.828 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-03-12T04:04:57.828 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-03-12T04:04:57.837 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T04:04:57.838 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-03-12T04:04:57.846 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T04:04:57.846 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-03-12T04:04:57.846 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-03-12T04:04:57.846 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-12T04:04:57.846 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr to start... 2024-03-12T04:04:57.846 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr... 2024-03-12T04:04:58.065 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:57 smithi042 bash[32477]: 58dfb605d35e894405fd599cdf53d3a84a4db3ff0c250d94dd5c611ecbceb75b 2024-03-12T04:04:58.066 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:57 smithi042 systemd[1]: Started Ceph mgr.x for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:04:58.528 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:58.192+0000 7f2a3451c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:04:58.528 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:58.287+0000 7f2a3451c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:04:58.528 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:58.428+0000 7f2a3451c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:04:58.529 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:58.525+0000 7f2a3451c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:04:58.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:58.609+0000 7f2a3451c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:04:59.173 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:58.875+0000 7f2a3451c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:04:59.173 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:58.987+0000 7f2a3451c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:04:59.173 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:59.074+0000 7f2a3451c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:04:59.173 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:59.169+0000 7f2a3451c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:04:59.555 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:04:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:04:59.346+0000 7f2a3451c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:05:00.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:04:59 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3075850342' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T04:05:00.448 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:00.352+0000 7f2a3451c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T04:05:00.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T04:05:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T04:05:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T04:04:45.008603+0000", 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.768+0000 7fba9c4ce700 1 Processor -- start 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.769+0000 7fba9c4ce700 1 -- start start 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.769+0000 7fba9c4ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba94107d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.769+0000 7fba9c4ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba94108350 con 0x7fba94107960 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.769+0000 7fba9a26a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba94107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.769+0000 7fba9a26a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba94107d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43180/0 (socket says 172.21.15.42:43180) 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.769+0000 7fba9a26a700 1 -- 172.21.15.42:0/3704579153 learned_addr learned my addr 172.21.15.42:0/3704579153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.770+0000 7fba9a26a700 1 -- 172.21.15.42:0/3704579153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba941065c0 con 0x7fba94107960 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.770+0000 7fba9a26a700 1 --2- 172.21.15.42:0/3704579153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba94107d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba840098a0 tx=0x7fba84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65794a64e6229ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.770+0000 7fba99268700 1 -- 172.21.15.42:0/3704579153 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba84004160 con 0x7fba94107960 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.770+0000 7fba99268700 1 -- 172.21.15.42:0/3704579153 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fba840042c0 con 0x7fba94107960 2024-03-12T04:05:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.770+0000 7fba99268700 1 -- 172.21.15.42:0/3704579153 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba84011a50 con 0x7fba94107960 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3704579153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 msgr2=0x7fba94107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 --2- 172.21.15.42:0/3704579153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba94107d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba840098a0 tx=0x7fba84009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3704579153 shutdown_connections 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 --2- 172.21.15.42:0/3704579153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba94107d80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3704579153 >> 172.21.15.42:0/3704579153 conn(0x7fba94101110 msgr2=0x7fba94103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3704579153 shutdown_connections 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3704579153 wait complete. 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.771+0000 7fba9c4ce700 1 Processor -- start 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9c4ce700 1 -- start start 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9c4ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba9419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9c4ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9419b7d0 con 0x7fba94107960 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9a26a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba9419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9a26a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba9419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43186/0 (socket says 172.21.15.42:43186) 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9a26a700 1 -- 172.21.15.42:0/3075850342 learned_addr learned my addr 172.21.15.42:0/3075850342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:00.466 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9a26a700 1 -- 172.21.15.42:0/3075850342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84009550 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba9a26a700 1 --2- 172.21.15.42:0/3075850342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba9419b290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba84011710 tx=0x7fba84011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.772+0000 7fba8b7fe700 1 -- 172.21.15.42:0/3075850342 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba84003af0 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.773+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9419b9d0 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.773+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9419be70 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.773+0000 7fba8b7fe700 1 -- 172.21.15.42:0/3075850342 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fba84003c50 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.773+0000 7fba8b7fe700 1 -- 172.21.15.42:0/3075850342 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba840197a0 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.773+0000 7fba8b7fe700 1 -- 172.21.15.42:0/3075850342 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fba84025460 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.773+0000 7fba8b7fe700 1 -- 172.21.15.42:0/3075850342 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba8400b890 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.774+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba78005330 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.776+0000 7fba8b7fe700 1 -- 172.21.15.42:0/3075850342 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fba8402a060 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.849+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fba780058b0 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.850+0000 7fba8b7fe700 1 -- 172.21.15.42:0/3075850342 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fba84019e40 con 0x7fba94107960 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.850+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 msgr2=0x7fba9419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.850+0000 7fba9c4ce700 1 --2- 172.21.15.42:0/3075850342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba9419b290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba84011710 tx=0x7fba84011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:00.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.851+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 shutdown_connections 2024-03-12T04:05:00.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.851+0000 7fba9c4ce700 1 --2- 172.21.15.42:0/3075850342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba94107960 0x7fba9419b290 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:00.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.851+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 >> 172.21.15.42:0/3075850342 conn(0x7fba94101110 msgr2=0x7fba94102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:00.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.851+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 shutdown_connections 2024-03-12T04:05:00.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:04:59.851+0000 7fba9c4ce700 1 -- 172.21.15.42:0/3075850342 wait complete. 2024-03-12T04:05:00.468 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (1/15)... 2024-03-12T04:05:00.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:00.485+0000 7f2a3451c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:05:01.112 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:00.819+0000 7f2a3451c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:05:01.113 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:00.910+0000 7f2a3451c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:05:01.113 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:01.110+0000 7f2a3451c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:05:01.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:01.296+0000 7f2a3451c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:05:01.846 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:01.584+0000 7f2a3451c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:05:01.847 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:01.844+0000 7f2a3451c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:05:02.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:02 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:02.082+0000 7f2a3451c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:05:03.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2840555798' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T04:05:03.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:02 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:02.911+0000 7f2a3451c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:05:03.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:05:03.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T04:05:03.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T04:05:03.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T04:05:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T04:04:45.008603+0000", 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-03-12T04:05:03.495 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.814+0000 7f945d293700 1 Processor -- start 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.814+0000 7f945d293700 1 -- start start 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.814+0000 7f945d293700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f945810a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.814+0000 7f945d293700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945807ae50 con 0x7f9458107f60 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.814+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f945810a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.814+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f945810a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50760/0 (socket says 172.21.15.42:50760) 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.814+0000 7f9456ffd700 1 -- 172.21.15.42:0/2987225360 learned_addr learned my addr 172.21.15.42:0/2987225360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.815+0000 7f9456ffd700 1 -- 172.21.15.42:0/2987225360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945810a890 con 0x7f9458107f60 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.815+0000 7f9456ffd700 1 --2- 172.21.15.42:0/2987225360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f945810a350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f94480098a0 tx=0x7f9448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2097341dcb60cdc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.815+0000 7f94567fc700 1 -- 172.21.15.42:0/2987225360 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9448004160 con 0x7f9458107f60 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.815+0000 7f94567fc700 1 -- 172.21.15.42:0/2987225360 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f94480042c0 con 0x7f9458107f60 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.815+0000 7f94567fc700 1 -- 172.21.15.42:0/2987225360 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9448011a50 con 0x7f9458107f60 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 -- 172.21.15.42:0/2987225360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 msgr2=0x7f945810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 --2- 172.21.15.42:0/2987225360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f945810a350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f94480098a0 tx=0x7f9448009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 -- 172.21.15.42:0/2987225360 shutdown_connections 2024-03-12T04:05:03.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 --2- 172.21.15.42:0/2987225360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f945810a350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 -- 172.21.15.42:0/2987225360 >> 172.21.15.42:0/2987225360 conn(0x7f9458101990 msgr2=0x7f9458103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 -- 172.21.15.42:0/2987225360 shutdown_connections 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 -- 172.21.15.42:0/2987225360 wait complete. 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 Processor -- start 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.816+0000 7f945d293700 1 -- start start 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f945d293700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f94581077e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f945d293700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9458107d20 con 0x7f9458107f60 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f94581077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f94581077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50776/0 (socket says 172.21.15.42:50776) 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9456ffd700 1 -- 172.21.15.42:0/2840555798 learned_addr learned my addr 172.21.15.42:0/2840555798 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9456ffd700 1 -- 172.21.15.42:0/2840555798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9448009550 con 0x7f9458107f60 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9456ffd700 1 --2- 172.21.15.42:0/2840555798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f94581077e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9448003f40 tx=0x7f9448011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9454ff9700 1 -- 172.21.15.42:0/2840555798 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9448003a40 con 0x7f9458107f60 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9458105f10 con 0x7f9458107f60 2024-03-12T04:05:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9454ff9700 1 -- 172.21.15.42:0/2840555798 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9448003ba0 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f9454ff9700 1 -- 172.21.15.42:0/2840555798 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9448019710 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.817+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94581063b0 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.818+0000 7f9454ff9700 1 -- 172.21.15.42:0/2840555798 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f944802d430 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.818+0000 7f9454ff9700 1 -- 172.21.15.42:0/2840555798 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9448024a40 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.818+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94580632a0 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.821+0000 7f9454ff9700 1 -- 172.21.15.42:0/2840555798 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9448019d60 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.894+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f945810b840 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.894+0000 7f9454ff9700 1 -- 172.21.15.42:0/2840555798 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f9448024ca0 con 0x7f9458107f60 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.895+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 msgr2=0x7f94581077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.895+0000 7f945d293700 1 --2- 172.21.15.42:0/2840555798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f94581077e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9448003f40 tx=0x7f9448011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.895+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 shutdown_connections 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.896+0000 7f945d293700 1 --2- 172.21.15.42:0/2840555798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458107f60 0x7f94581077e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.896+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 >> 172.21.15.42:0/2840555798 conn(0x7f9458101990 msgr2=0x7f9458102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.896+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 shutdown_connections 2024-03-12T04:05:03.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:02.896+0000 7f945d293700 1 -- 172.21.15.42:0/2840555798 wait complete. 2024-03-12T04:05:03.499 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (2/15)... 2024-03-12T04:05:03.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:03.498+0000 7f2a3451c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:05:03.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:03.598+0000 7f2a3451c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:05:04.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:03.817+0000 7f2a3451c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:05:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: Activating manager daemon x 2024-03-12T04:05:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: mgrmap e2: x(active, starting, since 0.00312599s) 2024-03-12T04:05:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T04:05:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: Manager daemon x is now available 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' 2024-03-12T04:05:04.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:03 smithi042 ceph-mon[32260]: from='mgr.14100 172.21.15.42:0/1131614014' entity='mgr.x' 2024-03-12T04:05:06.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:05 smithi042 ceph-mon[32260]: mgrmap e3: x(active, since 1.00598s) 2024-03-12T04:05:08.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:07 smithi042 ceph-mon[32260]: mgrmap e4: x(active, since 3s) 2024-03-12T04:05:08.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T04:05:08.681 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T04:05:08.682 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T04:05:08.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T04:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-03-12T04:05:08.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T04:04:45.008603+0000", 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.828+0000 7fe996a82700 1 Processor -- start 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.828+0000 7fe996a82700 1 -- start start 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.828+0000 7fe996a82700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe9901061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.828+0000 7fe996a82700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99007ae10 con 0x7fe990105d90 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.829+0000 7fe99481e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe9901061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.829+0000 7fe99481e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe9901061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50844/0 (socket says 172.21.15.42:50844) 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.829+0000 7fe99481e700 1 -- 172.21.15.42:0/4077055362 learned_addr learned my addr 172.21.15.42:0/4077055362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:08.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.829+0000 7fe99481e700 1 -- 172.21.15.42:0/4077055362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9901066f0 con 0x7fe990105d90 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.829+0000 7fe99481e700 1 --2- 172.21.15.42:0/4077055362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe9901061b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098a0 tx=0x7fe984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bdcb04cfc203a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.829+0000 7fe98f7fe700 1 -- 172.21.15.42:0/4077055362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe984004160 con 0x7fe990105d90 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe98f7fe700 1 -- 172.21.15.42:0/4077055362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe9840042c0 con 0x7fe990105d90 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe98f7fe700 1 -- 172.21.15.42:0/4077055362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe984011a50 con 0x7fe990105d90 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe996a82700 1 -- 172.21.15.42:0/4077055362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 msgr2=0x7fe9901061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe996a82700 1 --2- 172.21.15.42:0/4077055362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe9901061b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098a0 tx=0x7fe984009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe996a82700 1 -- 172.21.15.42:0/4077055362 shutdown_connections 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe996a82700 1 --2- 172.21.15.42:0/4077055362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe9901061b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe996a82700 1 -- 172.21.15.42:0/4077055362 >> 172.21.15.42:0/4077055362 conn(0x7fe990101990 msgr2=0x7fe990103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe996a82700 1 -- 172.21.15.42:0/4077055362 shutdown_connections 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.830+0000 7fe996a82700 1 -- 172.21.15.42:0/4077055362 wait complete. 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe996a82700 1 Processor -- start 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe996a82700 1 -- start start 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe996a82700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe99019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:08.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe996a82700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019d960 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe99481e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe99019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe99481e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe99019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50860/0 (socket says 172.21.15.42:50860) 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe99481e700 1 -- 172.21.15.42:0/2408962498 learned_addr learned my addr 172.21.15.42:0/2408962498 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.831+0000 7fe99481e700 1 -- 172.21.15.42:0/2408962498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984009550 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.832+0000 7fe99481e700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe99019d420 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe984003f40 tx=0x7fe984011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.832+0000 7fe98dffb700 1 -- 172.21.15.42:0/2408962498 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe984003a40 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.832+0000 7fe98dffb700 1 -- 172.21.15.42:0/2408962498 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe984003ba0 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.832+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99019db60 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.832+0000 7fe98dffb700 1 -- 172.21.15.42:0/2408962498 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe984019710 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.832+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99019e000 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.833+0000 7fe98dffb700 1 -- 172.21.15.42:0/2408962498 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7fe984019980 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.833+0000 7fe98dffb700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.833+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9901970c0 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.833+0000 7fe98dffb700 1 -- 172.21.15.42:0/2408962498 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe98404ced0 con 0x7fe990105d90 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.833+0000 7fe98ffff700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.833+0000 7fe98ffff700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe97c006fd0 tx=0x7fe97c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:07.838+0000 7fe98dffb700 1 -- 172.21.15.42:0/2408962498 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe984018a40 con 0x7fe990105d90 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.131+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe990107560 con 0x7fe990105d90 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.132+0000 7fe98dffb700 1 -- 172.21.15.42:0/2408962498 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fe984035090 con 0x7fe990105d90 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 msgr2=0x7fe98003b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe97c006fd0 tx=0x7fe97c006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 msgr2=0x7fe99019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe99019d420 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe984003f40 tx=0x7fe984011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 shutdown_connections 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 --2- 172.21.15.42:0/2408962498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe990105d90 0x7fe99019d420 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 >> 172.21.15.42:0/2408962498 conn(0x7fe990101990 msgr2=0x7fe990194200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 shutdown_connections 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:08.134+0000 7fe996a82700 1 -- 172.21.15.42:0/2408962498 wait complete. 2024-03-12T04:05:08.696 INFO:teuthology.orchestra.run.smithi042.stdout:mgr is available 2024-03-12T04:05:09.037 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:08 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2408962498' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-12T04:05:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed3107700 1 Processor -- start 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed3107700 1 -- start start 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed3107700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc10a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed3107700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc07adf0 con 0x7faecc107fd0 2024-03-12T04:05:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed0ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc10a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed0ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc10a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50872/0 (socket says 172.21.15.42:50872) 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed0ea3700 1 -- 172.21.15.42:0/4293547170 learned_addr learned my addr 172.21.15.42:0/4293547170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed0ea3700 1 -- 172.21.15.42:0/4293547170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc07af30 con 0x7faecc107fd0 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.061+0000 7faed0ea3700 1 --2- 172.21.15.42:0/4293547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc10a3c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faec00098a0 tx=0x7faec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd4013c4feb0559 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faecb7fe700 1 -- 172.21.15.42:0/4293547170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec0004160 con 0x7faecc107fd0 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faecb7fe700 1 -- 172.21.15.42:0/4293547170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7faec00042c0 con 0x7faecc107fd0 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faecb7fe700 1 -- 172.21.15.42:0/4293547170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec0011a50 con 0x7faecc107fd0 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faed3107700 1 -- 172.21.15.42:0/4293547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 msgr2=0x7faecc10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faed3107700 1 --2- 172.21.15.42:0/4293547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc10a3c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faec00098a0 tx=0x7faec0009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faed3107700 1 -- 172.21.15.42:0/4293547170 shutdown_connections 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faed3107700 1 --2- 172.21.15.42:0/4293547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc10a3c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faed3107700 1 -- 172.21.15.42:0/4293547170 >> 172.21.15.42:0/4293547170 conn(0x7faecc1019a0 msgr2=0x7faecc103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faed3107700 1 -- 172.21.15.42:0/4293547170 shutdown_connections 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.062+0000 7faed3107700 1 -- 172.21.15.42:0/4293547170 wait complete. 2024-03-12T04:05:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed3107700 1 Processor -- start 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed3107700 1 -- start start 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed3107700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed3107700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc19b760 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed0ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed0ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc19b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50882/0 (socket says 172.21.15.42:50882) 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed0ea3700 1 -- 172.21.15.42:0/1412481488 learned_addr learned my addr 172.21.15.42:0/1412481488 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed0ea3700 1 -- 172.21.15.42:0/1412481488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0009550 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.063+0000 7faed0ea3700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc19b220 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faec0003f40 tx=0x7faec0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.064+0000 7faec9ffb700 1 -- 172.21.15.42:0/1412481488 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec0003a40 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.064+0000 7faec9ffb700 1 -- 172.21.15.42:0/1412481488 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7faec0003ba0 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.064+0000 7faec9ffb700 1 -- 172.21.15.42:0/1412481488 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec0019710 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.064+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc19b960 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.064+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc19be00 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.065+0000 7faec9ffb700 1 -- 172.21.15.42:0/1412481488 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7faec001abc0 con 0x7faecc107fd0 2024-03-12T04:05:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.065+0000 7faec9ffb700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7faebc038c60 0x7faebc03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.065+0000 7faec9ffb700 1 -- 172.21.15.42:0/1412481488 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faec004cf30 con 0x7faecc107fd0 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.065+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc10b630 con 0x7faecc107fd0 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.065+0000 7faecbfff700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7faebc038c60 0x7faebc03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.065+0000 7faecbfff700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7faebc038c60 0x7faebc03b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faeb8006fd0 tx=0x7faeb8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.070+0000 7faec9ffb700 1 -- 172.21.15.42:0/1412481488 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faec002d430 con 0x7faecc107fd0 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.270+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7faecc10b7f0 con 0x7faecc107fd0 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.271+0000 7faec9ffb700 1 -- 172.21.15.42:0/1412481488 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7faec004a020 con 0x7faecc107fd0 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7faebc038c60 msgr2=0x7faebc03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7faebc038c60 0x7faebc03b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faeb8006fd0 tx=0x7faeb8006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 msgr2=0x7faecc19b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc19b220 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faec0003f40 tx=0x7faec0011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 shutdown_connections 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7faebc038c60 0x7faebc03b120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 --2- 172.21.15.42:0/1412481488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc107fd0 0x7faecc19b220 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 >> 172.21.15.42:0/1412481488 conn(0x7faecc1019a0 msgr2=0x7faecc102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:09.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 shutdown_connections 2024-03-12T04:05:09.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:09.273+0000 7faed3107700 1 -- 172.21.15.42:0/1412481488 wait complete. 2024-03-12T04:05:09.838 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling cephadm module... 2024-03-12T04:05:09.953 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:09 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1412481488' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-12T04:05:11.163 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:10 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3414094568' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-12T04:05:11.163 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: ignoring --setuser ceph since I am not root 2024-03-12T04:05:11.164 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: ignoring --setgroup ceph since I am not root 2024-03-12T04:05:11.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.217+0000 7f6e23b39700 1 Processor -- start 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.217+0000 7f6e23b39700 1 -- start start 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.217+0000 7f6e23b39700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c1078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.217+0000 7f6e23b39700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c107ea0 con 0x7f6e1c1074b0 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.217+0000 7f6e218d5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c1078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.217+0000 7f6e218d5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c1078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50890/0 (socket says 172.21.15.42:50890) 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.217+0000 7f6e218d5700 1 -- 172.21.15.42:0/60908081 learned_addr learned my addr 172.21.15.42:0/60908081 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e218d5700 1 -- 172.21.15.42:0/60908081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c108660 con 0x7f6e1c1074b0 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e218d5700 1 --2- 172.21.15.42:0/60908081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c1078d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e180098a0 tx=0x7f6e18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec99b8df0e0f1de server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e208d3700 1 -- 172.21.15.42:0/60908081 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e18004160 con 0x7f6e1c1074b0 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e208d3700 1 -- 172.21.15.42:0/60908081 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f6e180042c0 con 0x7f6e1c1074b0 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e208d3700 1 -- 172.21.15.42:0/60908081 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e18011a50 con 0x7f6e1c1074b0 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e23b39700 1 -- 172.21.15.42:0/60908081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 msgr2=0x7f6e1c1078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:11.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e23b39700 1 --2- 172.21.15.42:0/60908081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c1078d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e180098a0 tx=0x7f6e18009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e23b39700 1 -- 172.21.15.42:0/60908081 shutdown_connections 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e23b39700 1 --2- 172.21.15.42:0/60908081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c1078d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.218+0000 7f6e23b39700 1 -- 172.21.15.42:0/60908081 >> 172.21.15.42:0/60908081 conn(0x7f6e1c102b10 msgr2=0x7f6e1c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e23b39700 1 -- 172.21.15.42:0/60908081 shutdown_connections 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e23b39700 1 -- 172.21.15.42:0/60908081 wait complete. 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e23b39700 1 Processor -- start 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e23b39700 1 -- start start 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e23b39700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c07cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e23b39700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c07d090 con 0x7f6e1c1074b0 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e218d5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c07cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e218d5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c07cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50896/0 (socket says 172.21.15.42:50896) 2024-03-12T04:05:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.219+0000 7f6e218d5700 1 -- 172.21.15.42:0/3414094568 learned_addr learned my addr 172.21.15.42:0/3414094568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.220+0000 7f6e218d5700 1 -- 172.21.15.42:0/3414094568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18009550 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.220+0000 7f6e218d5700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c07cb50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e18011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.220+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e18003a40 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.220+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f6e18003ba0 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.220+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c07b280 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.220+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e1801a700 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.220+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c07b720 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.221+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f6e18024440 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.221+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c0632a0 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.221+0000 7f6e12ffd700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f6e08038c60 0x7f6e0803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.221+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e1804ca90 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.221+0000 7f6e210d4700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f6e08038c60 0x7f6e0803b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.222+0000 7f6e210d4700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f6e08038c60 0x7f6e0803b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006fd0 tx=0x7f6e0c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.226+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e18018a40 con 0x7f6e1c1074b0 2024-03-12T04:05:11.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.485+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6e1c07bcf0 con 0x7f6e1c1074b0 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.843+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6e18018350 con 0x7f6e1c1074b0 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.844+0000 7f6e12ffd700 1 -- 172.21.15.42:0/3414094568 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f6e1802d8c0 con 0x7f6e1c1074b0 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f6e08038c60 msgr2=0x7f6e0803b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f6e08038c60 0x7f6e0803b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006fd0 tx=0x7f6e0c006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 msgr2=0x7f6e1c07cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c07cb50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e18011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 shutdown_connections 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f6e08038c60 0x7f6e0803b120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 --2- 172.21.15.42:0/3414094568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e1c1074b0 0x7f6e1c07cb50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 >> 172.21.15.42:0/3414094568 conn(0x7f6e1c102b10 msgr2=0x7f6e1c103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 shutdown_connections 2024-03-12T04:05:11.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:10.846+0000 7f6e23b39700 1 -- 172.21.15.42:0/3414094568 wait complete. 2024-03-12T04:05:11.673 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:11.419+0000 7f59544e5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:05:11.673 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:11.519+0000 7f59544e5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:05:12.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:11 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3414094568' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-12T04:05:12.056 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:11 smithi042 ceph-mon[32260]: mgrmap e5: x(active, since 7s) 2024-03-12T04:05:12.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:12.193+0000 7f59544e5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:05:12.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:12.288+0000 7f59544e5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:05:12.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-03-12T04:05:12.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-12T04:05:12.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T04:05:12.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-12T04:05:12.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-12T04:05:12.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.846+0000 7f3b1e6c9700 1 Processor -- start 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b1e6c9700 1 -- start start 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b1e6c9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b18106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b1e6c9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18106d40 con 0x7f3b18106350 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b17fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b18106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b17fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b18106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45336/0 (socket says 172.21.15.42:45336) 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b17fff700 1 -- 172.21.15.42:0/4083779184 learned_addr learned my addr 172.21.15.42:0/4083779184 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b17fff700 1 -- 172.21.15.42:0/4083779184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18107550 con 0x7f3b18106350 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.847+0000 7f3b17fff700 1 --2- 172.21.15.42:0/4083779184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b18106770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b080098a0 tx=0x7f3b08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e50089cb793284 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b16ffd700 1 -- 172.21.15.42:0/4083779184 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b08004160 con 0x7f3b18106350 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b16ffd700 1 -- 172.21.15.42:0/4083779184 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3b080042c0 con 0x7f3b18106350 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b16ffd700 1 -- 172.21.15.42:0/4083779184 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b08011a50 con 0x7f3b18106350 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/4083779184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 msgr2=0x7f3b18106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b1e6c9700 1 --2- 172.21.15.42:0/4083779184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b18106770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b080098a0 tx=0x7f3b08009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/4083779184 shutdown_connections 2024-03-12T04:05:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b1e6c9700 1 --2- 172.21.15.42:0/4083779184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b18106770 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.848+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/4083779184 >> 172.21.15.42:0/4083779184 conn(0x7f3b181018f0 msgr2=0x7f3b18103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.849+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/4083779184 shutdown_connections 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.849+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/4083779184 wait complete. 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.849+0000 7f3b1e6c9700 1 Processor -- start 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.849+0000 7f3b1e6c9700 1 -- start start 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b1e6c9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b181913b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b1e6c9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b181918f0 con 0x7f3b18106350 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b17fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b181913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b17fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b181913b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45352/0 (socket says 172.21.15.42:45352) 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b17fff700 1 -- 172.21.15.42:0/2217117990 learned_addr learned my addr 172.21.15.42:0/2217117990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b17fff700 1 -- 172.21.15.42:0/2217117990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009550 con 0x7f3b18106350 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b17fff700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b181913b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006af0 tx=0x7f3b08011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b157fa700 1 -- 172.21.15.42:0/2217117990 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b08003a30 con 0x7f3b18106350 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b157fa700 1 -- 172.21.15.42:0/2217117990 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3b08003b90 con 0x7f3b18106350 2024-03-12T04:05:12.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b157fa700 1 -- 172.21.15.42:0/2217117990 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b08019750 con 0x7f3b18106350 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1818fb30 con 0x7f3b18106350 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.850+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1818ffd0 con 0x7f3b18106350 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.851+0000 7f3b157fa700 1 -- 172.21.15.42:0/2217117990 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f3b080199c0 con 0x7f3b18106350 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.851+0000 7f3b157fa700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 0x7f3b0003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.851+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b181909e0 con 0x7f3b18106350 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.851+0000 7f3b177fe700 1 -- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 msgr2=0x7f3b0003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2755197547 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.851+0000 7f3b157fa700 1 -- 172.21.15.42:0/2217117990 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3b0804dbc0 con 0x7f3b18106350 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.852+0000 7f3b177fe700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 0x7f3b0003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:11.857+0000 7f3b157fa700 1 -- 172.21.15.42:0/2217117990 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b08023370 con 0x7f3b18106350 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.053+0000 7f3b177fe700 1 -- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 msgr2=0x7f3b0003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2755197547 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.053+0000 7f3b177fe700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 0x7f3b0003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T04:05:12.650 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.117+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3b18194df0 con 0x7f3b18106350 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.117+0000 7f3b157fa700 1 -- 172.21.15.42:0/2217117990 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3b08010bc0 con 0x7f3b18106350 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 msgr2=0x7f3b0003b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 0x7f3b0003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 msgr2=0x7f3b181913b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b181913b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006af0 tx=0x7f3b08011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 shutdown_connections 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7f3b00038d00 0x7f3b0003b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 --2- 172.21.15.42:0/2217117990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b18106350 0x7f3b181913b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 >> 172.21.15.42:0/2217117990 conn(0x7f3b181018f0 msgr2=0x7f3b181028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 shutdown_connections 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:12.119+0000 7f3b1e6c9700 1 -- 172.21.15.42:0/2217117990 wait complete. 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-03-12T04:05:12.651 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 5... 2024-03-12T04:05:12.754 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:12.433+0000 7f59544e5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:05:12.754 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:12.531+0000 7f59544e5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:05:12.754 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:12.618+0000 7f59544e5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:05:13.008 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:12 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2217117990' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-12T04:05:13.008 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:12.891+0000 7f59544e5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:05:13.008 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:13.005+0000 7f59544e5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:05:13.270 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:13.096+0000 7f59544e5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:05:13.271 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:13.193+0000 7f59544e5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:05:13.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:13.371+0000 7f59544e5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:05:14.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:14.361+0000 7f59544e5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:05:14.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:14.497+0000 7f59544e5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:05:15.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:14.835+0000 7f59544e5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:05:15.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:14.927+0000 7f59544e5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:05:15.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:15.131+0000 7f59544e5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:05:15.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:15.323+0000 7f59544e5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:05:15.914 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:15.623+0000 7f59544e5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:05:16.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:15.908+0000 7f59544e5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:05:16.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:16 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:16.151+0000 7f59544e5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:05:17.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:16 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:16.979+0000 7f59544e5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:05:17.917 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:17.588+0000 7f59544e5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:05:17.917 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:17.689+0000 7f59544e5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:05:17.917 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:17.914+0000 7f59544e5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:05:18.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: Active manager daemon x restarted 2024-03-12T04:05:18.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: Activating manager daemon x 2024-03-12T04:05:18.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: osdmap e2: 0 total, 0 up, 0 in 2024-03-12T04:05:18.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: mgrmap e6: x(active, starting, since 0.00321665s) 2024-03-12T04:05:18.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:05:18.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: Manager daemon x is now available 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:18.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:17 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:19.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:18 smithi042 ceph-mon[32260]: Found migration_current of "None". Setting to last migration. 2024-03-12T04:05:19.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:18 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:19.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:18 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T04:05:19.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:18 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T04:05:19.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:18 smithi042 ceph-mon[32260]: mgrmap e7: x(active, since 1.00543s) 2024-03-12T04:05:19.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.062+0000 7fe99d159700 1 Processor -- start 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.062+0000 7fe99d159700 1 -- start start 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.062+0000 7fe99d159700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99810a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.062+0000 7fe99d159700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99807ae90 con 0x7fe998107ef0 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe996d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99810a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe996d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99810a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45362/0 (socket says 172.21.15.42:45362) 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe996d9d700 1 -- 172.21.15.42:0/1160801729 learned_addr learned my addr 172.21.15.42:0/1160801729 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe996d9d700 1 -- 172.21.15.42:0/1160801729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99810a820 con 0x7fe998107ef0 2024-03-12T04:05:19.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe996d9d700 1 --2- 172.21.15.42:0/1160801729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99810a2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c857a66ec386545 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe995d9b700 1 -- 172.21.15.42:0/1160801729 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c004160 con 0x7fe998107ef0 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe995d9b700 1 -- 172.21.15.42:0/1160801729 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe98c0042c0 con 0x7fe998107ef0 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.063+0000 7fe995d9b700 1 -- 172.21.15.42:0/1160801729 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c011a50 con 0x7fe998107ef0 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.064+0000 7fe99d159700 1 -- 172.21.15.42:0/1160801729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 msgr2=0x7fe99810a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.064+0000 7fe99d159700 1 --2- 172.21.15.42:0/1160801729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99810a2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.064+0000 7fe99d159700 1 -- 172.21.15.42:0/1160801729 shutdown_connections 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.064+0000 7fe99d159700 1 --2- 172.21.15.42:0/1160801729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99810a2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.064+0000 7fe99d159700 1 -- 172.21.15.42:0/1160801729 >> 172.21.15.42:0/1160801729 conn(0x7fe998101900 msgr2=0x7fe998103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.064+0000 7fe99d159700 1 -- 172.21.15.42:0/1160801729 shutdown_connections 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.064+0000 7fe99d159700 1 -- 172.21.15.42:0/1160801729 wait complete. 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe99d159700 1 Processor -- start 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe99d159700 1 -- start start 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe99d159700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe99d159700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99819b7a0 con 0x7fe998107ef0 2024-03-12T04:05:19.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe996d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe996d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99819b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45368/0 (socket says 172.21.15.42:45368) 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe996d9d700 1 -- 172.21.15.42:0/4105912961 learned_addr learned my addr 172.21.15.42:0/4105912961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.065+0000 7fe996d9d700 1 -- 172.21.15.42:0/4105912961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009550 con 0x7fe998107ef0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.066+0000 7fe996d9d700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99819b260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe98c003f40 tx=0x7fe98c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.066+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c003a40 con 0x7fe998107ef0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.066+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99819b9a0 con 0x7fe998107ef0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.066+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99819be40 con 0x7fe998107ef0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.066+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe98c003ba0 con 0x7fe998107ef0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.066+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c019670 con 0x7fe998107ef0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.067+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe98c0198f0 con 0x7fe998107ef0 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.067+0000 7fe97ffff700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.067+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 --> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe98003b830 con 0x7fe980038c60 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.067+0000 7fe99659c700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 msgr2=0x7fe98003b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2755197547 2024-03-12T04:05:19.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.067+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe98c04dea0 con 0x7fe998107ef0 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.067+0000 7fe99659c700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.268+0000 7fe99659c700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 msgr2=0x7fe98003b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2755197547 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.268+0000 7fe99659c700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.669+0000 7fe99659c700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 msgr2=0x7fe98003b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2755197547 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:13.669+0000 7fe99659c700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:14.470+0000 7fe99659c700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 msgr2=0x7fe98003b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2755197547 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:14.470+0000 7fe99659c700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:16.071+0000 7fe99659c700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 msgr2=0x7fe98003b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2755197547 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:16.071+0000 7fe99659c700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:17.929+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 6) v1 ==== 45536+0+0 (secure 0 0 0) 0x7fe98c02da00 con 0x7fe998107ef0 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:17.929+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 msgr2=0x7fe98003b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:17.929+0000 7fe97ffff700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.930+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7fe98c017d20 con 0x7fe998107ef0 2024-03-12T04:05:19.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.930+0000 7fe97ffff700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe98003c550 0x7fe98003e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.930+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe98003b830 con 0x7fe98003c550 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.930+0000 7fe99659c700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe98003c550 0x7fe98003e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.931+0000 7fe99659c700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe98003c550 0x7fe98003e940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe988003a10 tx=0x7fe988009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.932+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fe98003b830 con 0x7fe98003c550 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.938+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe984002630 con 0x7fe98003c550 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe97ffff700 1 -- 172.21.15.42:0/4105912961 <== mgr.14118 v2:172.21.15.42:6800/2670488995 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe984002630 con 0x7fe98003c550 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe98003c550 msgr2=0x7fe98003e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe98003c550 0x7fe98003e940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe988003a10 tx=0x7fe988009290 comp rx=0 tx=0).stop 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 msgr2=0x7fe99819b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99819b260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe98c003f40 tx=0x7fe98c011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 shutdown_connections 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe998107ef0 0x7fe99819b260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe98003c550 0x7fe98003e940 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:19.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 --2- 172.21.15.42:0/4105912961 >> [v2:172.21.15.42:6800/2755197547,v1:172.21.15.42:6801/2755197547] conn(0x7fe980038c60 0x7fe98003b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:19.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 >> 172.21.15.42:0/4105912961 conn(0x7fe998101900 msgr2=0x7fe998102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:19.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 shutdown_connections 2024-03-12T04:05:19.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:18.939+0000 7fe99d159700 1 -- 172.21.15.42:0/4105912961 wait complete. 2024-03-12T04:05:19.492 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 5 is available 2024-03-12T04:05:19.492 INFO:teuthology.orchestra.run.smithi042.stdout:Setting orchestrator backend to cephadm... 2024-03-12T04:05:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:20 smithi042 ceph-mon[32260]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-12T04:05:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:20 smithi042 ceph-mon[32260]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-12T04:05:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:20 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:20 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:20 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:20 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:20 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:20.848 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.893+0000 7f3e89e45700 1 Processor -- start 2024-03-12T04:05:20.848 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.893+0000 7f3e89e45700 1 -- start start 2024-03-12T04:05:20.848 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e89e45700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e84106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e89e45700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e84107290 con 0x7f3e841068a0 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e84106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e84106cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45442/0 (socket says 172.21.15.42:45442) 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e837fe700 1 -- 172.21.15.42:0/2141656302 learned_addr learned my addr 172.21.15.42:0/2141656302 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e837fe700 1 -- 172.21.15.42:0/2141656302 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e841075c0 con 0x7f3e841068a0 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e837fe700 1 --2- 172.21.15.42:0/2141656302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e84106cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098a0 tx=0x7f3e74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ff0b5668874637 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e827fc700 1 -- 172.21.15.42:0/2141656302 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74004160 con 0x7f3e841068a0 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e827fc700 1 -- 172.21.15.42:0/2141656302 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3e740042c0 con 0x7f3e841068a0 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.894+0000 7f3e827fc700 1 -- 172.21.15.42:0/2141656302 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74011a50 con 0x7f3e841068a0 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.895+0000 7f3e89e45700 1 -- 172.21.15.42:0/2141656302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 msgr2=0x7f3e84106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.895+0000 7f3e89e45700 1 --2- 172.21.15.42:0/2141656302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e84106cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098a0 tx=0x7f3e74009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:20.849 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.895+0000 7f3e89e45700 1 -- 172.21.15.42:0/2141656302 shutdown_connections 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.895+0000 7f3e89e45700 1 --2- 172.21.15.42:0/2141656302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e84106cc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.895+0000 7f3e89e45700 1 -- 172.21.15.42:0/2141656302 >> 172.21.15.42:0/2141656302 conn(0x7f3e84101140 msgr2=0x7f3e84103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.895+0000 7f3e89e45700 1 -- 172.21.15.42:0/2141656302 shutdown_connections 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.895+0000 7f3e89e45700 1 -- 172.21.15.42:0/2141656302 wait complete. 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e89e45700 1 Processor -- start 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e89e45700 1 -- start start 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e89e45700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e8419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e89e45700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8419b7d0 con 0x7f3e841068a0 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e8419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e8419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45450/0 (socket says 172.21.15.42:45450) 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e837fe700 1 -- 172.21.15.42:0/3086724088 learned_addr learned my addr 172.21.15.42:0/3086724088 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:20.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e837fe700 1 -- 172.21.15.42:0/3086724088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74009550 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.896+0000 7f3e837fe700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e8419b290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3e74003f40 tx=0x7f3e74011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.897+0000 7f3e80ff9700 1 -- 172.21.15.42:0/3086724088 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74003a40 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.897+0000 7f3e80ff9700 1 -- 172.21.15.42:0/3086724088 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3e74003ba0 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.897+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8419b9d0 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.897+0000 7f3e80ff9700 1 -- 172.21.15.42:0/3086724088 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74019710 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.897+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8419be70 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.898+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e84195370 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.898+0000 7f3e80ff9700 1 -- 172.21.15.42:0/3086724088 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7f3e7401a430 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.898+0000 7f3e80ff9700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f3e6c038ba0 0x7f3e6c03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.898+0000 7f3e80ff9700 1 -- 172.21.15.42:0/3086724088 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3e7404ca20 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.898+0000 7f3e82ffd700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f3e6c038ba0 0x7f3e6c03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.899+0000 7f3e82ffd700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f3e6c038ba0 0x7f3e6c03b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e78006fd0 tx=0x7f3e78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:19.907+0000 7f3e80ff9700 1 -- 172.21.15.42:0/3086724088 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e74023750 con 0x7f3e841068a0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.127+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e8402da40 con 0x7f3e6c038ba0 2024-03-12T04:05:20.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.131+0000 7f3e80ff9700 1 -- 172.21.15.42:0/3086724088 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3e8402da40 con 0x7f3e6c038ba0 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.133+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f3e6c038ba0 msgr2=0x7f3e6c03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f3e6c038ba0 0x7f3e6c03b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e78006fd0 tx=0x7f3e78006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 msgr2=0x7f3e8419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e8419b290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3e74003f40 tx=0x7f3e74011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 shutdown_connections 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f3e6c038ba0 0x7f3e6c03b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 --2- 172.21.15.42:0/3086724088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e841068a0 0x7f3e8419b290 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 >> 172.21.15.42:0/3086724088 conn(0x7f3e84101140 msgr2=0x7f3e84102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 shutdown_connections 2024-03-12T04:05:20.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:20.134+0000 7f3e89e45700 1 -- 172.21.15.42:0/3086724088 wait complete. 2024-03-12T04:05:21.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:21 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:19] ENGINE Bus STARTING 2024-03-12T04:05:21.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:21 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:20] ENGINE Serving on https://172.21.15.42:7150 2024-03-12T04:05:21.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:21 smithi042 ceph-mon[32260]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:21.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:21 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:20] ENGINE Serving on http://172.21.15.42:8765 2024-03-12T04:05:21.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:21 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:20] ENGINE Bus STARTED 2024-03-12T04:05:21.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:21 smithi042 ceph-mon[32260]: mgrmap e8: x(active, since 2s) 2024-03-12T04:05:22.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-12T04:05:22.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.276+0000 7fe44b1ed700 1 Processor -- start 2024-03-12T04:05:22.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.276+0000 7fe44b1ed700 1 -- start start 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe44b1ed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe444108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe44b1ed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe444109570 con 0x7fe444108b80 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe448f89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe444108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe448f89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe444108fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45462/0 (socket says 172.21.15.42:45462) 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe448f89700 1 -- 172.21.15.42:0/265642569 learned_addr learned my addr 172.21.15.42:0/265642569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe448f89700 1 -- 172.21.15.42:0/265642569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4441053d0 con 0x7fe444108b80 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe448f89700 1 --2- 172.21.15.42:0/265642569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe444108fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098a0 tx=0x7fe438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3229ea71f319624 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe4437fe700 1 -- 172.21.15.42:0/265642569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe438004160 con 0x7fe444108b80 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe4437fe700 1 -- 172.21.15.42:0/265642569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe4380042c0 con 0x7fe444108b80 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.277+0000 7fe4437fe700 1 -- 172.21.15.42:0/265642569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe438011a50 con 0x7fe444108b80 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.278+0000 7fe44b1ed700 1 -- 172.21.15.42:0/265642569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 msgr2=0x7fe444108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.278+0000 7fe44b1ed700 1 --2- 172.21.15.42:0/265642569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe444108fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098a0 tx=0x7fe438009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.278+0000 7fe44b1ed700 1 -- 172.21.15.42:0/265642569 shutdown_connections 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.278+0000 7fe44b1ed700 1 --2- 172.21.15.42:0/265642569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe444108fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.278+0000 7fe44b1ed700 1 -- 172.21.15.42:0/265642569 >> 172.21.15.42:0/265642569 conn(0x7fe444101050 msgr2=0x7fe4441034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.278+0000 7fe44b1ed700 1 -- 172.21.15.42:0/265642569 shutdown_connections 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.278+0000 7fe44b1ed700 1 -- 172.21.15.42:0/265642569 wait complete. 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe44b1ed700 1 Processor -- start 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe44b1ed700 1 -- start start 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe44b1ed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe44419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe44b1ed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44419b890 con 0x7fe444108b80 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe448f89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe44419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe448f89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe44419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45468/0 (socket says 172.21.15.42:45468) 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe448f89700 1 -- 172.21.15.42:0/1201875880 learned_addr learned my addr 172.21.15.42:0/1201875880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe448f89700 1 -- 172.21.15.42:0/1201875880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe438009550 con 0x7fe444108b80 2024-03-12T04:05:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.279+0000 7fe448f89700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe44419b350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe438006af0 tx=0x7fe438011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.280+0000 7fe441ffb700 1 -- 172.21.15.42:0/1201875880 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe438003930 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.280+0000 7fe441ffb700 1 -- 172.21.15.42:0/1201875880 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe438003a90 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.280+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44419ba90 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.280+0000 7fe441ffb700 1 -- 172.21.15.42:0/1201875880 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe438019670 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.280+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44419bf30 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.281+0000 7fe441ffb700 1 -- 172.21.15.42:0/1201875880 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe4380198e0 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.281+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4440632a0 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.281+0000 7fe441ffb700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe434038cb0 0x7fe43403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.281+0000 7fe441ffb700 1 -- 172.21.15.42:0/1201875880 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe43804d0a0 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.281+0000 7fe443fff700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe434038cb0 0x7fe43403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.281+0000 7fe443fff700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe434038cb0 0x7fe43403b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe430006fd0 tx=0x7fe430006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.286+0000 7fe441ffb700 1 -- 172.21.15.42:0/1201875880 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe438018530 con 0x7fe444108b80 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.490+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe44410b8c0 con 0x7fe434038cb0 2024-03-12T04:05:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.490+0000 7fe441ffb700 1 -- 172.21.15.42:0/1201875880 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe44410b8c0 con 0x7fe434038cb0 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe434038cb0 msgr2=0x7fe43403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe434038cb0 0x7fe43403b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe430006fd0 tx=0x7fe430006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 msgr2=0x7fe44419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe44419b350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe438006af0 tx=0x7fe438011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 shutdown_connections 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe434038cb0 0x7fe43403b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 --2- 172.21.15.42:0/1201875880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe444108b80 0x7fe44419b350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 >> 172.21.15.42:0/1201875880 conn(0x7fe444101050 msgr2=0x7fe444101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 shutdown_connections 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:21.493+0000 7fe44b1ed700 1 -- 172.21.15.42:0/1201875880 wait complete. 2024-03-12T04:05:22.076 INFO:teuthology.orchestra.run.smithi042.stdout:Generating ssh key... 2024-03-12T04:05:22.730 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:22 smithi042 ceph-mon[32260]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:23.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-mon[32260]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:23.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-mon[32260]: Generating ssh key... 2024-03-12T04:05:23.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:23.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:23.670 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: Generating public/private rsa key pair. 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: Your identification has been saved in /tmp/tmpxfmsbfsr/key. 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: Your public key has been saved in /tmp/tmpxfmsbfsr/key.pub. 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: The key fingerprint is: 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: SHA256:5RGqheQiTBTBkrPZzbm2m3uVFZhhUUY7/+oA68bii3g ceph-7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: The key's randomart image is: 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: +---[RSA 3072]----+ 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: | +=o . +B= | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: |+o. o ooo.o | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: | *oo..o o =. | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: |o ..+. o o.+ | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: | .. So. . | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: | o oo . | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: | . . o. . . | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: | .Eoo.o . . | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: | ..==o+. .o | 2024-03-12T04:05:23.671 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: +----[SHA256]-----+ 2024-03-12T04:05:23.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.515+0000 7f27dd61a700 1 Processor -- start 2024-03-12T04:05:23.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.515+0000 7f27dd61a700 1 -- start start 2024-03-12T04:05:23.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.515+0000 7f27dd61a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d81078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:23.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.515+0000 7f27dd61a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8107eb0 con 0x7f27d81074c0 2024-03-12T04:05:23.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.515+0000 7f27d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d81078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.515+0000 7f27d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d81078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32996/0 (socket says 172.21.15.42:32996) 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.515+0000 7f27d6ffd700 1 -- 172.21.15.42:0/1432597720 learned_addr learned my addr 172.21.15.42:0/1432597720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.516+0000 7f27d6ffd700 1 -- 172.21.15.42:0/1432597720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8108670 con 0x7f27d81074c0 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.516+0000 7f27d6ffd700 1 --2- 172.21.15.42:0/1432597720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d81078e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb847ecf816877a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.516+0000 7f27d5ffb700 1 -- 172.21.15.42:0/1432597720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27cc004160 con 0x7f27d81074c0 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.516+0000 7f27d5ffb700 1 -- 172.21.15.42:0/1432597720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f27cc0042c0 con 0x7f27d81074c0 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.516+0000 7f27d5ffb700 1 -- 172.21.15.42:0/1432597720 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27cc011a50 con 0x7f27d81074c0 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.516+0000 7f27dd61a700 1 -- 172.21.15.42:0/1432597720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 msgr2=0x7f27d81078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:23.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.516+0000 7f27dd61a700 1 --2- 172.21.15.42:0/1432597720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d81078e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.517+0000 7f27dd61a700 1 -- 172.21.15.42:0/1432597720 shutdown_connections 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.517+0000 7f27dd61a700 1 --2- 172.21.15.42:0/1432597720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d81078e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.517+0000 7f27dd61a700 1 -- 172.21.15.42:0/1432597720 >> 172.21.15.42:0/1432597720 conn(0x7f27d8102b00 msgr2=0x7f27d8104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.517+0000 7f27dd61a700 1 -- 172.21.15.42:0/1432597720 shutdown_connections 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.517+0000 7f27dd61a700 1 -- 172.21.15.42:0/1432597720 wait complete. 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.517+0000 7f27dd61a700 1 Processor -- start 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.517+0000 7f27dd61a700 1 -- start start 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27dd61a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27dd61a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819b6f0 con 0x7f27d81074c0 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33010/0 (socket says 172.21.15.42:33010) 2024-03-12T04:05:23.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27d6ffd700 1 -- 172.21.15.42:0/564440458 learned_addr learned my addr 172.21.15.42:0/564440458 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27d6ffd700 1 -- 172.21.15.42:0/564440458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc009550 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27d6ffd700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d819b1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27bffff700 1 -- 172.21.15.42:0/564440458 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27cc003a40 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27bffff700 1 -- 172.21.15.42:0/564440458 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f27cc003ba0 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d819b8f0 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.518+0000 7f27bffff700 1 -- 172.21.15.42:0/564440458 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27cc01a700 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.519+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d819bd90 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.519+0000 7f27bffff700 1 -- 172.21.15.42:0/564440458 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f27cc01a980 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.520+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d810b8a0 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.520+0000 7f27bffff700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f27c0038cb0 0x7f27c003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.520+0000 7f27bffff700 1 -- 172.21.15.42:0/564440458 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f27cc04ce90 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.520+0000 7f27d67fc700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f27c0038cb0 0x7f27c003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.520+0000 7f27d67fc700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f27c0038cb0 0x7f27c003b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006fd0 tx=0x7f27c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.525+0000 7f27bffff700 1 -- 172.21.15.42:0/564440458 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27cc018530 con 0x7f27d81074c0 2024-03-12T04:05:23.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:22.727+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f27d8103d10 con 0x7f27c0038cb0 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.376+0000 7f27bffff700 1 -- 172.21.15.42:0/564440458 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f27d8103d10 con 0x7f27c0038cb0 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f27c0038cb0 msgr2=0x7f27c003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f27c0038cb0 0x7f27c003b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006fd0 tx=0x7f27c8006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 msgr2=0x7f27d819b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d819b1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 shutdown_connections 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f27c0038cb0 0x7f27c003b170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 --2- 172.21.15.42:0/564440458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27d81074c0 0x7f27d819b1b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 >> 172.21.15.42:0/564440458 conn(0x7f27d8102b00 msgr2=0x7f27d8103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 shutdown_connections 2024-03-12T04:05:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:23.378+0000 7f27dd61a700 1 -- 172.21.15.42:0/564440458 wait complete. 2024-03-12T04:05:25.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7UlJbF4BrOZNjs10tfQK9RuENNLDEqylCd14bUBA0NDi9B7DR8e5evzEhnlN3DPy1J4dals4ZqwZWrFHAUK1n2NuAXNWFxkZk6d6wBDyWmqIkaPyRHzhXmOd3yjzMTqBBHO7ARhZVeEjGJzWFeX+OYKUJIzkpPiArRy+j8vIRdTmPRfMuOIqDOmT2zav6sIcgRQjBPatQQHid7wEmKTW9/Z/qR9CVGp3+vPP0JvnA/REVdg8DBb/CMSkF17bV+LUnbVN8Sy7X0sxhLnzJQSRQ1OcG055CqyzDAi2/ed9QdDkqmD5d1pn6R5nr8OxLDJZPq0Gz1a2J+fpFUKiyjUEzLOaQESHamzF9+sejp7y3+qRq6QQG6FZCtQ6o0W7ZcHHKdCUH+PsbKUItHY6nFpep66suxl6MpgBtW7T7mO4PFe2pbEhDfGWsZp0KNgPvQR3fJ6k7kGR1bDsgNGLVOiiFzLWXahWRuxQ3Rb8DnyThsIDW80UEvKchrcb3Xxbu4HU= ceph-7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:05:25.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6b11fa700 1 Processor -- start 2024-03-12T04:05:25.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6b11fa700 1 -- start start 2024-03-12T04:05:25.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6b11fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac1067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:25.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6b11fa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac106db0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6aad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac1067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:25.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6aad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac1067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33018/0 (socket says 172.21.15.42:33018) 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6aad9d700 1 -- 172.21.15.42:0/377296494 learned_addr learned my addr 172.21.15.42:0/377296494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.319+0000 7fe6aad9d700 1 -- 172.21.15.42:0/377296494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac1075c0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6aad9d700 1 --2- 172.21.15.42:0/377296494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac1067e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0098a0 tx=0x7fe69c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacc7ef204154457 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6a9d9b700 1 -- 172.21.15.42:0/377296494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c004160 con 0x7fe6ac1063c0 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6a9d9b700 1 -- 172.21.15.42:0/377296494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe69c0042c0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6a9d9b700 1 -- 172.21.15.42:0/377296494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c011a50 con 0x7fe6ac1063c0 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6b11fa700 1 -- 172.21.15.42:0/377296494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 msgr2=0x7fe6ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6b11fa700 1 --2- 172.21.15.42:0/377296494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac1067e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0098a0 tx=0x7fe69c009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6b11fa700 1 -- 172.21.15.42:0/377296494 shutdown_connections 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6b11fa700 1 --2- 172.21.15.42:0/377296494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac1067e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.320+0000 7fe6b11fa700 1 -- 172.21.15.42:0/377296494 >> 172.21.15.42:0/377296494 conn(0x7fe6ac1019a0 msgr2=0x7fe6ac103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.321+0000 7fe6b11fa700 1 -- 172.21.15.42:0/377296494 shutdown_connections 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.321+0000 7fe6b11fa700 1 -- 172.21.15.42:0/377296494 wait complete. 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.321+0000 7fe6b11fa700 1 Processor -- start 2024-03-12T04:05:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.321+0000 7fe6b11fa700 1 -- start start 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6b11fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6b11fa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19d9e0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6aad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6aad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33026/0 (socket says 172.21.15.42:33026) 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6aad9d700 1 -- 172.21.15.42:0/3008014239 learned_addr learned my addr 172.21.15.42:0/3008014239 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6aad9d700 1 -- 172.21.15.42:0/3008014239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c009550 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6aad9d700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac19d4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe69c003f40 tx=0x7fe69c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe693fff700 1 -- 172.21.15.42:0/3008014239 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c003a40 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe693fff700 1 -- 172.21.15.42:0/3008014239 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe69c003ba0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac19dbe0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe693fff700 1 -- 172.21.15.42:0/3008014239 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c0196f0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.322+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac19e080 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.323+0000 7fe693fff700 1 -- 172.21.15.42:0/3008014239 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe69c019960 con 0x7fe6ac1063c0 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.323+0000 7fe693fff700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe694038cb0 0x7fe69403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.323+0000 7fe693fff700 1 -- 172.21.15.42:0/3008014239 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe69c04dbb0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.324+0000 7fe6aa59c700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe694038cb0 0x7fe69403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.324+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac1977a0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.324+0000 7fe6aa59c700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe694038cb0 0x7fe69403b170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.329+0000 7fe693fff700 1 -- 172.21.15.42:0/3008014239 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe69c0243f0 con 0x7fe6ac1063c0 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.533+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6ac02d8a0 con 0x7fe694038cb0 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.534+0000 7fe693fff700 1 -- 172.21.15.42:0/3008014239 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe6ac02d8a0 con 0x7fe694038cb0 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe694038cb0 msgr2=0x7fe69403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe694038cb0 0x7fe69403b170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 msgr2=0x7fe6ac19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac19d4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe69c003f40 tx=0x7fe69c011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 shutdown_connections 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe694038cb0 0x7fe69403b170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 --2- 172.21.15.42:0/3008014239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6ac1063c0 0x7fe6ac19d4a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 >> 172.21.15.42:0/3008014239 conn(0x7fe6ac1019a0 msgr2=0x7fe6ac1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 shutdown_connections 2024-03-12T04:05:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:24.536+0000 7fe6b11fa700 1 -- 172.21.15.42:0/3008014239 wait complete. 2024-03-12T04:05:25.072 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-12T04:05:25.072 INFO:teuthology.orchestra.run.smithi042.stdout:Adding key to root@localhost authorized_keys... 2024-03-12T04:05:25.072 INFO:teuthology.orchestra.run.smithi042.stdout:Adding host smithi042... 2024-03-12T04:05:25.177 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:24 smithi042 ceph-mon[32260]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:26.061 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:25 smithi042 ceph-mon[32260]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:26 smithi042 ceph-mon[32260]: Deploying cephadm binary to smithi042 2024-03-12T04:05:30.550 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Added host 'smithi042' with addr '172.21.15.42' 2024-03-12T04:05:30.550 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f10a5bb6700 1 Processor -- start 2024-03-12T04:05:30.550 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f10a5bb6700 1 -- start start 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f10a5bb6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a01067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f10a5bb6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0106d90 con 0x7f10a01063a0 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f109f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a01067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f109f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a01067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33040/0 (socket says 172.21.15.42:33040) 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f109f7fe700 1 -- 172.21.15.42:0/2331576584 learned_addr learned my addr 172.21.15.42:0/2331576584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f109f7fe700 1 -- 172.21.15.42:0/2331576584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a01075a0 con 0x7f10a01063a0 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.508+0000 7f109f7fe700 1 --2- 172.21.15.42:0/2331576584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a01067c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1090009e80 tx=0x7f109000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e4ab3c1fb9e19d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f109e7fc700 1 -- 172.21.15.42:0/2331576584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f109000e070 con 0x7f10a01063a0 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f109e7fc700 1 -- 172.21.15.42:0/2331576584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f109000b8c0 con 0x7f10a01063a0 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f109e7fc700 1 -- 172.21.15.42:0/2331576584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10900044b0 con 0x7f10a01063a0 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f10a5bb6700 1 -- 172.21.15.42:0/2331576584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 msgr2=0x7f10a01067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f10a5bb6700 1 --2- 172.21.15.42:0/2331576584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a01067c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1090009e80 tx=0x7f109000b330 comp rx=0 tx=0).stop 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f10a5bb6700 1 -- 172.21.15.42:0/2331576584 shutdown_connections 2024-03-12T04:05:30.551 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f10a5bb6700 1 --2- 172.21.15.42:0/2331576584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a01067c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f10a5bb6700 1 -- 172.21.15.42:0/2331576584 >> 172.21.15.42:0/2331576584 conn(0x7f10a01019a0 msgr2=0x7f10a0103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f10a5bb6700 1 -- 172.21.15.42:0/2331576584 shutdown_connections 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.509+0000 7f10a5bb6700 1 -- 172.21.15.42:0/2331576584 wait complete. 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.510+0000 7f10a5bb6700 1 Processor -- start 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.510+0000 7f10a5bb6700 1 -- start start 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.510+0000 7f10a5bb6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.510+0000 7f10a5bb6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a019b720 con 0x7f10a01063a0 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.510+0000 7f109f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.510+0000 7f109f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a019b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33048/0 (socket says 172.21.15.42:33048) 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.510+0000 7f109f7fe700 1 -- 172.21.15.42:0/3573533942 learned_addr learned my addr 172.21.15.42:0/3573533942 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.511+0000 7f109f7fe700 1 -- 172.21.15.42:0/3573533942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090009b30 con 0x7f10a01063a0 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.511+0000 7f109f7fe700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a019b1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1090004100 tx=0x7f1090004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.511+0000 7f109cff9700 1 -- 172.21.15.42:0/3573533942 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f109000e040 con 0x7f10a01063a0 2024-03-12T04:05:30.552 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.511+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a019b920 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.511+0000 7f109cff9700 1 -- 172.21.15.42:0/3573533942 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f109000bd70 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.511+0000 7f109cff9700 1 -- 172.21.15.42:0/3573533942 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f109001c670 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.511+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a019be20 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.512+0000 7f109cff9700 1 -- 172.21.15.42:0/3573533942 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f109001c8b0 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.512+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0195380 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.512+0000 7f109cff9700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f1088038cb0 0x7f108803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.512+0000 7f109cff9700 1 -- 172.21.15.42:0/3573533942 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f109004cb30 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.512+0000 7f109effd700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f1088038cb0 0x7f108803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.513+0000 7f109effd700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f1088038cb0 0x7f108803b170 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1094006fd0 tx=0x7f1094006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.517+0000 7f109cff9700 1 -- 172.21.15.42:0/3573533942 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1090011980 con 0x7f10a01063a0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:25.723+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}) v1 -- 0x7f10a002da00 con 0x7f1088038cb0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.060+0000 7f109cff9700 1 -- 172.21.15.42:0/3573533942 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f10a002da00 con 0x7f1088038cb0 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f1088038cb0 msgr2=0x7f108803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:30.553 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f1088038cb0 0x7f108803b170 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1094006fd0 tx=0x7f1094006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 msgr2=0x7f10a019b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a019b1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1090004100 tx=0x7f1090004600 comp rx=0 tx=0).stop 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 shutdown_connections 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f1088038cb0 0x7f108803b170 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 --2- 172.21.15.42:0/3573533942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a01063a0 0x7f10a019b1e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.062+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 >> 172.21.15.42:0/3573533942 conn(0x7f10a01019a0 msgr2=0x7f10a0102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.063+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 shutdown_connections 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.063+0000 7f10a5bb6700 1 -- 172.21.15.42:0/3573533942 wait complete. 2024-03-12T04:05:30.554 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mon service... 2024-03-12T04:05:31.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:31 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:31.181 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:31 smithi042 ceph-mon[32260]: Added host smithi042 2024-03-12T04:05:31.181 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:31 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.945+0000 7faf857fa700 1 Processor -- start 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.945+0000 7faf857fa700 1 -- start start 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.945+0000 7faf857fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf8010a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.945+0000 7faf857fa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007ae50 con 0x7faf80107f20 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf8010a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf8010a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33064/0 (socket says 172.21.15.42:33064) 2024-03-12T04:05:31.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7effd700 1 -- 172.21.15.42:0/832423936 learned_addr learned my addr 172.21.15.42:0/832423936 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7effd700 1 -- 172.21.15.42:0/832423936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8010a850 con 0x7faf80107f20 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7effd700 1 --2- 172.21.15.42:0/832423936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf8010a310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf70009e80 tx=0x7faf7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9344b4bfba5ce21b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7dffb700 1 -- 172.21.15.42:0/832423936 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf7000e070 con 0x7faf80107f20 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7dffb700 1 -- 172.21.15.42:0/832423936 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7faf7000b8c0 con 0x7faf80107f20 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.946+0000 7faf7dffb700 1 -- 172.21.15.42:0/832423936 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf700044b0 con 0x7faf80107f20 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 -- 172.21.15.42:0/832423936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 msgr2=0x7faf8010a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 --2- 172.21.15.42:0/832423936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf8010a310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf70009e80 tx=0x7faf7000b330 comp rx=0 tx=0).stop 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 -- 172.21.15.42:0/832423936 shutdown_connections 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 --2- 172.21.15.42:0/832423936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf8010a310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 -- 172.21.15.42:0/832423936 >> 172.21.15.42:0/832423936 conn(0x7faf80101950 msgr2=0x7faf80103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 -- 172.21.15.42:0/832423936 shutdown_connections 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 -- 172.21.15.42:0/832423936 wait complete. 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.947+0000 7faf857fa700 1 Processor -- start 2024-03-12T04:05:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf857fa700 1 -- start start 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf857fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf801077a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf857fa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf80107ce0 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf7effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf801077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf7effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf801077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33080/0 (socket says 172.21.15.42:33080) 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf7effd700 1 -- 172.21.15.42:0/1783497604 learned_addr learned my addr 172.21.15.42:0/1783497604 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf7effd700 1 -- 172.21.15.42:0/1783497604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf70009b30 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf7effd700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf801077a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf70004240 tx=0x7faf70004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf67fff700 1 -- 172.21.15.42:0/1783497604 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf7000e040 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf67fff700 1 -- 172.21.15.42:0/1783497604 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7faf7000bd70 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.948+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf80105ed0 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.949+0000 7faf67fff700 1 -- 172.21.15.42:0/1783497604 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf7001c6b0 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.949+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf801063d0 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.949+0000 7faf67fff700 1 -- 172.21.15.42:0/1783497604 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7faf7001c8a0 con 0x7faf80107f20 2024-03-12T04:05:31.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.949+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf800632a0 con 0x7faf80107f20 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.950+0000 7faf67fff700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7faf68038d00 0x7faf6803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.950+0000 7faf67fff700 1 -- 172.21.15.42:0/1783497604 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faf7004cd00 con 0x7faf80107f20 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.950+0000 7faf7e7fc700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7faf68038d00 0x7faf6803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.950+0000 7faf7e7fc700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7faf68038d00 0x7faf6803b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf74006fd0 tx=0x7faf74006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:30.955+0000 7faf67fff700 1 -- 172.21.15.42:0/1783497604 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf70011980 con 0x7faf80107f20 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.173+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7faf80106a00 con 0x7faf68038d00 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.178+0000 7faf67fff700 1 -- 172.21.15.42:0/1783497604 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faf80106a00 con 0x7faf68038d00 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7faf68038d00 msgr2=0x7faf6803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7faf68038d00 0x7faf6803b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf74006fd0 tx=0x7faf74006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 msgr2=0x7faf801077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf801077a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf70004240 tx=0x7faf70004600 comp rx=0 tx=0).stop 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 shutdown_connections 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7faf68038d00 0x7faf6803b1c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:31.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 --2- 172.21.15.42:0/1783497604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf80107f20 0x7faf801077a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:31.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 >> 172.21.15.42:0/1783497604 conn(0x7faf80101950 msgr2=0x7faf80102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:31.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 shutdown_connections 2024-03-12T04:05:31.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:31.181+0000 7faf857fa700 1 -- 172.21.15.42:0/1783497604 wait complete. 2024-03-12T04:05:31.694 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mgr service... 2024-03-12T04:05:32.434 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:32 smithi042 ceph-mon[32260]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:32.434 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:32 smithi042 ceph-mon[32260]: Saving service mon spec with placement count:5 2024-03-12T04:05:32.434 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:32 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.201+0000 7f94ae3d6700 1 Processor -- start 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.201+0000 7f94ae3d6700 1 -- start start 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.201+0000 7f94ae3d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a807b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.201+0000 7f94ae3d6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a807b8d0 con 0x7f94a807ce50 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a807b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a807b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46108/0 (socket says 172.21.15.42:46108) 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a7fff700 1 -- 172.21.15.42:0/750461154 learned_addr learned my addr 172.21.15.42:0/750461154 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a7fff700 1 -- 172.21.15.42:0/750461154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a807ba10 con 0x7f94a807ce50 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a7fff700 1 --2- 172.21.15.42:0/750461154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a807b300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94980098a0 tx=0x7f9498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5669f553332c03e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a6ffd700 1 -- 172.21.15.42:0/750461154 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9498004160 con 0x7f94a807ce50 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a6ffd700 1 -- 172.21.15.42:0/750461154 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f94980042c0 con 0x7f94a807ce50 2024-03-12T04:05:33.077 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.202+0000 7f94a6ffd700 1 -- 172.21.15.42:0/750461154 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9498011a50 con 0x7f94a807ce50 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 -- 172.21.15.42:0/750461154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 msgr2=0x7f94a807b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 --2- 172.21.15.42:0/750461154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a807b300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94980098a0 tx=0x7f9498009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 -- 172.21.15.42:0/750461154 shutdown_connections 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 --2- 172.21.15.42:0/750461154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a807b300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 -- 172.21.15.42:0/750461154 >> 172.21.15.42:0/750461154 conn(0x7f94a8102a40 msgr2=0x7f94a8104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 -- 172.21.15.42:0/750461154 shutdown_connections 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 -- 172.21.15.42:0/750461154 wait complete. 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 Processor -- start 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.203+0000 7f94ae3d6700 1 -- start start 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94ae3d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94ae3d6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a819d920 con 0x7f94a807ce50 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46114/0 (socket says 172.21.15.42:46114) 2024-03-12T04:05:33.078 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a7fff700 1 -- 172.21.15.42:0/2759516149 learned_addr learned my addr 172.21.15.42:0/2759516149 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a7fff700 1 -- 172.21.15.42:0/2759516149 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498009550 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a7fff700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a819d3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9498006af0 tx=0x7f9498011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a57fa700 1 -- 172.21.15.42:0/2759516149 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9498003930 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a819db20 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a57fa700 1 -- 172.21.15.42:0/2759516149 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9498003a90 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94a57fa700 1 -- 172.21.15.42:0/2759516149 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9498019670 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.204+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a819dfc0 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.205+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a810d810 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.206+0000 7f94a57fa700 1 -- 172.21.15.42:0/2759516149 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f949801aa30 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.206+0000 7f94a57fa700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f9490038c60 0x7f949003b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.206+0000 7f94a57fa700 1 -- 172.21.15.42:0/2759516149 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f949804d430 con 0x7f94a807ce50 2024-03-12T04:05:33.079 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.206+0000 7f94a77fe700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f9490038c60 0x7f949003b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.206+0000 7f94a77fe700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f9490038c60 0x7f949003b120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f949c006fd0 tx=0x7f949c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.210+0000 7f94a57fa700 1 -- 172.21.15.42:0/2759516149 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9498023390 con 0x7f94a807ce50 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.431+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f94a8103cd0 con 0x7f9490038c60 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.436+0000 7f94a57fa700 1 -- 172.21.15.42:0/2759516149 <== mgr.14118 v2:172.21.15.42:6800/2670488995 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f94a8103cd0 con 0x7f9490038c60 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f9490038c60 msgr2=0x7f949003b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f9490038c60 0x7f949003b120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f949c006fd0 tx=0x7f949c006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 msgr2=0x7f94a819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a819d3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9498006af0 tx=0x7f9498011ab0 comp rx=0 tx=0).stop 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 shutdown_connections 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7f9490038c60 0x7f949003b120 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 --2- 172.21.15.42:0/2759516149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94a807ce50 0x7f94a819d3e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 >> 172.21.15.42:0/2759516149 conn(0x7f94a8102a40 msgr2=0x7f94a81035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 shutdown_connections 2024-03-12T04:05:33.080 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:32.438+0000 7f94ae3d6700 1 -- 172.21.15.42:0/2759516149 wait complete. 2024-03-12T04:05:33.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:33 smithi042 ceph-mon[32260]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:33.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:33 smithi042 ceph-mon[32260]: Saving service mgr spec with placement count:2 2024-03-12T04:05:33.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:33 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:35.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc66b63700 1 Processor -- start 2024-03-12T04:05:35.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc66b63700 1 -- start start 2024-03-12T04:05:35.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc66b63700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc601078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:35.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc66b63700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60107e90 con 0x7fcc601074a0 2024-03-12T04:05:35.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc648ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc601078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc648ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc601078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46122/0 (socket says 172.21.15.42:46122) 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc648ff700 1 -- 172.21.15.42:0/1769234928 learned_addr learned my addr 172.21.15.42:0/1769234928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc648ff700 1 -- 172.21.15.42:0/1769234928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60108650 con 0x7fcc601074a0 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.205+0000 7fcc648ff700 1 --2- 172.21.15.42:0/1769234928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc601078c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc540098a0 tx=0x7fcc54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdb220097bf1c17 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc5f7fe700 1 -- 172.21.15.42:0/1769234928 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc54004160 con 0x7fcc601074a0 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc5f7fe700 1 -- 172.21.15.42:0/1769234928 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc540042c0 con 0x7fcc601074a0 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc5f7fe700 1 -- 172.21.15.42:0/1769234928 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc540119b0 con 0x7fcc601074a0 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc66b63700 1 -- 172.21.15.42:0/1769234928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 msgr2=0x7fcc601078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc66b63700 1 --2- 172.21.15.42:0/1769234928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc601078c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc540098a0 tx=0x7fcc54009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc66b63700 1 -- 172.21.15.42:0/1769234928 shutdown_connections 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc66b63700 1 --2- 172.21.15.42:0/1769234928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc601078c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc66b63700 1 -- 172.21.15.42:0/1769234928 >> 172.21.15.42:0/1769234928 conn(0x7fcc60102a80 msgr2=0x7fcc60104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.206+0000 7fcc66b63700 1 -- 172.21.15.42:0/1769234928 shutdown_connections 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc66b63700 1 -- 172.21.15.42:0/1769234928 wait complete. 2024-03-12T04:05:35.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc66b63700 1 Processor -- start 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc66b63700 1 -- start start 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc66b63700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc6019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc66b63700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6019d990 con 0x7fcc601074a0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc648ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc6019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc648ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc6019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46130/0 (socket says 172.21.15.42:46130) 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.207+0000 7fcc648ff700 1 -- 172.21.15.42:0/4149302533 learned_addr learned my addr 172.21.15.42:0/4149302533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.208+0000 7fcc648ff700 1 -- 172.21.15.42:0/4149302533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54009550 con 0x7fcc601074a0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.208+0000 7fcc648ff700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc6019d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc54011670 tx=0x7fcc54011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.208+0000 7fcc5dffb700 1 -- 172.21.15.42:0/4149302533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc540038b0 con 0x7fcc601074a0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.208+0000 7fcc5dffb700 1 -- 172.21.15.42:0/4149302533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc54003a10 con 0x7fcc601074a0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.208+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6019db90 con 0x7fcc601074a0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.208+0000 7fcc5dffb700 1 -- 172.21.15.42:0/4149302533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc54019530 con 0x7fcc601074a0 2024-03-12T04:05:35.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.208+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6019e030 con 0x7fcc601074a0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.209+0000 7fcc5dffb700 1 -- 172.21.15.42:0/4149302533 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fcc540197a0 con 0x7fcc601074a0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.209+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6010b880 con 0x7fcc601074a0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.209+0000 7fcc5dffb700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fcc50038cb0 0x7fcc5003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.210+0000 7fcc5ffff700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fcc50038cb0 0x7fcc5003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.210+0000 7fcc5dffb700 1 -- 172.21.15.42:0/4149302533 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc5404ce90 con 0x7fcc601074a0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.210+0000 7fcc5ffff700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fcc50038cb0 0x7fcc5003b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006fd0 tx=0x7fcc4c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.218+0000 7fcc5dffb700 1 -- 172.21.15.42:0/4149302533 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc5402b340 con 0x7fcc601074a0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.419+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fcc6010ba80 con 0x7fcc601074a0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.422+0000 7fcc5dffb700 1 -- 172.21.15.42:0/4149302533 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fcc54018be0 con 0x7fcc601074a0 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fcc50038cb0 msgr2=0x7fcc5003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fcc50038cb0 0x7fcc5003b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006fd0 tx=0x7fcc4c006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 msgr2=0x7fcc6019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc6019d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc54011670 tx=0x7fcc54011a30 comp rx=0 tx=0).stop 2024-03-12T04:05:35.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 shutdown_connections 2024-03-12T04:05:35.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fcc50038cb0 0x7fcc5003b170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:35.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 --2- 172.21.15.42:0/4149302533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc601074a0 0x7fcc6019d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:35.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 >> 172.21.15.42:0/4149302533 conn(0x7fcc60102a80 msgr2=0x7fcc601035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:35.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 shutdown_connections 2024-03-12T04:05:35.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:34.425+0000 7fcc66b63700 1 -- 172.21.15.42:0/4149302533 wait complete. 2024-03-12T04:05:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:35 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:35 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/4149302533' entity='client.admin' 2024-03-12T04:05:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:35 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:36.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4d136a700 1 Processor -- start 2024-03-12T04:05:36.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4d136a700 1 -- start start 2024-03-12T04:05:36.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4d136a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc10a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:36.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4d136a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc07ae50 con 0x7fe4cc107f20 2024-03-12T04:05:36.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4caffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc10a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:36.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4caffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc10a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46138/0 (socket says 172.21.15.42:46138) 2024-03-12T04:05:36.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4caffd700 1 -- 172.21.15.42:0/2460301383 learned_addr learned my addr 172.21.15.42:0/2460301383 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.458+0000 7fe4caffd700 1 -- 172.21.15.42:0/2460301383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc10a850 con 0x7fe4cc107f20 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.459+0000 7fe4caffd700 1 --2- 172.21.15.42:0/2460301383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc10a310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0098a0 tx=0x7fe4bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972c0b823ca181d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.459+0000 7fe4c9ffb700 1 -- 172.21.15.42:0/2460301383 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4bc004160 con 0x7fe4cc107f20 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.460+0000 7fe4c9ffb700 1 -- 172.21.15.42:0/2460301383 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4bc0042c0 con 0x7fe4cc107f20 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.460+0000 7fe4c9ffb700 1 -- 172.21.15.42:0/2460301383 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4bc011960 con 0x7fe4cc107f20 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.460+0000 7fe4d136a700 1 -- 172.21.15.42:0/2460301383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 msgr2=0x7fe4cc10a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.460+0000 7fe4d136a700 1 --2- 172.21.15.42:0/2460301383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc10a310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0098a0 tx=0x7fe4bc009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.461+0000 7fe4d136a700 1 -- 172.21.15.42:0/2460301383 shutdown_connections 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.461+0000 7fe4d136a700 1 --2- 172.21.15.42:0/2460301383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc10a310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.461+0000 7fe4d136a700 1 -- 172.21.15.42:0/2460301383 >> 172.21.15.42:0/2460301383 conn(0x7fe4cc101950 msgr2=0x7fe4cc103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.461+0000 7fe4d136a700 1 -- 172.21.15.42:0/2460301383 shutdown_connections 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.461+0000 7fe4d136a700 1 -- 172.21.15.42:0/2460301383 wait complete. 2024-03-12T04:05:36.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.461+0000 7fe4d136a700 1 Processor -- start 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4d136a700 1 -- start start 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4d136a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4d136a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19b7f0 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4caffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4caffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46154/0 (socket says 172.21.15.42:46154) 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4caffd700 1 -- 172.21.15.42:0/1856544741 learned_addr learned my addr 172.21.15.42:0/1856544741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4caffd700 1 -- 172.21.15.42:0/1856544741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc009550 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.462+0000 7fe4caffd700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc19b2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc011e00 tx=0x7fe4bc011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.463+0000 7fe4b3fff700 1 -- 172.21.15.42:0/1856544741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4bc011a20 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.463+0000 7fe4b3fff700 1 -- 172.21.15.42:0/1856544741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4bc003b50 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.463+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc19b9f0 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.464+0000 7fe4b3fff700 1 -- 172.21.15.42:0/1856544741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4bc01ae20 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.464+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc19be90 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.464+0000 7fe4b3fff700 1 -- 172.21.15.42:0/1856544741 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe4bc019490 con 0x7fe4cc107f20 2024-03-12T04:05:36.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.464+0000 7fe4b3fff700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe4b40388a0 0x7fe4b403ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.464+0000 7fe4b3fff700 1 -- 172.21.15.42:0/1856544741 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe4bc051820 con 0x7fe4cc107f20 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.464+0000 7fe4ca7fc700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe4b40388a0 0x7fe4b403ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.465+0000 7fe4ca7fc700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe4b40388a0 0x7fe4b403ad60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006fd0 tx=0x7fe4c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.465+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc10b630 con 0x7fe4cc107f20 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.470+0000 7fe4b3fff700 1 -- 172.21.15.42:0/1856544741 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4bc019830 con 0x7fe4cc107f20 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.755+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe4cc10b810 con 0x7fe4cc107f20 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.758+0000 7fe4b3fff700 1 -- 172.21.15.42:0/1856544741 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe4bc010840 con 0x7fe4cc107f20 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe4b40388a0 msgr2=0x7fe4b403ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe4b40388a0 0x7fe4b403ad60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006fd0 tx=0x7fe4c0006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 msgr2=0x7fe4cc19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc19b2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc011e00 tx=0x7fe4bc011ee0 comp rx=0 tx=0).stop 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 shutdown_connections 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fe4b40388a0 0x7fe4b403ad60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 --2- 172.21.15.42:0/1856544741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4cc107f20 0x7fe4cc19b2b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 >> 172.21.15.42:0/1856544741 conn(0x7fe4cc101950 msgr2=0x7fe4cc102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.760+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 shutdown_connections 2024-03-12T04:05:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:35.761+0000 7fe4d136a700 1 -- 172.21.15.42:0/1856544741 wait complete. 2024-03-12T04:05:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling the dashboard module... 2024-03-12T04:05:36.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:36 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1856544741' entity='client.admin' 2024-03-12T04:05:38.057 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:37 smithi042 ceph-mon[32260]: from='mgr.14118 172.21.15.42:0/1915261902' entity='mgr.x' 2024-03-12T04:05:38.057 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:37 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/4014769044' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-12T04:05:38.057 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: ignoring --setuser ceph since I am not root 2024-03-12T04:05:38.057 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: ignoring --setgroup ceph since I am not root 2024-03-12T04:05:38.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.185+0000 7fd7578d8700 1 Processor -- start 2024-03-12T04:05:38.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.185+0000 7fd7578d8700 1 -- start start 2024-03-12T04:05:38.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.185+0000 7fd7578d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd75010a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:38.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.185+0000 7fd7578d8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75007ae50 con 0x7fd750107f50 2024-03-12T04:05:38.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.185+0000 7fd755674700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd75010a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:38.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.185+0000 7fd755674700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd75010a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46156/0 (socket says 172.21.15.42:46156) 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.185+0000 7fd755674700 1 -- 172.21.15.42:0/1539871863 learned_addr learned my addr 172.21.15.42:0/1539871863 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.186+0000 7fd755674700 1 -- 172.21.15.42:0/1539871863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75010a880 con 0x7fd750107f50 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.186+0000 7fd755674700 1 --2- 172.21.15.42:0/1539871863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd75010a340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0098a0 tx=0x7fd74c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e04a3942d3fbf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.186+0000 7fd747fff700 1 -- 172.21.15.42:0/1539871863 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd74c004160 con 0x7fd750107f50 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.186+0000 7fd747fff700 1 -- 172.21.15.42:0/1539871863 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd74c0042c0 con 0x7fd750107f50 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.186+0000 7fd747fff700 1 -- 172.21.15.42:0/1539871863 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd74c0119b0 con 0x7fd750107f50 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 -- 172.21.15.42:0/1539871863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 msgr2=0x7fd75010a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 --2- 172.21.15.42:0/1539871863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd75010a340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0098a0 tx=0x7fd74c009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 -- 172.21.15.42:0/1539871863 shutdown_connections 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 --2- 172.21.15.42:0/1539871863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd75010a340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 -- 172.21.15.42:0/1539871863 >> 172.21.15.42:0/1539871863 conn(0x7fd7501019a0 msgr2=0x7fd750103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 -- 172.21.15.42:0/1539871863 shutdown_connections 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 -- 172.21.15.42:0/1539871863 wait complete. 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.187+0000 7fd7578d8700 1 Processor -- start 2024-03-12T04:05:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7578d8700 1 -- start start 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7578d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd7501077d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7578d8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd750107d10 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd755674700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd7501077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd755674700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd7501077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46172/0 (socket says 172.21.15.42:46172) 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd755674700 1 -- 172.21.15.42:0/4014769044 learned_addr learned my addr 172.21.15.42:0/4014769044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd755674700 1 -- 172.21.15.42:0/4014769044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c009550 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd755674700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd7501077d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd74c011670 tx=0x7fd74c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd74c0038b0 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd74c003a10 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd74c019550 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd750105f00 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.188+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7501063a0 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.189+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fd74c01a970 con 0x7fd750107f50 2024-03-12T04:05:38.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.189+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75010b660 con 0x7fd750107f50 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.190+0000 7fd7467fc700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd73c038cb0 0x7fd73c03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.190+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd74c04cec0 con 0x7fd750107f50 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.190+0000 7fd754e73700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd73c038cb0 0x7fd73c03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.190+0000 7fd754e73700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd73c038cb0 0x7fd73c03b170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.195+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd74c003b80 con 0x7fd750107f50 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.463+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd750106960 con 0x7fd750107f50 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.782+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd74c023be0 con 0x7fd750107f50 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.783+0000 7fd7467fc700 1 -- 172.21.15.42:0/4014769044 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fd74c019af0 con 0x7fd750107f50 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd73c038cb0 msgr2=0x7fd73c03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd73c038cb0 0x7fd73c03b170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 msgr2=0x7fd7501077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd7501077d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd74c011670 tx=0x7fd74c011a30 comp rx=0 tx=0).stop 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 shutdown_connections 2024-03-12T04:05:38.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd73c038cb0 0x7fd73c03b170 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:38.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 --2- 172.21.15.42:0/4014769044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd750107f50 0x7fd7501077d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:38.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 >> 172.21.15.42:0/4014769044 conn(0x7fd7501019a0 msgr2=0x7fd7501024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:38.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 shutdown_connections 2024-03-12T04:05:38.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:37.785+0000 7fd7578d8700 1 -- 172.21.15.42:0/4014769044 wait complete. 2024-03-12T04:05:38.663 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:38.370+0000 7efd4d953200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:05:38.663 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:38.471+0000 7efd4d953200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:05:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:38 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/4014769044' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-12T04:05:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:38 smithi042 ceph-mon[32260]: mgrmap e9: x(active, since 19s) 2024-03-12T04:05:39.387 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:39.143+0000 7efd4d953200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:05:39.387 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:39.241+0000 7efd4d953200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:05:39.388 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:39.385+0000 7efd4d953200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:05:39.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:39.484+0000 7efd4d953200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:05:39.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:39.569+0000 7efd4d953200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.864+0000 7fd6c3e8c700 1 Processor -- start 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.864+0000 7fd6c3e8c700 1 -- start start 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.864+0000 7fd6c3e8c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc106170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.864+0000 7fd6c3e8c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc07adf0 con 0x7fd6bc105d50 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.864+0000 7fd6c1c28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc106170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.864+0000 7fd6c1c28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc106170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46196/0 (socket says 172.21.15.42:46196) 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.864+0000 7fd6c1c28700 1 -- 172.21.15.42:0/3865118735 learned_addr learned my addr 172.21.15.42:0/3865118735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.865+0000 7fd6c1c28700 1 -- 172.21.15.42:0/3865118735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc07af30 con 0x7fd6bc105d50 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.865+0000 7fd6c1c28700 1 --2- 172.21.15.42:0/3865118735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc106170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80098a0 tx=0x7fd6b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c9219ec25adda5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.865+0000 7fd6c0c26700 1 -- 172.21.15.42:0/3865118735 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6b8004160 con 0x7fd6bc105d50 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.865+0000 7fd6c0c26700 1 -- 172.21.15.42:0/3865118735 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6b80042c0 con 0x7fd6bc105d50 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.865+0000 7fd6c0c26700 1 -- 172.21.15.42:0/3865118735 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6b80119b0 con 0x7fd6bc105d50 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.866+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/3865118735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 msgr2=0x7fd6bc106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.866+0000 7fd6c3e8c700 1 --2- 172.21.15.42:0/3865118735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc106170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80098a0 tx=0x7fd6b8009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.866+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/3865118735 shutdown_connections 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.866+0000 7fd6c3e8c700 1 --2- 172.21.15.42:0/3865118735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc106170 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.866+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/3865118735 >> 172.21.15.42:0/3865118735 conn(0x7fd6bc1018f0 msgr2=0x7fd6bc103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.866+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/3865118735 shutdown_connections 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.866+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/3865118735 wait complete. 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c3e8c700 1 Processor -- start 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c3e8c700 1 -- start start 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c3e8c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c3e8c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc19d9e0 con 0x7fd6bc105d50 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c1c28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c1c28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46202/0 (socket says 172.21.15.42:46202) 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c1c28700 1 -- 172.21.15.42:0/2639745756 learned_addr learned my addr 172.21.15.42:0/2639745756 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c1c28700 1 -- 172.21.15.42:0/2639745756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b8009550 con 0x7fd6bc105d50 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.867+0000 7fd6c1c28700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc19d4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8011980 tx=0x7fd6b8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.868+0000 7fd6b2ffd700 1 -- 172.21.15.42:0/2639745756 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6b8003970 con 0x7fd6bc105d50 2024-03-12T04:05:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.868+0000 7fd6b2ffd700 1 -- 172.21.15.42:0/2639745756 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6b8003ad0 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.868+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc19dbe0 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.868+0000 7fd6b2ffd700 1 -- 172.21.15.42:0/2639745756 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6b80195e0 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.868+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc19e080 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.869+0000 7fd6b2ffd700 1 -- 172.21.15.42:0/2639745756 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fd6b80197d0 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.869+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc197170 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.869+0000 7fd6b2ffd700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 0x7fd6a803b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.869+0000 7fd6b2ffd700 1 -- 172.21.15.42:0/2639745756 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd6b804cc80 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.869+0000 7fd6c1427700 1 -- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 msgr2=0x7fd6a803b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2670488995 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.869+0000 7fd6c1427700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 0x7fd6a803b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:38.874+0000 7fd6b2ffd700 1 -- 172.21.15.42:0/2639745756 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6b800bc70 con 0x7fd6bc105d50 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.070+0000 7fd6c1427700 1 -- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 msgr2=0x7fd6a803b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2670488995 2024-03-12T04:05:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.070+0000 7fd6c1427700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 0x7fd6a803b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.139+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd6bc02d940 con 0x7fd6bc105d50 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.139+0000 7fd6b2ffd700 1 -- 172.21.15.42:0/2639745756 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fd6b800bc70 con 0x7fd6bc105d50 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.141+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 msgr2=0x7fd6a803b1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.141+0000 7fd6c3e8c700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 0x7fd6a803b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.141+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 msgr2=0x7fd6bc19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.141+0000 7fd6c3e8c700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc19d4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8011980 tx=0x7fd6b8011a60 comp rx=0 tx=0).stop 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.141+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 shutdown_connections 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.141+0000 7fd6c3e8c700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7fd6a8038d10 0x7fd6a803b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.141+0000 7fd6c3e8c700 1 --2- 172.21.15.42:0/2639745756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc105d50 0x7fd6bc19d4a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.142+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 >> 172.21.15.42:0/2639745756 conn(0x7fd6bc1018f0 msgr2=0x7fd6bc1942b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.142+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 shutdown_connections 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:39.142+0000 7fd6c3e8c700 1 -- 172.21.15.42:0/2639745756 wait complete. 2024-03-12T04:05:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-03-12T04:05:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 9... 2024-03-12T04:05:39.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2639745756' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-12T04:05:39.962 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:39.844+0000 7efd4d953200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:05:39.962 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:39.959+0000 7efd4d953200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:05:40.296 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:40 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:40.050+0000 7efd4d953200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:05:40.296 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:40 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:40.150+0000 7efd4d953200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:05:40.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:40 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:40.328+0000 7efd4d953200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:05:41.794 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:41 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:41.319+0000 7efd4d953200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:05:41.794 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:41 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:41.453+0000 7efd4d953200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:05:41.794 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:41 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:41.791+0000 7efd4d953200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:05:42.066 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:41 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:41.884+0000 7efd4d953200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:05:42.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:42 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:42.084+0000 7efd4d953200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:05:42.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:42 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:42.274+0000 7efd4d953200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:05:42.829 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:42 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:42.566+0000 7efd4d953200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:05:42.829 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:42 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:42.826+0000 7efd4d953200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:05:43.100 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:43 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:43.069+0000 7efd4d953200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:05:44.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:43 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:43.896+0000 7efd4d953200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:05:44.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:44.477+0000 7efd4d953200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:05:44.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:44.579+0000 7efd4d953200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:05:44.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:05:44.801+0000 7efd4d953200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:05:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: Active manager daemon x restarted 2024-03-12T04:05:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: Activating manager daemon x 2024-03-12T04:05:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: osdmap e3: 0 total, 0 up, 0 in 2024-03-12T04:05:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: mgrmap e10: x(active, starting, since 0.00344727s) 2024-03-12T04:05:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:05:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T04:05:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T04:05:45.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T04:05:45.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T04:05:45.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: Manager daemon x is now available 2024-03-12T04:05:45.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:45.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:46.020 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T04:05:46.020 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T04:05:46.020 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:45] ENGINE Bus STARTING 2024-03-12T04:05:46.020 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:45] ENGINE Serving on https://172.21.15.42:7150 2024-03-12T04:05:46.020 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:45] ENGINE Serving on http://172.21.15.42:8765 2024-03-12T04:05:46.020 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: [12/Mar/2024:04:05:45] ENGINE Bus STARTED 2024-03-12T04:05:46.021 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-12T04:05:46.021 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: mgrmap e11: x(active, since 1.00561s) 2024-03-12T04:05:46.021 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:45 smithi042 ceph-mon[32260]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-12T04:05:46.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.088+0000 7ff4d340b700 1 Processor -- start 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.088+0000 7ff4d340b700 1 -- start start 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.088+0000 7ff4d340b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc106170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.088+0000 7ff4d340b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc07adf0 con 0x7ff4cc105d50 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4d11a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc106170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4d11a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc106170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46216/0 (socket says 172.21.15.42:46216) 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4d11a7700 1 -- 172.21.15.42:0/4064872071 learned_addr learned my addr 172.21.15.42:0/4064872071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4d11a7700 1 -- 172.21.15.42:0/4064872071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc07af30 con 0x7ff4cc105d50 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4d11a7700 1 --2- 172.21.15.42:0/4064872071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc106170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009e80 tx=0x7ff4bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a567495dadefc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4cbfff700 1 -- 172.21.15.42:0/4064872071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc00e070 con 0x7ff4cc105d50 2024-03-12T04:05:46.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4cbfff700 1 -- 172.21.15.42:0/4064872071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff4bc00b8c0 con 0x7ff4cc105d50 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.089+0000 7ff4cbfff700 1 -- 172.21.15.42:0/4064872071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc004420 con 0x7ff4cc105d50 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.090+0000 7ff4d340b700 1 -- 172.21.15.42:0/4064872071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 msgr2=0x7ff4cc106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.090+0000 7ff4d340b700 1 --2- 172.21.15.42:0/4064872071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc106170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009e80 tx=0x7ff4bc00b330 comp rx=0 tx=0).stop 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.090+0000 7ff4d340b700 1 -- 172.21.15.42:0/4064872071 shutdown_connections 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.090+0000 7ff4d340b700 1 --2- 172.21.15.42:0/4064872071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc106170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.090+0000 7ff4d340b700 1 -- 172.21.15.42:0/4064872071 >> 172.21.15.42:0/4064872071 conn(0x7ff4cc1018f0 msgr2=0x7ff4cc103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.090+0000 7ff4d340b700 1 -- 172.21.15.42:0/4064872071 shutdown_connections 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.090+0000 7ff4d340b700 1 -- 172.21.15.42:0/4064872071 wait complete. 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d340b700 1 Processor -- start 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d340b700 1 -- start start 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d340b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d340b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc19d900 con 0x7ff4cc105d50 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d11a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:46.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d11a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc19d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46228/0 (socket says 172.21.15.42:46228) 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d11a7700 1 -- 172.21.15.42:0/3865173847 learned_addr learned my addr 172.21.15.42:0/3865173847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d11a7700 1 -- 172.21.15.42:0/3865173847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009b30 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d11a7700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc19d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0072a0 tx=0x7ff4bc00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc00e040 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc19db00 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc19e000 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.091+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff4bc003de0 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.092+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc01c620 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.093+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7ff4bc01c890 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.093+0000 7ff4ca7fc700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.093+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 --> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4b403b8e0 con 0x7ff4b4038d10 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.093+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff4bc04cc10 con 0x7ff4cc105d50 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.093+0000 7ff4d09a6700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 msgr2=0x7ff4b403b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2670488995 2024-03-12T04:05:46.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.093+0000 7ff4d09a6700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.294+0000 7ff4d09a6700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 msgr2=0x7ff4b403b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2670488995 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.294+0000 7ff4d09a6700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.695+0000 7ff4d09a6700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 msgr2=0x7ff4b403b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2670488995 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:40.695+0000 7ff4d09a6700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:41.496+0000 7ff4d09a6700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 msgr2=0x7ff4b403b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2670488995 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:41.496+0000 7ff4d09a6700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:43.097+0000 7ff4d09a6700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 msgr2=0x7ff4b403b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2670488995 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:43.097+0000 7ff4d09a6700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:44.817+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 10) v1 ==== 45549+0+0 (secure 0 0 0) 0x7ff4bc01b440 con 0x7ff4cc105d50 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:44.817+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 msgr2=0x7ff4b403b1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:44.817+0000 7ff4ca7fc700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.818+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7ff4bc01ada0 con 0x7ff4cc105d50 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.818+0000 7ff4ca7fc700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4b403c7c0 0x7ff4b403ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:46.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.818+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4b403b8e0 con 0x7ff4b403c7c0 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.819+0000 7ff4d09a6700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4b403c7c0 0x7ff4b403ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.820+0000 7ff4d09a6700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4b403c7c0 0x7ff4b403ebb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0003a10 tx=0x7ff4c0009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.820+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff4b403b8e0 con 0x7ff4b403c7c0 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff4b8002630 con 0x7ff4b403c7c0 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4ca7fc700 1 -- 172.21.15.42:0/3865173847 <== mgr.14150 v2:172.21.15.42:6800/2757555113 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff4b8002630 con 0x7ff4b403c7c0 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4b403c7c0 msgr2=0x7ff4b403ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4b403c7c0 0x7ff4b403ebb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0003a10 tx=0x7ff4c0009270 comp rx=0 tx=0).stop 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 msgr2=0x7ff4cc19d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc19d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0072a0 tx=0x7ff4bc00bd80 comp rx=0 tx=0).stop 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 shutdown_connections 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4b403c7c0 0x7ff4b403ebb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc105d50 0x7ff4cc19d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 --2- 172.21.15.42:0/3865173847 >> [v2:172.21.15.42:6800/2670488995,v1:172.21.15.42:6801/2670488995] conn(0x7ff4b4038d10 0x7ff4b403b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.827+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 >> 172.21.15.42:0/3865173847 conn(0x7ff4cc1018f0 msgr2=0x7ff4cc194210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.828+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 shutdown_connections 2024-03-12T04:05:46.578 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:45.828+0000 7ff4d340b700 1 -- 172.21.15.42:0/3865173847 wait complete. 2024-03-12T04:05:46.578 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 9 is available 2024-03-12T04:05:46.578 INFO:teuthology.orchestra.run.smithi042.stdout:Generating a dashboard self-signed certificate... 2024-03-12T04:05:47.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:47.817 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-12T04:05:47.817 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.990+0000 7f8bb49ec700 1 Processor -- start 2024-03-12T04:05:47.817 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.990+0000 7f8bb49ec700 1 -- start start 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.990+0000 7f8bb49ec700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb0107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.990+0000 7f8bb49ec700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0107e30 con 0x7f8bb0107440 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bae59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb0107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bae59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb0107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55110/0 (socket says 172.21.15.42:55110) 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bae59c700 1 -- 172.21.15.42:0/3433509273 learned_addr learned my addr 172.21.15.42:0/3433509273 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bae59c700 1 -- 172.21.15.42:0/3433509273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0108640 con 0x7f8bb0107440 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bae59c700 1 --2- 172.21.15.42:0/3433509273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb0107860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009e80 tx=0x7f8ba800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e46116c5b64397 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bad59a700 1 -- 172.21.15.42:0/3433509273 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ba800e070 con 0x7f8bb0107440 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bad59a700 1 -- 172.21.15.42:0/3433509273 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ba800b8c0 con 0x7f8bb0107440 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.991+0000 7f8bad59a700 1 -- 172.21.15.42:0/3433509273 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ba8004420 con 0x7f8bb0107440 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.992+0000 7f8bb49ec700 1 -- 172.21.15.42:0/3433509273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 msgr2=0x7f8bb0107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.992+0000 7f8bb49ec700 1 --2- 172.21.15.42:0/3433509273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb0107860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009e80 tx=0x7f8ba800b330 comp rx=0 tx=0).stop 2024-03-12T04:05:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.992+0000 7f8bb49ec700 1 -- 172.21.15.42:0/3433509273 shutdown_connections 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.992+0000 7f8bb49ec700 1 --2- 172.21.15.42:0/3433509273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb0107860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.992+0000 7f8bb49ec700 1 -- 172.21.15.42:0/3433509273 >> 172.21.15.42:0/3433509273 conn(0x7f8bb01029e0 msgr2=0x7f8bb0104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.992+0000 7f8bb49ec700 1 -- 172.21.15.42:0/3433509273 shutdown_connections 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.992+0000 7f8bb49ec700 1 -- 172.21.15.42:0/3433509273 wait complete. 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.993+0000 7f8bb49ec700 1 Processor -- start 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.993+0000 7f8bb49ec700 1 -- start start 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.993+0000 7f8bb49ec700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb007cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.993+0000 7f8bb49ec700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb007d090 con 0x7f8bb0107440 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.993+0000 7f8bae59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb007cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.993+0000 7f8bae59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb007cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55116/0 (socket says 172.21.15.42:55116) 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.993+0000 7f8bae59c700 1 -- 172.21.15.42:0/371816948 learned_addr learned my addr 172.21.15.42:0/371816948 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.994+0000 7f8bae59c700 1 -- 172.21.15.42:0/371816948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8009b30 con 0x7f8bb0107440 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.994+0000 7f8bae59c700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb007cb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80072a0 tx=0x7f8ba800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.994+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ba800e040 con 0x7f8bb0107440 2024-03-12T04:05:47.819 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.994+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ba8003de0 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.994+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb007b280 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.994+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ba801c6b0 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.994+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb007b780 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.995+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7f8ba801c920 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.995+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb010b870 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.995+0000 7f8ba77fe700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b9c038be0 0x7f8b9c03b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.995+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ba804dae0 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.995+0000 7f8badd9b700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b9c038be0 0x7f8b9c03b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:46.996+0000 7f8badd9b700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b9c038be0 0x7f8b9c03b0a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8b98006fd0 tx=0x7f8b98006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.000+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ba801b490 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.209+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bb007be40 con 0x7f8b9c038be0 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.236+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f8ba8012ac0 con 0x7f8bb0107440 2024-03-12T04:05:47.820 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.295+0000 7f8ba77fe700 1 -- 172.21.15.42:0/371816948 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f8bb007be40 con 0x7f8b9c038be0 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b9c038be0 msgr2=0x7f8b9c03b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b9c038be0 0x7f8b9c03b0a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8b98006fd0 tx=0x7f8b98006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 msgr2=0x7f8bb007cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb007cb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80072a0 tx=0x7f8ba800bd80 comp rx=0 tx=0).stop 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 shutdown_connections 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b9c038be0 0x7f8b9c03b0a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 --2- 172.21.15.42:0/371816948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bb0107440 0x7f8bb007cb50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.297+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 >> 172.21.15.42:0/371816948 conn(0x7f8bb01029e0 msgr2=0x7f8bb01034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.298+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 shutdown_connections 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:47.298+0000 7f8bb49ec700 1 -- 172.21.15.42:0/371816948 wait complete. 2024-03-12T04:05:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial admin user... 2024-03-12T04:05:48.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:48 smithi042 ceph-mon[32260]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:48.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:48 smithi042 ceph-mon[32260]: mgrmap e12: x(active, since 2s) 2024-03-12T04:05:48.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:48 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:48.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:48 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:48.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:48 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:49.939 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:49 smithi042 ceph-mon[32260]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:05:49.939 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$68kxlGaSMExDfCB5FgkjBu2Oi5.nxvLlnVgIcOgrWQHnt0VRaRz8q", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710216348, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f2bbf700 1 Processor -- start 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f2bbf700 1 -- start start 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f2bbf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f2bbf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec106d40 con 0x7fc2ec106350 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f095b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f095b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55126/0 (socket says 172.21.15.42:55126) 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f095b700 1 -- 172.21.15.42:0/115969581 learned_addr learned my addr 172.21.15.42:0/115969581 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.282+0000 7fc2f095b700 1 -- 172.21.15.42:0/115969581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec107550 con 0x7fc2ec106350 2024-03-12T04:05:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.283+0000 7fc2f095b700 1 --2- 172.21.15.42:0/115969581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec106770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00098a0 tx=0x7fc2e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25437e904f7180d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.283+0000 7fc2eb7fe700 1 -- 172.21.15.42:0/115969581 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e0004160 con 0x7fc2ec106350 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.283+0000 7fc2eb7fe700 1 -- 172.21.15.42:0/115969581 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2e00042c0 con 0x7fc2ec106350 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.283+0000 7fc2eb7fe700 1 -- 172.21.15.42:0/115969581 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e00119b0 con 0x7fc2ec106350 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.284+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/115969581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 msgr2=0x7fc2ec106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.284+0000 7fc2f2bbf700 1 --2- 172.21.15.42:0/115969581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec106770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00098a0 tx=0x7fc2e0009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.284+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/115969581 shutdown_connections 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.284+0000 7fc2f2bbf700 1 --2- 172.21.15.42:0/115969581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec106770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.284+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/115969581 >> 172.21.15.42:0/115969581 conn(0x7fc2ec1018f0 msgr2=0x7fc2ec103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.284+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/115969581 shutdown_connections 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.284+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/115969581 wait complete. 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f2bbf700 1 Processor -- start 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f2bbf700 1 -- start start 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f2bbf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f2bbf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec19a700 con 0x7fc2ec106350 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f095b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f095b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55130/0 (socket says 172.21.15.42:55130) 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f095b700 1 -- 172.21.15.42:0/207475625 learned_addr learned my addr 172.21.15.42:0/207475625 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.285+0000 7fc2f095b700 1 -- 172.21.15.42:0/207475625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0009550 con 0x7fc2ec106350 2024-03-12T04:05:49.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.286+0000 7fc2f095b700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec19a1c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0011670 tx=0x7fc2e0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.286+0000 7fc2e9ffb700 1 -- 172.21.15.42:0/207475625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e0003890 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.286+0000 7fc2e9ffb700 1 -- 172.21.15.42:0/207475625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2e0020070 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.286+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec19a900 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.286+0000 7fc2e9ffb700 1 -- 172.21.15.42:0/207475625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e001b5e0 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.286+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec19ada0 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.287+0000 7fc2e9ffb700 1 -- 172.21.15.42:0/207475625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fc2e0022030 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.287+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec10a780 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.287+0000 7fc2e9ffb700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2dc038d60 0x7fc2dc03b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.287+0000 7fc2e9ffb700 1 -- 172.21.15.42:0/207475625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc2e004f530 con 0x7fc2ec106350 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.287+0000 7fc2ebfff700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2dc038d60 0x7fc2dc03b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.288+0000 7fc2ebfff700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2dc038d60 0x7fc2dc03b220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8006fd0 tx=0x7fc2d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:49.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.292+0000 7fc2e9ffb700 1 -- 172.21.15.42:0/207475625 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2e001ac60 con 0x7fc2ec106350 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.523+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc2ec0630c0 con 0x7fc2dc038d60 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.770+0000 7fc2e9ffb700 1 -- 172.21.15.42:0/207475625 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fc2ec0630c0 con 0x7fc2dc038d60 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2dc038d60 msgr2=0x7fc2dc03b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2dc038d60 0x7fc2dc03b220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8006fd0 tx=0x7fc2d8006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 msgr2=0x7fc2ec19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec19a1c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0011670 tx=0x7fc2e0011a30 comp rx=0 tx=0).stop 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 shutdown_connections 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2dc038d60 0x7fc2dc03b220 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 --2- 172.21.15.42:0/207475625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2ec106350 0x7fc2ec19a1c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 >> 172.21.15.42:0/207475625 conn(0x7fc2ec1018f0 msgr2=0x7fc2ec102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:49.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 shutdown_connections 2024-03-12T04:05:49.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:48.772+0000 7fc2f2bbf700 1 -- 172.21.15.42:0/207475625 wait complete. 2024-03-12T04:05:49.954 INFO:teuthology.orchestra.run.smithi042.stdout:Fetching dashboard port number... 2024-03-12T04:05:51.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:50 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/401762571' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 8443 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.542+0000 7f9f80cad700 1 Processor -- start 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.542+0000 7f9f80cad700 1 -- start start 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.542+0000 7f9f80cad700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c1078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.542+0000 7f9f80cad700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c107ea0 con 0x7f9f7c1074b0 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c1078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c1078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55136/0 (socket says 172.21.15.42:55136) 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7a59c700 1 -- 172.21.15.42:0/3179518350 learned_addr learned my addr 172.21.15.42:0/3179518350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7a59c700 1 -- 172.21.15.42:0/3179518350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c108660 con 0x7f9f7c1074b0 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7a59c700 1 --2- 172.21.15.42:0/3179518350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c1078d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009e80 tx=0x7f9f7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a2e40f25cdaa1b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7959a700 1 -- 172.21.15.42:0/3179518350 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f7400e070 con 0x7f9f7c1074b0 2024-03-12T04:05:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7959a700 1 -- 172.21.15.42:0/3179518350 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f7400b8c0 con 0x7f9f7c1074b0 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.543+0000 7f9f7959a700 1 -- 172.21.15.42:0/3179518350 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f74004420 con 0x7f9f7c1074b0 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.544+0000 7f9f80cad700 1 -- 172.21.15.42:0/3179518350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 msgr2=0x7f9f7c1078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.544+0000 7f9f80cad700 1 --2- 172.21.15.42:0/3179518350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c1078d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009e80 tx=0x7f9f7400b330 comp rx=0 tx=0).stop 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.544+0000 7f9f80cad700 1 -- 172.21.15.42:0/3179518350 shutdown_connections 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.544+0000 7f9f80cad700 1 --2- 172.21.15.42:0/3179518350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c1078d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.544+0000 7f9f80cad700 1 -- 172.21.15.42:0/3179518350 >> 172.21.15.42:0/3179518350 conn(0x7f9f7c102b10 msgr2=0x7f9f7c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.544+0000 7f9f80cad700 1 -- 172.21.15.42:0/3179518350 shutdown_connections 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.544+0000 7f9f80cad700 1 -- 172.21.15.42:0/3179518350 wait complete. 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f80cad700 1 Processor -- start 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f80cad700 1 -- start start 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f80cad700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c07cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f80cad700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c07d090 con 0x7f9f7c1074b0 2024-03-12T04:05:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f7a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c07cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f7a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c07cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55142/0 (socket says 172.21.15.42:55142) 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f7a59c700 1 -- 172.21.15.42:0/401762571 learned_addr learned my addr 172.21.15.42:0/401762571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f7a59c700 1 -- 172.21.15.42:0/401762571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f74009b30 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f7a59c700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c07cb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f740072a0 tx=0x7f9f7400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.545+0000 7f9f737fe700 1 -- 172.21.15.42:0/401762571 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f7400e040 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.546+0000 7f9f737fe700 1 -- 172.21.15.42:0/401762571 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f74003de0 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.546+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c07b280 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.546+0000 7f9f737fe700 1 -- 172.21.15.42:0/401762571 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f7401c6b0 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.546+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c07b780 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.547+0000 7f9f737fe700 1 -- 172.21.15.42:0/401762571 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f9f7401c920 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.547+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7c0632a0 con 0x7f9f7c1074b0 2024-03-12T04:05:51.498 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.547+0000 7f9f737fe700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9f68038d10 0x7f9f6803b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.547+0000 7f9f737fe700 1 -- 172.21.15.42:0/401762571 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9f7404cf00 con 0x7f9f7c1074b0 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.547+0000 7f9f79d9b700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9f68038d10 0x7f9f6803b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.547+0000 7f9f79d9b700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9f68038d10 0x7f9f6803b1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f64006fd0 tx=0x7f9f64006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.552+0000 7f9f737fe700 1 -- 172.21.15.42:0/401762571 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f7401cd60 con 0x7f9f7c1074b0 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.754+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f9f7c10b890 con 0x7f9f7c1074b0 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.754+0000 7f9f737fe700 1 -- 172.21.15.42:0/401762571 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f9f74017070 con 0x7f9f7c1074b0 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9f68038d10 msgr2=0x7f9f6803b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9f68038d10 0x7f9f6803b1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f64006fd0 tx=0x7f9f64006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 msgr2=0x7f9f7c07cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c07cb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f740072a0 tx=0x7f9f7400bd80 comp rx=0 tx=0).stop 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 shutdown_connections 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9f68038d10 0x7f9f6803b1d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 --2- 172.21.15.42:0/401762571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7c1074b0 0x7f9f7c07cb50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:51.499 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 >> 172.21.15.42:0/401762571 conn(0x7f9f7c102b10 msgr2=0x7f9f7c103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:51.500 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 shutdown_connections 2024-03-12T04:05:51.500 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:50.757+0000 7f9f80cad700 1 -- 172.21.15.42:0/401762571 wait complete. 2024-03-12T04:05:51.503 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T04:05:51.503 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-03-12T04:05:51.510 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T04:05:51.510 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-03-12T04:05:51.510 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-03-12T04:05:51.510 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-12T04:05:51.512 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph Dashboard is now available at: 2024-03-12T04:05:51.512 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:51.512 INFO:teuthology.orchestra.run.smithi042.stdout: URL: https://smithi042.front.sepia.ceph.com:8443/ 2024-03-12T04:05:51.512 INFO:teuthology.orchestra.run.smithi042.stdout: User: admin 2024-03-12T04:05:51.512 INFO:teuthology.orchestra.run.smithi042.stdout: Password: qf1s2fle09 2024-03-12T04:05:51.512 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:51.512 INFO:teuthology.orchestra.run.smithi042.stdout:Saving cluster configuration to /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config directory 2024-03-12T04:05:51.513 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling autotune for osd_memory_target 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870c9e5700 1 Processor -- start 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870c9e5700 1 -- start start 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870c9e5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f87081067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870c9e5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708106da0 con 0x7f87081063b0 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f87081067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f87081067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35324/0 (socket says 172.21.15.42:35324) 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870659c700 1 -- 172.21.15.42:0/1931304269 learned_addr learned my addr 172.21.15.42:0/1931304269 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.006+0000 7f870659c700 1 -- 172.21.15.42:0/1931304269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87081075b0 con 0x7f87081063b0 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.007+0000 7f870659c700 1 --2- 172.21.15.42:0/1931304269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f87081067d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86f80098a0 tx=0x7f86f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60d93bbd2895466 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.007+0000 7f870559a700 1 -- 172.21.15.42:0/1931304269 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86f8004160 con 0x7f87081063b0 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.007+0000 7f870559a700 1 -- 172.21.15.42:0/1931304269 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86f80042c0 con 0x7f87081063b0 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.007+0000 7f870559a700 1 -- 172.21.15.42:0/1931304269 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86f80119b0 con 0x7f87081063b0 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.007+0000 7f870c9e5700 1 -- 172.21.15.42:0/1931304269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 msgr2=0x7f87081067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.007+0000 7f870c9e5700 1 --2- 172.21.15.42:0/1931304269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f87081067d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86f80098a0 tx=0x7f86f8009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.008+0000 7f870c9e5700 1 -- 172.21.15.42:0/1931304269 shutdown_connections 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.008+0000 7f870c9e5700 1 --2- 172.21.15.42:0/1931304269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f87081067d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.008+0000 7f870c9e5700 1 -- 172.21.15.42:0/1931304269 >> 172.21.15.42:0/1931304269 conn(0x7f8708101990 msgr2=0x7f8708103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.008+0000 7f870c9e5700 1 -- 172.21.15.42:0/1931304269 shutdown_connections 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.008+0000 7f870c9e5700 1 -- 172.21.15.42:0/1931304269 wait complete. 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.008+0000 7f870c9e5700 1 Processor -- start 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.008+0000 7f870c9e5700 1 -- start start 2024-03-12T04:05:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870c9e5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f870819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870c9e5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870819b850 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f870819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f870819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35338/0 (socket says 172.21.15.42:35338) 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870659c700 1 -- 172.21.15.42:0/575715170 learned_addr learned my addr 172.21.15.42:0/575715170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870659c700 1 -- 172.21.15.42:0/575715170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8009550 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870659c700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f870819b310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86f8011670 tx=0x7f86f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f86f77fe700 1 -- 172.21.15.42:0/575715170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86f80038b0 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870819ba50 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f86f77fe700 1 -- 172.21.15.42:0/575715170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86f8003a10 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f86f77fe700 1 -- 172.21.15.42:0/575715170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86f801a630 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.009+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870819bef0 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.010+0000 7f86f77fe700 1 -- 172.21.15.42:0/575715170 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f86f801a8a0 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.010+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8708195450 con 0x7f87081063b0 2024-03-12T04:05:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.010+0000 7f86f77fe700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f86f0038db0 0x7f86f003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.010+0000 7f86f77fe700 1 -- 172.21.15.42:0/575715170 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f86f804cff0 con 0x7f87081063b0 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.011+0000 7f8705d9b700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f86f0038db0 0x7f86f003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.011+0000 7f8705d9b700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f86f0038db0 0x7f86f003b270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f86fc006fd0 tx=0x7f86fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.015+0000 7f86f77fe700 1 -- 172.21.15.42:0/575715170 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86f8018a80 con 0x7f87081063b0 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.214+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f870810a7e0 con 0x7f87081063b0 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.215+0000 7f86f77fe700 1 -- 172.21.15.42:0/575715170 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f86f80188a0 con 0x7f87081063b0 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.216+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f86f0038db0 msgr2=0x7f86f003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.216+0000 7f870c9e5700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f86f0038db0 0x7f86f003b270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f86fc006fd0 tx=0x7f86fc006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.216+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 msgr2=0x7f870819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.216+0000 7f870c9e5700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f870819b310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86f8011670 tx=0x7f86f8011a30 comp rx=0 tx=0).stop 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.217+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 shutdown_connections 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.217+0000 7f870c9e5700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f86f0038db0 0x7f86f003b270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.217+0000 7f870c9e5700 1 --2- 172.21.15.42:0/575715170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87081063b0 0x7f870819b310 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.217+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 >> 172.21.15.42:0/575715170 conn(0x7f8708101990 msgr2=0x7f87081028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.217+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 shutdown_connections 2024-03-12T04:05:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:52.217+0000 7f870c9e5700 1 -- 172.21.15.42:0/575715170 wait complete. 2024-03-12T04:05:52.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:52 smithi042 ceph-mon[32260]: mgrmap e13: x(active, since 6s) 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.179+0000 7f8efc06b700 1 Processor -- start 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8efc06b700 1 -- start start 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8efc06b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef4107e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8efc06b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4108410 con 0x7f8ef4107a20 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8ef9e07700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef4107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8ef9e07700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef4107e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35348/0 (socket says 172.21.15.42:35348) 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8ef9e07700 1 -- 172.21.15.42:0/757033291 learned_addr learned my addr 172.21.15.42:0/757033291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:54.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8ef9e07700 1 -- 172.21.15.42:0/757033291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4108550 con 0x7f8ef4107a20 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.180+0000 7f8ef9e07700 1 --2- 172.21.15.42:0/757033291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef4107e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00098a0 tx=0x7f8ef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69a2a3b8c2f5fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8ef8e05700 1 -- 172.21.15.42:0/757033291 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ef0004160 con 0x7f8ef4107a20 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8ef8e05700 1 -- 172.21.15.42:0/757033291 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ef00042c0 con 0x7f8ef4107a20 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8ef8e05700 1 -- 172.21.15.42:0/757033291 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ef00119b0 con 0x7f8ef4107a20 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8efc06b700 1 -- 172.21.15.42:0/757033291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 msgr2=0x7f8ef4107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8efc06b700 1 --2- 172.21.15.42:0/757033291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef4107e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00098a0 tx=0x7f8ef0009bb0 comp rx=0 tx=0).stop 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8efc06b700 1 -- 172.21.15.42:0/757033291 shutdown_connections 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8efc06b700 1 --2- 172.21.15.42:0/757033291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef4107e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8efc06b700 1 -- 172.21.15.42:0/757033291 >> 172.21.15.42:0/757033291 conn(0x7f8ef4101140 msgr2=0x7f8ef4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8efc06b700 1 -- 172.21.15.42:0/757033291 shutdown_connections 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.181+0000 7f8efc06b700 1 -- 172.21.15.42:0/757033291 wait complete. 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8efc06b700 1 Processor -- start 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8efc06b700 1 -- start start 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8efc06b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:54.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8efc06b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef419d9e0 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8ef9e07700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8ef9e07700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35356/0 (socket says 172.21.15.42:35356) 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8ef9e07700 1 -- 172.21.15.42:0/3106899472 learned_addr learned my addr 172.21.15.42:0/3106899472 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.182+0000 7f8ef9e07700 1 -- 172.21.15.42:0/3106899472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0009550 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.183+0000 7f8ef9e07700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef419d4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0011980 tx=0x7f8ef0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.183+0000 7f8eeaffd700 1 -- 172.21.15.42:0/3106899472 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ef0003970 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.183+0000 7f8eeaffd700 1 -- 172.21.15.42:0/3106899472 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ef0003ad0 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.183+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef419dbe0 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.183+0000 7f8eeaffd700 1 -- 172.21.15.42:0/3106899472 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ef00195e0 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.183+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef419e080 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.184+0000 7f8eeaffd700 1 -- 172.21.15.42:0/3106899472 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f8ef001aae0 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.184+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef41061d0 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.184+0000 7f8eeaffd700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8ee0038d50 0x7f8ee003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.184+0000 7f8eeaffd700 1 -- 172.21.15.42:0/3106899472 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ef004d0b0 con 0x7f8ef4107a20 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.184+0000 7f8ef9606700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8ee0038d50 0x7f8ee003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.184+0000 7f8ef9606700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8ee0038d50 0x7f8ee003b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006fd0 tx=0x7f8ee4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:54.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.189+0000 7f8eeaffd700 1 -- 172.21.15.42:0/3106899472 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ef0018720 con 0x7f8ef4107a20 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.504+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f8ef4196c10 con 0x7f8ef4107a20 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.506+0000 7f8eeaffd700 1 -- 172.21.15.42:0/3106899472 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f8ef0023390 con 0x7f8ef4107a20 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8ee0038d50 msgr2=0x7f8ee003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8ee0038d50 0x7f8ee003b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006fd0 tx=0x7f8ee4006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 msgr2=0x7f8ef419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef419d4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0011980 tx=0x7f8ef0011a60 comp rx=0 tx=0).stop 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 shutdown_connections 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8ee0038d50 0x7f8ee003b210 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 --2- 172.21.15.42:0/3106899472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef4107a20 0x7f8ef419d4a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 >> 172.21.15.42:0/3106899472 conn(0x7f8ef4101140 msgr2=0x7f8ef4193cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 shutdown_connections 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-03-12T04:05:53.508+0000 7f8efc06b700 1 -- 172.21.15.42:0/3106899472 wait complete. 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-12T04:05:54.089 INFO:teuthology.orchestra.run.smithi042.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout:Or, if you are only running a single cluster on this host: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: ceph telemetry on 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout:For more information see: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-12T04:05:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:05:54.091 INFO:teuthology.orchestra.run.smithi042.stdout:Bootstrap complete. 2024-03-12T04:05:54.127 INFO:tasks.cephadm:Fetching config... 2024-03-12T04:05:54.127 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:05:54.127 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-12T04:05:54.143 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-12T04:05:54.143 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:05:54.143 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-12T04:05:54.197 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-12T04:05:54.198 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:05:54.198 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-12T04:05:54.265 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-12T04:05:54.265 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:05:54.265 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-12T04:05:54.319 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-12T04:05:54.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7UlJbF4BrOZNjs10tfQK9RuENNLDEqylCd14bUBA0NDi9B7DR8e5evzEhnlN3DPy1J4dals4ZqwZWrFHAUK1n2NuAXNWFxkZk6d6wBDyWmqIkaPyRHzhXmOd3yjzMTqBBHO7ARhZVeEjGJzWFeX+OYKUJIzkpPiArRy+j8vIRdTmPRfMuOIqDOmT2zav6sIcgRQjBPatQQHid7wEmKTW9/Z/qR9CVGp3+vPP0JvnA/REVdg8DBb/CMSkF17bV+LUnbVN8Sy7X0sxhLnzJQSRQ1OcG055CqyzDAi2/ed9QdDkqmD5d1pn6R5nr8OxLDJZPq0Gz1a2J+fpFUKiyjUEzLOaQESHamzF9+sejp7y3+qRq6QQG6FZCtQ6o0W7ZcHHKdCUH+PsbKUItHY6nFpep66suxl6MpgBtW7T7mO4PFe2pbEhDfGWsZp0KNgPvQR3fJ6k7kGR1bDsgNGLVOiiFzLWXahWRuxQ3Rb8DnyThsIDW80UEvKchrcb3Xxbu4HU= ceph-7cca013a-e025-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T04:05:54.404 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7UlJbF4BrOZNjs10tfQK9RuENNLDEqylCd14bUBA0NDi9B7DR8e5evzEhnlN3DPy1J4dals4ZqwZWrFHAUK1n2NuAXNWFxkZk6d6wBDyWmqIkaPyRHzhXmOd3yjzMTqBBHO7ARhZVeEjGJzWFeX+OYKUJIzkpPiArRy+j8vIRdTmPRfMuOIqDOmT2zav6sIcgRQjBPatQQHid7wEmKTW9/Z/qR9CVGp3+vPP0JvnA/REVdg8DBb/CMSkF17bV+LUnbVN8Sy7X0sxhLnzJQSRQ1OcG055CqyzDAi2/ed9QdDkqmD5d1pn6R5nr8OxLDJZPq0Gz1a2J+fpFUKiyjUEzLOaQESHamzF9+sejp7y3+qRq6QQG6FZCtQ6o0W7ZcHHKdCUH+PsbKUItHY6nFpep66suxl6MpgBtW7T7mO4PFe2pbEhDfGWsZp0KNgPvQR3fJ6k7kGR1bDsgNGLVOiiFzLWXahWRuxQ3Rb8DnyThsIDW80UEvKchrcb3Xxbu4HU= ceph-7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:05:54.419 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7UlJbF4BrOZNjs10tfQK9RuENNLDEqylCd14bUBA0NDi9B7DR8e5evzEhnlN3DPy1J4dals4ZqwZWrFHAUK1n2NuAXNWFxkZk6d6wBDyWmqIkaPyRHzhXmOd3yjzMTqBBHO7ARhZVeEjGJzWFeX+OYKUJIzkpPiArRy+j8vIRdTmPRfMuOIqDOmT2zav6sIcgRQjBPatQQHid7wEmKTW9/Z/qR9CVGp3+vPP0JvnA/REVdg8DBb/CMSkF17bV+LUnbVN8Sy7X0sxhLnzJQSRQ1OcG055CqyzDAi2/ed9QdDkqmD5d1pn6R5nr8OxLDJZPq0Gz1a2J+fpFUKiyjUEzLOaQESHamzF9+sejp7y3+qRq6QQG6FZCtQ6o0W7ZcHHKdCUH+PsbKUItHY6nFpep66suxl6MpgBtW7T7mO4PFe2pbEhDfGWsZp0KNgPvQR3fJ6k7kGR1bDsgNGLVOiiFzLWXahWRuxQ3Rb8DnyThsIDW80UEvKchrcb3Xxbu4HU= ceph-7cca013a-e025-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T04:05:54.464 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7UlJbF4BrOZNjs10tfQK9RuENNLDEqylCd14bUBA0NDi9B7DR8e5evzEhnlN3DPy1J4dals4ZqwZWrFHAUK1n2NuAXNWFxkZk6d6wBDyWmqIkaPyRHzhXmOd3yjzMTqBBHO7ARhZVeEjGJzWFeX+OYKUJIzkpPiArRy+j8vIRdTmPRfMuOIqDOmT2zav6sIcgRQjBPatQQHid7wEmKTW9/Z/qR9CVGp3+vPP0JvnA/REVdg8DBb/CMSkF17bV+LUnbVN8Sy7X0sxhLnzJQSRQ1OcG055CqyzDAi2/ed9QdDkqmD5d1pn6R5nr8OxLDJZPq0Gz1a2J+fpFUKiyjUEzLOaQESHamzF9+sejp7y3+qRq6QQG6FZCtQ6o0W7ZcHHKdCUH+PsbKUItHY6nFpep66suxl6MpgBtW7T7mO4PFe2pbEhDfGWsZp0KNgPvQR3fJ6k7kGR1bDsgNGLVOiiFzLWXahWRuxQ3Rb8DnyThsIDW80UEvKchrcb3Xxbu4HU= ceph-7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:05:54.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7UlJbF4BrOZNjs10tfQK9RuENNLDEqylCd14bUBA0NDi9B7DR8e5evzEhnlN3DPy1J4dals4ZqwZWrFHAUK1n2NuAXNWFxkZk6d6wBDyWmqIkaPyRHzhXmOd3yjzMTqBBHO7ARhZVeEjGJzWFeX+OYKUJIzkpPiArRy+j8vIRdTmPRfMuOIqDOmT2zav6sIcgRQjBPatQQHid7wEmKTW9/Z/qR9CVGp3+vPP0JvnA/REVdg8DBb/CMSkF17bV+LUnbVN8Sy7X0sxhLnzJQSRQ1OcG055CqyzDAi2/ed9QdDkqmD5d1pn6R5nr8OxLDJZPq0Gz1a2J+fpFUKiyjUEzLOaQESHamzF9+sejp7y3+qRq6QQG6FZCtQ6o0W7ZcHHKdCUH+PsbKUItHY6nFpep66suxl6MpgBtW7T7mO4PFe2pbEhDfGWsZp0KNgPvQR3fJ6k7kGR1bDsgNGLVOiiFzLWXahWRuxQ3Rb8DnyThsIDW80UEvKchrcb3Xxbu4HU= ceph-7cca013a-e025-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T04:05:54.519 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7UlJbF4BrOZNjs10tfQK9RuENNLDEqylCd14bUBA0NDi9B7DR8e5evzEhnlN3DPy1J4dals4ZqwZWrFHAUK1n2NuAXNWFxkZk6d6wBDyWmqIkaPyRHzhXmOd3yjzMTqBBHO7ARhZVeEjGJzWFeX+OYKUJIzkpPiArRy+j8vIRdTmPRfMuOIqDOmT2zav6sIcgRQjBPatQQHid7wEmKTW9/Z/qR9CVGp3+vPP0JvnA/REVdg8DBb/CMSkF17bV+LUnbVN8Sy7X0sxhLnzJQSRQ1OcG055CqyzDAi2/ed9QdDkqmD5d1pn6R5nr8OxLDJZPq0Gz1a2J+fpFUKiyjUEzLOaQESHamzF9+sejp7y3+qRq6QQG6FZCtQ6o0W7ZcHHKdCUH+PsbKUItHY6nFpep66suxl6MpgBtW7T7mO4PFe2pbEhDfGWsZp0KNgPvQR3fJ6k7kGR1bDsgNGLVOiiFzLWXahWRuxQ3Rb8DnyThsIDW80UEvKchrcb3Xxbu4HU= ceph-7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:05:54.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-12T04:05:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:05:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:54 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3106899472' entity='client.admin' 2024-03-12T04:05:57.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 -- 172.21.15.42:0/2451625980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 msgr2=0x7fb1601045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:57.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 --2- 172.21.15.42:0/2451625980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb1601045a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb150009910 tx=0x7fb150009c20 comp rx=0 tx=0).stop 2024-03-12T04:05:57.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 -- 172.21.15.42:0/2451625980 shutdown_connections 2024-03-12T04:05:57.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 --2- 172.21.15.42:0/2451625980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb1601045a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:57.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 -- 172.21.15.42:0/2451625980 >> 172.21.15.42:0/2451625980 conn(0x7fb1600fbb40 msgr2=0x7fb1600fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 -- 172.21.15.42:0/2451625980 shutdown_connections 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 -- 172.21.15.42:0/2451625980 wait complete. 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 Processor -- start 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.106+0000 7fb1666ed700 1 -- start start 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb1666ed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb160196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb1666ed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb160196fe0 con 0x7fb1601021b0 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb160196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb160196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35376/0 (socket says 172.21.15.42:35376) 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15ffff700 1 -- 172.21.15.42:0/18046264 learned_addr learned my addr 172.21.15.42:0/18046264 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15ffff700 1 -- 172.21.15.42:0/18046264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500095f0 con 0x7fb1601021b0 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15ffff700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb160196aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb150005190 tx=0x7fb150004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15d7fa700 1 -- 172.21.15.42:0/18046264 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb15001d070 con 0x7fb1601021b0 2024-03-12T04:05:57.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15d7fa700 1 -- 172.21.15.42:0/18046264 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb150022470 con 0x7fb1601021b0 2024-03-12T04:05:57.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1601971e0 con 0x7fb1601021b0 2024-03-12T04:05:57.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.107+0000 7fb15d7fa700 1 -- 172.21.15.42:0/18046264 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb15000f460 con 0x7fb1601021b0 2024-03-12T04:05:57.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.108+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb160197680 con 0x7fb1601021b0 2024-03-12T04:05:57.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.108+0000 7fb15d7fa700 1 -- 172.21.15.42:0/18046264 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fb15000f690 con 0x7fb1601021b0 2024-03-12T04:05:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.109+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1601903b0 con 0x7fb1601021b0 2024-03-12T04:05:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.109+0000 7fb15d7fa700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb148038db0 0x7fb14803b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.109+0000 7fb15d7fa700 1 -- 172.21.15.42:0/18046264 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb15004cf70 con 0x7fb1601021b0 2024-03-12T04:05:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.109+0000 7fb15f7fe700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb148038db0 0x7fb14803b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.109+0000 7fb15f7fe700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb148038db0 0x7fb14803b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb154006fd0 tx=0x7fb154006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.114+0000 7fb15d7fa700 1 -- 172.21.15.42:0/18046264 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb15002a910 con 0x7fb1601021b0 2024-03-12T04:05:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:57 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:57 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:57 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:05:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:57 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:57 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:57 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.317+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fb1600632a0 con 0x7fb1601021b0 2024-03-12T04:05:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.319+0000 7fb15d7fa700 1 -- 172.21.15.42:0/18046264 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb150027030 con 0x7fb1601021b0 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb148038db0 msgr2=0x7fb14803b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb148038db0 0x7fb14803b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb154006fd0 tx=0x7fb154006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 msgr2=0x7fb160196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb160196aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb150005190 tx=0x7fb150004f40 comp rx=0 tx=0).stop 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 shutdown_connections 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb148038db0 0x7fb14803b270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 --2- 172.21.15.42:0/18046264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1601021b0 0x7fb160196aa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 >> 172.21.15.42:0/18046264 conn(0x7fb1600fbb40 msgr2=0x7fb1600fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 shutdown_connections 2024-03-12T04:05:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:57.322+0000 7fb1666ed700 1 -- 172.21.15.42:0/18046264 wait complete. 2024-03-12T04:05:57.877 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-12T04:05:57.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-12T04:05:58.117 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:05:58.518 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:58 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/18046264' entity='client.admin' 2024-03-12T04:05:58.519 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:05:58.519 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:05:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.574+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1378203217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 msgr2=0x7f1aa00ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.574+0000 7f1aa7f6a700 1 --2- 172.21.15.42:0/1378203217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa00ffda0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009910 tx=0x7f1a90009c20 comp rx=0 tx=0).stop 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.574+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1378203217 shutdown_connections 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.574+0000 7f1aa7f6a700 1 --2- 172.21.15.42:0/1378203217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa00ffda0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.574+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1378203217 >> 172.21.15.42:0/1378203217 conn(0x7f1aa00fb320 msgr2=0x7f1aa00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.574+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1378203217 shutdown_connections 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1378203217 wait complete. 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa7f6a700 1 Processor -- start 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa7f6a700 1 -- start start 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa7f6a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa01021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa7f6a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0102720 con 0x7f1aa00ff980 2024-03-12T04:05:59.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa5d06700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa01021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:59.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa5d06700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa01021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35396/0 (socket says 172.21.15.42:35396) 2024-03-12T04:05:59.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.575+0000 7f1aa5d06700 1 -- 172.21.15.42:0/1164535373 learned_addr learned my addr 172.21.15.42:0/1164535373 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:05:59.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.576+0000 7f1aa5d06700 1 -- 172.21.15.42:0/1164535373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a900095f0 con 0x7f1aa00ff980 2024-03-12T04:05:59.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.576+0000 7f1aa5d06700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa01021e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1a90004f70 tx=0x7f1a90004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:59.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.576+0000 7f1a96ffd700 1 -- 172.21.15.42:0/1164535373 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a9001d070 con 0x7f1aa00ff980 2024-03-12T04:05:59.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.576+0000 7f1a96ffd700 1 -- 172.21.15.42:0/1164535373 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a9000be10 con 0x7f1aa00ff980 2024-03-12T04:05:59.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.576+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa0100910 con 0x7f1aa00ff980 2024-03-12T04:05:59.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.576+0000 7f1a96ffd700 1 -- 172.21.15.42:0/1164535373 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a90022e90 con 0x7f1aa00ff980 2024-03-12T04:05:59.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.576+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa0100db0 con 0x7f1aa00ff980 2024-03-12T04:05:59.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.577+0000 7f1a96ffd700 1 -- 172.21.15.42:0/1164535373 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f1a9000f460 con 0x7f1aa00ff980 2024-03-12T04:05:59.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.577+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0107d10 con 0x7f1aa00ff980 2024-03-12T04:05:59.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.577+0000 7f1a96ffd700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1a8c038d50 0x7f1a8c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:05:59.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.577+0000 7f1a96ffd700 1 -- 172.21.15.42:0/1164535373 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1a9004cec0 con 0x7f1aa00ff980 2024-03-12T04:05:59.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.577+0000 7f1aa5505700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1a8c038d50 0x7f1a8c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:05:59.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.577+0000 7f1aa5505700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1a8c038d50 0x7f1a8c03b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c006fd0 tx=0x7f1a9c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:05:59.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.582+0000 7f1a96ffd700 1 -- 172.21.15.42:0/1164535373 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a9002a740 con 0x7f1aa00ff980 2024-03-12T04:05:59.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.788+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f1aa01010b0 con 0x7f1a8c038d50 2024-03-12T04:05:59.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.791+0000 7f1a96ffd700 1 -- 172.21.15.42:0/1164535373 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1aa01010b0 con 0x7f1a8c038d50 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1a8c038d50 msgr2=0x7f1a8c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1a8c038d50 0x7f1a8c03b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c006fd0 tx=0x7f1a9c006e40 comp rx=0 tx=0).stop 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 msgr2=0x7f1aa01021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa01021e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1a90004f70 tx=0x7f1a90004fa0 comp rx=0 tx=0).stop 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 shutdown_connections 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1a8c038d50 0x7f1a8c03b210 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 --2- 172.21.15.42:0/1164535373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1aa00ff980 0x7f1aa01021e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 >> 172.21.15.42:0/1164535373 conn(0x7f1aa00fb320 msgr2=0x7f1aa00fbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 shutdown_connections 2024-03-12T04:05:59.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:05:59.793+0000 7f1aa7f6a700 1 -- 172.21.15.42:0/1164535373 wait complete. 2024-03-12T04:06:00.500 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi119 2024-03-12T04:06:00.501 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:06:00.501 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.conf 2024-03-12T04:06:00.519 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:06:00.519 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:00.576 INFO:tasks.cephadm:Adding host smithi119 to orchestrator... 2024-03-12T04:06:00.576 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch host add smithi119 2024-03-12T04:06:00.817 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:00.939 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:00.939 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:00.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:00.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:00.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:00.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: Updating smithi042:/etc/ceph/ceph.conf 2024-03-12T04:06:00.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:00.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:00.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:02.006 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:01 smithi042 ceph-mon[32260]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:02.006 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:01 smithi042 ceph-mon[32260]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:02.006 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:01 smithi042 ceph-mon[32260]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:06:02.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.204+0000 7f72236ad700 1 -- 172.21.15.42:0/3128740709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 msgr2=0x7f721c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:02.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.204+0000 7f72236ad700 1 --2- 172.21.15.42:0/3128740709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c1069f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f721400b0a0 tx=0x7f721400b3b0 comp rx=0 tx=0).stop 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.204+0000 7f72236ad700 1 -- 172.21.15.42:0/3128740709 shutdown_connections 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.204+0000 7f72236ad700 1 --2- 172.21.15.42:0/3128740709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c1069f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.204+0000 7f72236ad700 1 -- 172.21.15.42:0/3128740709 >> 172.21.15.42:0/3128740709 conn(0x7f721c0771b0 msgr2=0x7f721c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.204+0000 7f72236ad700 1 -- 172.21.15.42:0/3128740709 shutdown_connections 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.204+0000 7f72236ad700 1 -- 172.21.15.42:0/3128740709 wait complete. 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f72236ad700 1 Processor -- start 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f72236ad700 1 -- start start 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f72236ad700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f72236ad700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c197310 con 0x7f721c106610 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f7221449700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f7221449700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49274/0 (socket says 172.21.15.42:49274) 2024-03-12T04:06:02.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f7221449700 1 -- 172.21.15.42:0/2575744377 learned_addr learned my addr 172.21.15.42:0/2575744377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:06:02.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f7221449700 1 -- 172.21.15.42:0/2575744377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009d00 con 0x7f721c106610 2024-03-12T04:06:02.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f7221449700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c196dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7214009610 tx=0x7f721400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:02.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f721a7fc700 1 -- 172.21.15.42:0/2575744377 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f721400e040 con 0x7f721c106610 2024-03-12T04:06:02.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f721a7fc700 1 -- 172.21.15.42:0/2575744377 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7214009640 con 0x7f721c106610 2024-03-12T04:06:02.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.205+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c197510 con 0x7f721c106610 2024-03-12T04:06:02.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.206+0000 7f721a7fc700 1 -- 172.21.15.42:0/2575744377 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f721401d470 con 0x7f721c106610 2024-03-12T04:06:02.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.206+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c197a10 con 0x7f721c106610 2024-03-12T04:06:02.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.206+0000 7f721a7fc700 1 -- 172.21.15.42:0/2575744377 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f7214019070 con 0x7f721c106610 2024-03-12T04:06:02.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.206+0000 7f721a7fc700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7204038d50 0x7f720403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:02.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.207+0000 7f721a7fc700 1 -- 172.21.15.42:0/2575744377 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f721404ce40 con 0x7f721c106610 2024-03-12T04:06:02.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.207+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c0632a0 con 0x7f721c106610 2024-03-12T04:06:02.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.207+0000 7f7220c48700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7204038d50 0x7f720403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:02.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.207+0000 7f7220c48700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7204038d50 0x7f720403b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7210006fd0 tx=0x7f7210006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:02.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.212+0000 7f721a7fc700 1 -- 172.21.15.42:0/2575744377 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7214017030 con 0x7f721c106610 2024-03-12T04:06:02.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:02.418+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}) v1 -- 0x7f721c10a7d0 con 0x7f7204038d50 2024-03-12T04:06:03.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:02 smithi042 ceph-mon[32260]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:04.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:03 smithi042 ceph-mon[32260]: Deploying cephadm binary to smithi119 2024-03-12T04:06:06.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:05 smithi042 ceph-mon[32260]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:06.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.807+0000 7f721a7fc700 1 -- 172.21.15.42:0/2575744377 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f721c10a7d0 con 0x7f7204038d50 2024-03-12T04:06:06.809 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi119' with addr '172.21.15.119' 2024-03-12T04:06:06.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7204038d50 msgr2=0x7f720403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:06.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7204038d50 0x7f720403b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7210006fd0 tx=0x7f7210006e40 comp rx=0 tx=0).stop 2024-03-12T04:06:06.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 msgr2=0x7f721c196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:06.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c196dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7214009610 tx=0x7f721400bd40 comp rx=0 tx=0).stop 2024-03-12T04:06:06.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 shutdown_connections 2024-03-12T04:06:06.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7204038d50 0x7f720403b210 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:06.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 --2- 172.21.15.42:0/2575744377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c106610 0x7f721c196dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:06.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 >> 172.21.15.42:0/2575744377 conn(0x7f721c0771b0 msgr2=0x7f721c069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:06.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 shutdown_connections 2024-03-12T04:06:06.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:06.810+0000 7f72236ad700 1 -- 172.21.15.42:0/2575744377 wait complete. 2024-03-12T04:06:07.411 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-12T04:06:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:08.047 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:08.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:07 smithi042 ceph-mon[32260]: Added host smithi119 2024-03-12T04:06:08.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:07 smithi042 ceph-mon[32260]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:08.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:08.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:08.923 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:08 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.166+0000 7f1d37586700 1 -- 172.21.15.42:0/1191650090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 msgr2=0x7f1d30107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.166+0000 7f1d37586700 1 --2- 172.21.15.42:0/1191650090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30107a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009910 tx=0x7f1d2c009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.166+0000 7f1d37586700 1 -- 172.21.15.42:0/1191650090 shutdown_connections 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.166+0000 7f1d37586700 1 --2- 172.21.15.42:0/1191650090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30107a30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.166+0000 7f1d37586700 1 -- 172.21.15.42:0/1191650090 >> 172.21.15.42:0/1191650090 conn(0x7f1d300fd180 msgr2=0x7f1d300ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.166+0000 7f1d37586700 1 -- 172.21.15.42:0/1191650090 shutdown_connections 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.166+0000 7f1d37586700 1 -- 172.21.15.42:0/1191650090 wait complete. 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d37586700 1 Processor -- start 2024-03-12T04:06:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d37586700 1 -- start start 2024-03-12T04:06:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d37586700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d37586700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d301973f0 con 0x7f1d30107650 2024-03-12T04:06:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d35322700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d35322700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49294/0 (socket says 172.21.15.42:49294) 2024-03-12T04:06:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d35322700 1 -- 172.21.15.42:0/1886554910 learned_addr learned my addr 172.21.15.42:0/1886554910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:06:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.167+0000 7f1d35322700 1 -- 172.21.15.42:0/1886554910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0095f0 con 0x7f1d30107650 2024-03-12T04:06:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.168+0000 7f1d35322700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30196eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c004f40 tx=0x7f1d2c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.168+0000 7f1d267fc700 1 -- 172.21.15.42:0/1886554910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d2c01d070 con 0x7f1d30107650 2024-03-12T04:06:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.168+0000 7f1d267fc700 1 -- 172.21.15.42:0/1886554910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1d2c00b6f0 con 0x7f1d30107650 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.168+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d30197650 con 0x7f1d30107650 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.168+0000 7f1d267fc700 1 -- 172.21.15.42:0/1886554910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d2c0037d0 con 0x7f1d30107650 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.168+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d30197ad0 con 0x7f1d30107650 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.169+0000 7f1d267fc700 1 -- 172.21.15.42:0/1886554910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f1d2c003a40 con 0x7f1d30107650 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.169+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d30102d90 con 0x7f1d30107650 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.169+0000 7f1d267fc700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1d1c038db0 0x7f1d1c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.169+0000 7f1d267fc700 1 -- 172.21.15.42:0/1886554910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1d2c04cab0 con 0x7f1d30107650 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.169+0000 7f1d34b21700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1d1c038db0 0x7f1d1c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:09.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.169+0000 7f1d34b21700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1d1c038db0 0x7f1d1c03b270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006fd0 tx=0x7f1d20006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.174+0000 7f1d267fc700 1 -- 172.21.15.42:0/1886554910 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d2c02aaa0 con 0x7f1d30107650 2024-03-12T04:06:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.381+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d3010ba10 con 0x7f1d1c038db0 2024-03-12T04:06:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.382+0000 7f1d267fc700 1 -- 172.21.15.42:0/1886554910 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f1d3010ba10 con 0x7f1d1c038db0 2024-03-12T04:06:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:06:09.384 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}] 2024-03-12T04:06:09.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.384+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1d1c038db0 msgr2=0x7f1d1c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:09.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.384+0000 7f1d37586700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1d1c038db0 0x7f1d1c03b270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006fd0 tx=0x7f1d20006e40 comp rx=0 tx=0).stop 2024-03-12T04:06:09.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.384+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 msgr2=0x7f1d30196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:09.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.384+0000 7f1d37586700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30196eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c004f40 tx=0x7f1d2c005020 comp rx=0 tx=0).stop 2024-03-12T04:06:09.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.385+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 shutdown_connections 2024-03-12T04:06:09.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.385+0000 7f1d37586700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1d1c038db0 0x7f1d1c03b270 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:09.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.385+0000 7f1d37586700 1 --2- 172.21.15.42:0/1886554910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d30107650 0x7f1d30196eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:09.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.385+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 >> 172.21.15.42:0/1886554910 conn(0x7f1d300fd180 msgr2=0x7f1d301091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:09.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.385+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 shutdown_connections 2024-03-12T04:06:09.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:09.385+0000 7f1d37586700 1 -- 172.21.15.42:0/1886554910 wait complete. 2024-03-12T04:06:09.910 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-03-12T04:06:09.910 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:06:09.910 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-03-12T04:06:09.928 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:06:09.928 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:09.985 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-03-12T04:06:09.985 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch host add smithi136 2024-03-12T04:06:10.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:09 smithi042 ceph-mon[32260]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:10.011 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:10.223 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:10 smithi042 ceph-mon[32260]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 -- 172.21.15.42:0/3479477374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 msgr2=0x7f21c0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 --2- 172.21.15.42:0/3479477374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0107a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009910 tx=0x7f21b8009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 -- 172.21.15.42:0/3479477374 shutdown_connections 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 --2- 172.21.15.42:0/3479477374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0107a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 -- 172.21.15.42:0/3479477374 >> 172.21.15.42:0/3479477374 conn(0x7f21c00fd160 msgr2=0x7f21c00ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 -- 172.21.15.42:0/3479477374 shutdown_connections 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 -- 172.21.15.42:0/3479477374 wait complete. 2024-03-12T04:06:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.745+0000 7f21c5975700 1 Processor -- start 2024-03-12T04:06:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21c5975700 1 -- start start 2024-03-12T04:06:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21c5975700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21c5975700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0197430 con 0x7f21c0107650 2024-03-12T04:06:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21beffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21beffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38600/0 (socket says 172.21.15.42:38600) 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21beffd700 1 -- 172.21.15.42:0/3845976283 learned_addr learned my addr 172.21.15.42:0/3845976283 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21beffd700 1 -- 172.21.15.42:0/3845976283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b80095f0 con 0x7f21c0107650 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21beffd700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0196ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21b8004f40 tx=0x7f21b8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.746+0000 7f21c4973700 1 -- 172.21.15.42:0/3845976283 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b801d070 con 0x7f21c0107650 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.747+0000 7f21c4973700 1 -- 172.21.15.42:0/3845976283 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f21b800b6f0 con 0x7f21c0107650 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.747+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c0197690 con 0x7f21c0107650 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.747+0000 7f21c4973700 1 -- 172.21.15.42:0/3845976283 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b80037d0 con 0x7f21c0107650 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.747+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c0197b10 con 0x7f21c0107650 2024-03-12T04:06:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.747+0000 7f21c4973700 1 -- 172.21.15.42:0/3845976283 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f21b800b860 con 0x7f21c0107650 2024-03-12T04:06:11.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.748+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c00632a0 con 0x7f21c0107650 2024-03-12T04:06:11.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.748+0000 7f21c4973700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21ac038d50 0x7f21ac03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:11.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.748+0000 7f21c4973700 1 -- 172.21.15.42:0/3845976283 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f21b804ca50 con 0x7f21c0107650 2024-03-12T04:06:11.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.748+0000 7f21be7fc700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21ac038d50 0x7f21ac03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:11.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.748+0000 7f21be7fc700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21ac038d50 0x7f21ac03b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006fd0 tx=0x7f21a8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.753+0000 7f21c4973700 1 -- 172.21.15.42:0/3845976283 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f21b800edd0 con 0x7f21c0107650 2024-03-12T04:06:11.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:11.961+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) v1 -- 0x7f21c010ba80 con 0x7f21ac038d50 2024-03-12T04:06:12.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:11 smithi042 ceph-mon[32260]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:14.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:14.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:14.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:14.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:14.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:14.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:06:14.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:14.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:14.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-12T04:06:14.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: Deploying cephadm binary to smithi136 2024-03-12T04:06:14.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:14.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:13 smithi042 ceph-mon[32260]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:14 smithi042 ceph-mon[32260]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:14 smithi042 ceph-mon[32260]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:06:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:15 smithi042 ceph-mon[32260]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.510+0000 7f21c4973700 1 -- 172.21.15.42:0/3845976283 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f21c010ba80 con 0x7f21ac038d50 2024-03-12T04:06:16.513 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-03-12T04:06:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21ac038d50 msgr2=0x7f21ac03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21ac038d50 0x7f21ac03b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006fd0 tx=0x7f21a8006e40 comp rx=0 tx=0).stop 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 msgr2=0x7f21c0196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0196ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21b8004f40 tx=0x7f21b8005020 comp rx=0 tx=0).stop 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 shutdown_connections 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21ac038d50 0x7f21ac03b210 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 --2- 172.21.15.42:0/3845976283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c0107650 0x7f21c0196ef0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 >> 172.21.15.42:0/3845976283 conn(0x7f21c00fd160 msgr2=0x7f21c0109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 shutdown_connections 2024-03-12T04:06:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:16.513+0000 7f21c5975700 1 -- 172.21.15.42:0/3845976283 wait complete. 2024-03-12T04:06:17.150 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:17.151 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:17.151 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:17.151 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:17.151 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:16 smithi042 ceph-mon[32260]: Added host smithi136 2024-03-12T04:06:17.151 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:17.151 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:16 smithi042 ceph-mon[32260]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:17.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-12T04:06:17.436 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:18.194 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:18.194 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:18.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 -- 172.21.15.42:0/3890983046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 msgr2=0x7f6728107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:18.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 --2- 172.21.15.42:0/3890983046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728107a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6718009910 tx=0x7f6718009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 -- 172.21.15.42:0/3890983046 shutdown_connections 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 --2- 172.21.15.42:0/3890983046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728107a20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 -- 172.21.15.42:0/3890983046 >> 172.21.15.42:0/3890983046 conn(0x7f67280fd150 msgr2=0x7f67280ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 -- 172.21.15.42:0/3890983046 shutdown_connections 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 -- 172.21.15.42:0/3890983046 wait complete. 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.931+0000 7f672df0b700 1 Processor -- start 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f672df0b700 1 -- start start 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f672df0b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f672df0b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728199640 con 0x7f6728107640 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f67277fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:18.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f67277fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38628/0 (socket says 172.21.15.42:38628) 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f67277fe700 1 -- 172.21.15.42:0/3997381529 learned_addr learned my addr 172.21.15.42:0/3997381529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f67277fe700 1 -- 172.21.15.42:0/3997381529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180095f0 con 0x7f6728107640 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f67277fe700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728199100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6718004f40 tx=0x7f6718005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.932+0000 7f6724ff9700 1 -- 172.21.15.42:0/3997381529 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f671801d070 con 0x7f6728107640 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.933+0000 7f6724ff9700 1 -- 172.21.15.42:0/3997381529 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f671800b6f0 con 0x7f6728107640 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.933+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67281998a0 con 0x7f6728107640 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.933+0000 7f6724ff9700 1 -- 172.21.15.42:0/3997381529 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67180037d0 con 0x7f6728107640 2024-03-12T04:06:18.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.933+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6728199d20 con 0x7f6728107640 2024-03-12T04:06:18.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.933+0000 7f6724ff9700 1 -- 172.21.15.42:0/3997381529 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f6718003a90 con 0x7f6728107640 2024-03-12T04:06:18.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.933+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6728102d80 con 0x7f6728107640 2024-03-12T04:06:18.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.934+0000 7f6724ff9700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6710038d50 0x7f671003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:18.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.934+0000 7f6724ff9700 1 -- 172.21.15.42:0/3997381529 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f671804cc30 con 0x7f6728107640 2024-03-12T04:06:18.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.934+0000 7f6726ffd700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6710038d50 0x7f671003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:18.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.934+0000 7f6726ffd700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6710038d50 0x7f671003b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f671c006fd0 tx=0x7f671c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:18.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:18.939+0000 7f6724ff9700 1 -- 172.21.15.42:0/3997381529 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6718029720 con 0x7f6728107640 2024-03-12T04:06:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.147+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f672810ba00 con 0x7f6710038d50 2024-03-12T04:06:19.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.149+0000 7f6724ff9700 1 -- 172.21.15.42:0/3997381529 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f672810ba00 con 0x7f6710038d50 2024-03-12T04:06:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:06:19.151 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-03-12T04:06:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.150+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6710038d50 msgr2=0x7f671003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.150+0000 7f672df0b700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6710038d50 0x7f671003b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f671c006fd0 tx=0x7f671c006e40 comp rx=0 tx=0).stop 2024-03-12T04:06:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.150+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 msgr2=0x7f6728199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.150+0000 7f672df0b700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728199100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6718004f40 tx=0x7f6718005020 comp rx=0 tx=0).stop 2024-03-12T04:06:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.151+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 shutdown_connections 2024-03-12T04:06:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.151+0000 7f672df0b700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6710038d50 0x7f671003b210 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.151+0000 7f672df0b700 1 --2- 172.21.15.42:0/3997381529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6728107640 0x7f6728199100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.151+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 >> 172.21.15.42:0/3997381529 conn(0x7f67280fd150 msgr2=0x7f6728109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.151+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 shutdown_connections 2024-03-12T04:06:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:19.151+0000 7f672df0b700 1 -- 172.21.15.42:0/3997381529 wait complete. 2024-03-12T04:06:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:19 smithi042 ceph-mon[32260]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:19.731 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-12T04:06:19.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd crush tunables default 2024-03-12T04:06:19.969 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:20.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:20 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:20.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:20 smithi042 ceph-mon[32260]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:06:21.113 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:21 smithi042 ceph-mon[32260]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.330+0000 7f21e3e26700 1 -- 172.21.15.42:0/387659540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 msgr2=0x7f21dc107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.330+0000 7f21e3e26700 1 --2- 172.21.15.42:0/387659540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc107a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009910 tx=0x7f21d8009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.330+0000 7f21e3e26700 1 -- 172.21.15.42:0/387659540 shutdown_connections 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.330+0000 7f21e3e26700 1 --2- 172.21.15.42:0/387659540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc107a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.330+0000 7f21e3e26700 1 -- 172.21.15.42:0/387659540 >> 172.21.15.42:0/387659540 conn(0x7f21dc0fd180 msgr2=0x7f21dc0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.330+0000 7f21e3e26700 1 -- 172.21.15.42:0/387659540 shutdown_connections 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.330+0000 7f21e3e26700 1 -- 172.21.15.42:0/387659540 wait complete. 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.331+0000 7f21e3e26700 1 Processor -- start 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.331+0000 7f21e3e26700 1 -- start start 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.331+0000 7f21e3e26700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.331+0000 7f21e3e26700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc075e60 con 0x7f21dc107650 2024-03-12T04:06:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.331+0000 7f21e1bc2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.331+0000 7f21e1bc2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38652/0 (socket says 172.21.15.42:38652) 2024-03-12T04:06:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.331+0000 7f21e1bc2700 1 -- 172.21.15.42:0/2389006266 learned_addr learned my addr 172.21.15.42:0/2389006266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:06:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.332+0000 7f21e1bc2700 1 -- 172.21.15.42:0/2389006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80095f0 con 0x7f21dc107650 2024-03-12T04:06:21.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.332+0000 7f21e1bc2700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc073f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21d8004f40 tx=0x7f21d8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:21.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.332+0000 7f21d2ffd700 1 -- 172.21.15.42:0/2389006266 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21d801d070 con 0x7f21dc107650 2024-03-12T04:06:21.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.332+0000 7f21d2ffd700 1 -- 172.21.15.42:0/2389006266 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f21d800b6f0 con 0x7f21dc107650 2024-03-12T04:06:21.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.332+0000 7f21d2ffd700 1 -- 172.21.15.42:0/2389006266 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21d80037d0 con 0x7f21dc107650 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.332+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc0744b0 con 0x7f21dc107650 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.332+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc074930 con 0x7f21dc107650 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.333+0000 7f21d2ffd700 1 -- 172.21.15.42:0/2389006266 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f21d8003a90 con 0x7f21dc107650 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.333+0000 7f21d2ffd700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21c8038d50 0x7f21c803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.333+0000 7f21d2ffd700 1 -- 172.21.15.42:0/2389006266 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f21d804cad0 con 0x7f21dc107650 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.333+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc0632a0 con 0x7f21dc107650 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.333+0000 7f21e13c1700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21c8038d50 0x7f21c803b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.333+0000 7f21e13c1700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21c8038d50 0x7f21c803b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21cc006fd0 tx=0x7f21cc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:21.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.338+0000 7f21d2ffd700 1 -- 172.21.15.42:0/2389006266 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f21d8026020 con 0x7f21dc107650 2024-03-12T04:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:21.546+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f21dc074b20 con 0x7f21dc107650 2024-03-12T04:06:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.063+0000 7f21d2ffd700 1 -- 172.21.15.42:0/2389006266 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f21d8003e30 con 0x7f21dc107650 2024-03-12T04:06:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21c8038d50 msgr2=0x7f21c803b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21c8038d50 0x7f21c803b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21cc006fd0 tx=0x7f21cc006e40 comp rx=0 tx=0).stop 2024-03-12T04:06:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 msgr2=0x7f21dc073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc073f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21d8004f40 tx=0x7f21d8005020 comp rx=0 tx=0).stop 2024-03-12T04:06:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 shutdown_connections 2024-03-12T04:06:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21c8038d50 0x7f21c803b210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 --2- 172.21.15.42:0/2389006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc107650 0x7f21dc073f70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.067+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 >> 172.21.15.42:0/2389006266 conn(0x7f21dc0fd180 msgr2=0x7f21dc1091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.068+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 shutdown_connections 2024-03-12T04:06:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:22.068+0000 7f21e3e26700 1 -- 172.21.15.42:0/2389006266 wait complete. 2024-03-12T04:06:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-03-12T04:06:22.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2389006266' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-12T04:06:22.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:22.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:22.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:22.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:22.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:06:22.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:22.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:22.723 INFO:tasks.cephadm:Adding mon.a on smithi042 2024-03-12T04:06:22.723 INFO:tasks.cephadm:Adding mon.b on smithi119 2024-03-12T04:06:22.723 INFO:tasks.cephadm:Adding mon.c on smithi136 2024-03-12T04:06:22.723 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch apply mon '3;smithi042:172.21.15.42=a;smithi119:172.21.15.119=b;smithi136:172.21.15.136=c' 2024-03-12T04:06:22.961 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: Updating smithi136:/etc/ceph/ceph.conf 2024-03-12T04:06:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2389006266' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-12T04:06:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:06:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:23.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:23.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:23.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:24.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 -- 172.21.15.136:0/2583268125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 msgr2=0x7f2ce8106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:24.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 --2- 172.21.15.136:0/2583268125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8106a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009910 tx=0x7f2cd8009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:24.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 -- 172.21.15.136:0/2583268125 shutdown_connections 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 --2- 172.21.15.136:0/2583268125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8106a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 -- 172.21.15.136:0/2583268125 >> 172.21.15.136:0/2583268125 conn(0x7f2ce80fc040 msgr2=0x7f2ce80fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 -- 172.21.15.136:0/2583268125 shutdown_connections 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 -- 172.21.15.136:0/2583268125 wait complete. 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 Processor -- start 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.345+0000 7f2cee443700 1 -- start start 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.346+0000 7f2cee443700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.346+0000 7f2cee443700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce806aa30 con 0x7f2ce8106620 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.346+0000 7f2ce7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.346+0000 7f2ce7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.136:59568/0 (socket says 172.21.15.136:59568) 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.346+0000 7f2ce7fff700 1 -- 172.21.15.136:0/801338757 learned_addr learned my addr 172.21.15.136:0/801338757 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.346+0000 7f2ce7fff700 1 -- 172.21.15.136:0/801338757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80095f0 con 0x7f2ce8106620 2024-03-12T04:06:24.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.346+0000 7f2ce7fff700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8197c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.347+0000 7f2ce57fa700 1 -- 172.21.15.136:0/801338757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd801d070 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.347+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce8198220 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.347+0000 7f2ce57fa700 1 -- 172.21.15.136:0/801338757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cd800bc00 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.347+0000 7f2ce57fa700 1 -- 172.21.15.136:0/801338757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd8003930 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.347+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81986a0 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.348+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8101c90 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.348+0000 7f2ce57fa700 1 -- 172.21.15.136:0/801338757 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f2cd800bd70 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.348+0000 7f2ce57fa700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cd0038d50 0x7f2cd003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.348+0000 7f2ce57fa700 1 -- 172.21.15.136:0/801338757 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2cd804cb00 con 0x7f2ce8106620 2024-03-12T04:06:24.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.348+0000 7f2ce77fe700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cd0038d50 0x7f2cd003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:24.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.349+0000 7f2ce77fe700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cd0038d50 0x7f2cd003b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc006fd0 tx=0x7f2cdc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:24.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.353+0000 7f2ce57fa700 1 -- 172.21.15.136:0/801338757 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cd804ee60 con 0x7f2ce8106620 2024-03-12T04:06:24.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.559+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi119:172.21.15.119=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ce8198a60 con 0x7f2cd0038d50 2024-03-12T04:06:24.567 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.565+0000 7f2ce57fa700 1 -- 172.21.15.136:0/801338757 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2ce8198a60 con 0x7f2cd0038d50 2024-03-12T04:06:24.567 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-03-12T04:06:24.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cd0038d50 msgr2=0x7f2cd003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cd0038d50 0x7f2cd003b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc006fd0 tx=0x7f2cdc006e40 comp rx=0 tx=0).stop 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 msgr2=0x7f2ce8197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8197c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8004ff0 comp rx=0 tx=0).stop 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 shutdown_connections 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cd0038d50 0x7f2cd003b210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 --2- 172.21.15.136:0/801338757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ce8106620 0x7f2ce8197c20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 >> 172.21.15.136:0/801338757 conn(0x7f2ce80fc040 msgr2=0x7f2ce80fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 shutdown_connections 2024-03-12T04:06:24.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:24.567+0000 7f2cee443700 1 -- 172.21.15.136:0/801338757 wait complete. 2024-03-12T04:06:25.220 DEBUG:teuthology.orchestra.run.smithi119:mon.b> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.b.service 2024-03-12T04:06:25.222 DEBUG:teuthology.orchestra.run.smithi136:mon.c> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.c.service 2024-03-12T04:06:25.225 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-12T04:06:25.225 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T04:06:25.966 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi119:172.21.15.119=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi119:172.21.15.119=b;smithi136:172.21.15.136=c;count:3 2024-03-12T04:06:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:26.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:26.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:26.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:26.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: Deploying daemon mon.c on smithi136 2024-03-12T04:06:26.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:25 smithi042 ceph-mon[32260]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:27.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:26 smithi042 ceph-mon[32260]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:29.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.076+0000 7fc418601700 1 -- 172.21.15.136:0/3278118655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 msgr2=0x7fc4101069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:29.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.076+0000 7fc418601700 1 --2- 172.21.15.136:0/3278118655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc4101069e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009910 tx=0x7fc40c009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:29.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.076+0000 7fc418601700 1 -- 172.21.15.136:0/3278118655 shutdown_connections 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.076+0000 7fc418601700 1 --2- 172.21.15.136:0/3278118655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc4101069e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.076+0000 7fc418601700 1 -- 172.21.15.136:0/3278118655 >> 172.21.15.136:0/3278118655 conn(0x7fc4100fc020 msgr2=0x7fc4100fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.076+0000 7fc418601700 1 -- 172.21.15.136:0/3278118655 shutdown_connections 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.076+0000 7fc418601700 1 -- 172.21.15.136:0/3278118655 wait complete. 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.077+0000 7fc418601700 1 Processor -- start 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.077+0000 7fc418601700 1 -- start start 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.077+0000 7fc418601700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc41019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.077+0000 7fc418601700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41019be10 con 0x7fc410106600 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.077+0000 7fc41639d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc41019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.077+0000 7fc41639d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc41019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.136:46010/0 (socket says 172.21.15.136:46010) 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.077+0000 7fc41639d700 1 -- 172.21.15.136:0/4281252028 learned_addr learned my addr 172.21.15.136:0/4281252028 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:06:29.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.078+0000 7fc41639d700 1 -- 172.21.15.136:0/4281252028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0095f0 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.078+0000 7fc41639d700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc41019b8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc40c004f40 tx=0x7fc40c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.078+0000 7fc4077fe700 1 -- 172.21.15.136:0/4281252028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc40c01d070 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.078+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41019c070 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.078+0000 7fc4077fe700 1 -- 172.21.15.136:0/4281252028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc40c00b750 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.078+0000 7fc4077fe700 1 -- 172.21.15.136:0/4281252028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc40c0037d0 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.078+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc410195ae0 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.079+0000 7fc4077fe700 1 -- 172.21.15.136:0/4281252028 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fc40c003a90 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.079+0000 7fc4077fe700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc3fc038d50 0x7fc3fc03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.079+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410101c70 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.079+0000 7fc4077fe700 1 -- 172.21.15.136:0/4281252028 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc40c04cab0 con 0x7fc410106600 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.080+0000 7fc415b9c700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc3fc038d50 0x7fc3fc03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.080+0000 7fc415b9c700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc3fc038d50 0x7fc3fc03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:29.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.085+0000 7fc4077fe700 1 -- 172.21.15.136:0/4281252028 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc40c02ae20 con 0x7fc410106600 2024-03-12T04:06:29.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:28 smithi042 ceph-mon[32260]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:29.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.386+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc410197060 con 0x7fc410106600 2024-03-12T04:06:29.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.387+0000 7fc4077fe700 1 -- 172.21.15.136:0/4281252028 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7fc40c026020 con 0x7fc410106600 2024-03-12T04:06:29.389 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:06:29.389 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","modified":"2024-03-12T04:04:42.081208Z","created":"2024-03-12T04:04:42.081208Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc3fc038d50 msgr2=0x7fc3fc03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc3fc038d50 0x7fc3fc03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).stop 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 msgr2=0x7fc41019b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc41019b8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc40c004f40 tx=0x7fc40c005020 comp rx=0 tx=0).stop 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 shutdown_connections 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc3fc038d50 0x7fc3fc03b210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 --2- 172.21.15.136:0/4281252028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc410106600 0x7fc41019b8d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 >> 172.21.15.136:0/4281252028 conn(0x7fc4100fc020 msgr2=0x7fc4100fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 shutdown_connections 2024-03-12T04:06:29.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:29.389+0000 7fc418601700 1 -- 172.21.15.136:0/4281252028 wait complete. 2024-03-12T04:06:29.393 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-03-12T04:06:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:29 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/4281252028' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T04:06:30.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:30 smithi136 systemd[1]: Started Ceph mon.c for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:06:31.012 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-12T04:06:31.012 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T04:06:31.248 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:06:35.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:06:35.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: mon.a calling monitor election 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: mon.c calling monitor election 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: monmap e2: 2 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: fsmap 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: mgrmap e13: x(active, since 50s) 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: overall HEALTH_OK 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:35.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:35.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: mon.a calling monitor election 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: mon.c calling monitor election 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:35.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: monmap e2: 2 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: fsmap 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: mgrmap e13: x(active, since 50s) 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: overall HEALTH_OK 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:35.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:35.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.706+0000 7fb670202700 1 -- 172.21.15.136:0/1119891325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6540035c0 msgr2=0x7fb654005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:35.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.706+0000 7fb670202700 1 --2- 172.21.15.136:0/1119891325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6540035c0 0x7fb654005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb6681005a0 tx=0x7fb66400feb0 comp rx=0 tx=0).stop 2024-03-12T04:06:35.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.706+0000 7fb670202700 1 -- 172.21.15.136:0/1119891325 shutdown_connections 2024-03-12T04:06:35.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.706+0000 7fb670202700 1 --2- 172.21.15.136:0/1119891325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6540035c0 0x7fb654005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:35.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.706+0000 7fb670202700 1 -- 172.21.15.136:0/1119891325 >> 172.21.15.136:0/1119891325 conn(0x7fb6680fc180 msgr2=0x7fb6680fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.706+0000 7fb670202700 1 -- 172.21.15.136:0/1119891325 shutdown_connections 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.706+0000 7fb670202700 1 -- 172.21.15.136:0/1119891325 wait complete. 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.707+0000 7fb670202700 1 Processor -- start 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.707+0000 7fb670202700 1 -- start start 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb670202700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 0x7fb668197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb670202700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb668198460 0x7fb6681988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb670202700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66806ae00 con 0x7fb668198460 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb670202700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6681a0f90 con 0x7fb6540035c0 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66df9e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 0x7fb668197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66df9e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 0x7fb668197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:33974/0 (socket says 172.21.15.136:33974) 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66df9e700 1 -- 172.21.15.136:0/202789866 learned_addr learned my addr 172.21.15.136:0/202789866 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66d79d700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb668198460 0x7fb6681988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66df9e700 1 -- 172.21.15.136:0/202789866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 msgr2=0x7fb668197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66df9e700 1 -- 172.21.15.136:0/202789866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 msgr2=0x7fb668197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-12T04:06:35.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66df9e700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 0x7fb668197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66df9e700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 0x7fb668197f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66d79d700 1 -- 172.21.15.136:0/202789866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 msgr2=0x7fb668197f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66d79d700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 0x7fb668197f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66d79d700 1 -- 172.21.15.136:0/202789866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66400f900 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.708+0000 7fb66d79d700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb668198460 0x7fb6681988e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb65800b4d0 tx=0x7fb65800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.709+0000 7fb65effd700 1 -- 172.21.15.136:0/202789866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb658011630 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.709+0000 7fb65effd700 1 -- 172.21.15.136:0/202789866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb658011c40 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.709+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6681a1220 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.709+0000 7fb65effd700 1 -- 172.21.15.136:0/202789866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb658010cc0 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.709+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6681a1770 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.710+0000 7fb65effd700 1 -- 172.21.15.136:0/202789866 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fb65801a400 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.710+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6680632a0 con 0x7fb668198460 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.710+0000 7fb65effd700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb64c038b40 0x7fb64c03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:35.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.710+0000 7fb65effd700 1 -- 172.21.15.136:0/202789866 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb65804b380 con 0x7fb668198460 2024-03-12T04:06:35.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.710+0000 7fb66df9e700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb64c038b40 0x7fb64c03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:35.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.711+0000 7fb66df9e700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb64c038b40 0x7fb64c03b000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb664001eb0 tx=0x7fb664005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:35.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.715+0000 7fb65effd700 1 -- 172.21.15.136:0/202789866 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6580178b0 con 0x7fb668198460 2024-03-12T04:06:35.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.988+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb668108560 con 0x7fb668198460 2024-03-12T04:06:35.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.988+0000 7fb65effd700 1 -- 172.21.15.136:0/202789866 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fb6580172c0 con 0x7fb668198460 2024-03-12T04:06:35.991 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:06:35.991 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","modified":"2024-03-12T04:06:30.200718Z","created":"2024-03-12T04:04:42.081208Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-12T04:06:35.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb64c038b40 msgr2=0x7fb64c03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:35.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb64c038b40 0x7fb64c03b000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb664001eb0 tx=0x7fb664005a90 comp rx=0 tx=0).stop 2024-03-12T04:06:35.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb668198460 msgr2=0x7fb6681988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:35.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb668198460 0x7fb6681988e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb65800b4d0 tx=0x7fb65800b890 comp rx=0 tx=0).stop 2024-03-12T04:06:35.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 shutdown_connections 2024-03-12T04:06:35.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb64c038b40 0x7fb64c03b000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:35.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6540035c0 0x7fb668197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:35.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 --2- 172.21.15.136:0/202789866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb668198460 0x7fb6681988e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:35.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 >> 172.21.15.136:0/202789866 conn(0x7fb6680fc180 msgr2=0x7fb6680fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:35.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 shutdown_connections 2024-03-12T04:06:35.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:35.991+0000 7fb670202700 1 -- 172.21.15.136:0/202789866 wait complete. 2024-03-12T04:06:35.994 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-03-12T04:06:36.562 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:36 smithi136 ceph-mon[35571]: Deploying daemon mon.b on smithi119 2024-03-12T04:06:36.562 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:36 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/202789866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T04:06:36.562 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:36 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:36.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:36 smithi042 ceph-mon[32260]: Deploying daemon mon.b on smithi119 2024-03-12T04:06:36.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:36 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/202789866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T04:06:36.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:36 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:37.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:37 smithi042 ceph-mon[32260]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:37.605 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-12T04:06:37.606 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T04:06:37.633 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:37 smithi136 ceph-mon[35571]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:37.842 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:06:39.079 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:38 smithi136 ceph-mon[35571]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:39.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.266+0000 7f868a697700 1 -- 172.21.15.136:0/497231692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684106670 msgr2=0x7f8684106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:39.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.266+0000 7f868a697700 1 --2- 172.21.15.136:0/497231692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684106670 0x7f8684106a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8674009910 tx=0x7f8674009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:39.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.266+0000 7f868a697700 1 -- 172.21.15.136:0/497231692 shutdown_connections 2024-03-12T04:06:39.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.266+0000 7f868a697700 1 --2- 172.21.15.136:0/497231692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684106670 0x7f8684106a50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:39.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.266+0000 7f868a697700 1 -- 172.21.15.136:0/497231692 >> 172.21.15.136:0/497231692 conn(0x7f86840fc180 msgr2=0x7f86840fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:39.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.266+0000 7f868a697700 1 -- 172.21.15.136:0/497231692 shutdown_connections 2024-03-12T04:06:39.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.266+0000 7f868a697700 1 -- 172.21.15.136:0/497231692 wait complete. 2024-03-12T04:06:39.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.267+0000 7f868a697700 1 Processor -- start 2024-03-12T04:06:39.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.267+0000 7f868a697700 1 -- start start 2024-03-12T04:06:39.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.267+0000 7f868a697700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 0x7f8684196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:39.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.267+0000 7f868a697700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684196670 0x7f868419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:39.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.267+0000 7f868a697700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684196c00 con 0x7f8684196670 2024-03-12T04:06:39.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f868a697700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684196d70 con 0x7f8684106670 2024-03-12T04:06:39.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f8683fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 0x7f8684196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f8683fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 0x7f8684196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:33982/0 (socket says 172.21.15.136:33982) 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f8683fff700 1 -- 172.21.15.136:0/2755625914 learned_addr learned my addr 172.21.15.136:0/2755625914 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f8683fff700 1 -- 172.21.15.136:0/2755625914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 msgr2=0x7f8684196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f8683fff700 1 -- 172.21.15.136:0/2755625914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 msgr2=0x7f8684196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f8683fff700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 0x7f8684196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f8683fff700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 0x7f8684196130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f86837fe700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684196670 0x7f868419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f86837fe700 1 -- 172.21.15.136:0/2755625914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 msgr2=0x7f8684196130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f86837fe700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 0x7f8684196130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f86837fe700 1 -- 172.21.15.136:0/2755625914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740095f0 con 0x7f8684196670 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.268+0000 7f86837fe700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684196670 0x7f868419aa00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f867800b4d0 tx=0x7f867800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.269+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8678011630 con 0x7f8684196670 2024-03-12T04:06:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.269+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868419afa0 con 0x7f8684196670 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.269+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8678011c40 con 0x7f8684196670 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.269+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8678010cc0 con 0x7f8684196670 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.269+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868419b4f0 con 0x7f8684196670 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.269+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868419b820 con 0x7f8684196670 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.269+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f8678011790 con 0x7f8684196670 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.270+0000 7f86817fa700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f866c038af0 0x7f866c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.270+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f867804b4d0 con 0x7f8684196670 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.270+0000 7f8683fff700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f866c038af0 0x7f866c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.270+0000 7f8683fff700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f866c038af0 0x7f866c03afb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f867400b5c0 tx=0x7f8674009ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:39.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.275+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86780177d0 con 0x7f8684196670 2024-03-12T04:06:39.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.283+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780104c0 con 0x7f8684196670 2024-03-12T04:06:39.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:39 smithi119 ceph-mon[36119]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-12T04:06:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:38 smithi042 ceph-mon[32260]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:39.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:39.550+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8684197490 con 0x7f8684196670 2024-03-12T04:06:39.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:39 smithi119 bash[36099]: e687522f7f21f9bc97458c7441278fe93e0f2f96cacf0f19a7f80c649c6aa0b3 2024-03-12T04:06:39.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:39 smithi119 systemd[1]: Started Ceph mon.b for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:06:44.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.316+0000 7f86817fa700 1 -- 172.21.15.136:0/2755625914 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7f867800d1c0 con 0x7f8684196670 2024-03-12T04:06:44.320 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-12T04:06:44.320 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":3,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","modified":"2024-03-12T04:06:39.284223Z","created":"2024-03-12T04:04:42.081208Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-12T04:06:44.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f866c038af0 msgr2=0x7f866c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f866c038af0 0x7f866c03afb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f867400b5c0 tx=0x7f8674009ec0 comp rx=0 tx=0).stop 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684196670 msgr2=0x7f868419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684196670 0x7f868419aa00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f867800b4d0 tx=0x7f867800b890 comp rx=0 tx=0).stop 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 shutdown_connections 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f866c038af0 0x7f866c03afb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8684106670 0x7f8684196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 --2- 172.21.15.136:0/2755625914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8684196670 0x7f868419aa00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 >> 172.21.15.136:0/2755625914 conn(0x7f86840fc180 msgr2=0x7f8684102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 shutdown_connections 2024-03-12T04:06:44.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:44.319+0000 7f868a697700 1 -- 172.21.15.136:0/2755625914 wait complete. 2024-03-12T04:06:44.324 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 3 2024-03-12T04:06:44.673 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: mon.c calling monitor election 2024-03-12T04:06:44.673 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:06:44.673 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: mon.a calling monitor election 2024-03-12T04:06:44.673 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.673 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:44.673 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/2755625914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: fsmap 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: mgrmap e13: x(active, since 59s) 2024-03-12T04:06:44.674 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-12T04:06:44.675 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-12T04:06:44.675 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-12T04:06:44.675 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: mon.b (rank 2) addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] is down (out of quorum) 2024-03-12T04:06:44.675 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.675 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.675 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.675 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: mon.c calling monitor election 2024-03-12T04:06:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: mon.a calling monitor election 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/2755625914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:44.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: fsmap 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: mgrmap e13: x(active, since 59s) 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: mon.b (rank 2) addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] is down (out of quorum) 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:44.951 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-12T04:06:44.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph config generate-minimal-conf 2024-03-12T04:06:45.204 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:45.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:45.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:45 smithi042 ceph-mon[32260]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:45.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:45.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:45.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:45.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:45 smithi136 ceph-mon[35571]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:45.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:45.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.c calling monitor election 2024-03-12T04:06:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:06:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.a calling monitor election 2024-03-12T04:06:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:06:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/2755625914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T04:06:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: fsmap 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mgrmap e13: x(active, since 59s) 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-12T04:06:46.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.b (rank 2) addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] is down (out of quorum) 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:46.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.891+0000 7f17bfcc6700 1 -- 172.21.15.42:0/2221893933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b8102790 msgr2=0x7f17b8102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.891+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/2221893933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b8102790 0x7f17b8102b70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009910 tx=0x7f17b4009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.891+0000 7f17bfcc6700 1 -- 172.21.15.42:0/2221893933 shutdown_connections 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.891+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/2221893933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b8102790 0x7f17b8102b70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.891+0000 7f17bfcc6700 1 -- 172.21.15.42:0/2221893933 >> 172.21.15.42:0/2221893933 conn(0x7f17b80fc070 msgr2=0x7f17b80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.891+0000 7f17bfcc6700 1 -- 172.21.15.42:0/2221893933 shutdown_connections 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.891+0000 7f17bfcc6700 1 -- 172.21.15.42:0/2221893933 wait complete. 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 Processor -- start 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 -- start start 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b8102790 0x7f17b8197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b81976a0 0x7f17b819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17b819bf70 0x7f17b819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8197d90 con 0x7f17b81976a0 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81a3ce0 con 0x7f17b819bf70 2024-03-12T04:06:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bfcc6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81a3e20 con 0x7f17b8102790 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bd261700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b81976a0 0x7f17b819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bd261700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b81976a0 0x7f17b819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36244/0 (socket says 172.21.15.42:36244) 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.892+0000 7f17bd261700 1 -- 172.21.15.42:0/1275653905 learned_addr learned my addr 172.21.15.42:0/1275653905 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bd261700 1 -- 172.21.15.42:0/1275653905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b8102790 msgr2=0x7f17b8197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bd261700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b8102790 0x7f17b8197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bd261700 1 -- 172.21.15.42:0/1275653905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17b819bf70 msgr2=0x7f17b819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bd261700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17b819bf70 0x7f17b819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bd261700 1 -- 172.21.15.42:0/1275653905 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b40095f0 con 0x7f17b81976a0 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bd261700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b81976a0 0x7f17b819ba30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b780 tx=0x7f17a800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17aeffd700 1 -- 172.21.15.42:0/1275653905 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8004230 con 0x7f17b81976a0 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17aeffd700 1 -- 172.21.15.42:0/1275653905 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17a8004390 con 0x7f17b81976a0 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b81a4100 con 0x7f17b81976a0 2024-03-12T04:06:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17aeffd700 1 -- 172.21.15.42:0/1275653905 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8011b10 con 0x7f17b81976a0 2024-03-12T04:06:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.893+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b81a4650 con 0x7f17b81976a0 2024-03-12T04:06:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.894+0000 7f17aeffd700 1 -- 172.21.15.42:0/1275653905 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f17a8011cb0 con 0x7f17b81976a0 2024-03-12T04:06:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.894+0000 7f17aeffd700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17a40391e0 0x7f17a403b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.894+0000 7f17aeffd700 1 -- 172.21.15.42:0/1275653905 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f17a804b370 con 0x7f17b81976a0 2024-03-12T04:06:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.894+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b80632a0 con 0x7f17b81976a0 2024-03-12T04:06:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.894+0000 7f17bda62700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17a40391e0 0x7f17a403b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.895+0000 7f17bda62700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17a40391e0 0x7f17a403b6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b5c0 tx=0x7f17b40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:46.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:46.904+0000 7f17aeffd700 1 -- 172.21.15.42:0/1275653905 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17a80193a0 con 0x7f17b81976a0 2024-03-12T04:06:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: mon.b calling monitor election 2024-03-12T04:06:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: mon.b calling monitor election 2024-03-12T04:06:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: mon.c calling monitor election 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: mon.a calling monitor election 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: fsmap 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: mgrmap e13: x(active, since 61s) 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: Cluster is now healthy 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: overall HEALTH_OK 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:47.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:47.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:47.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.110+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f17b810b990 con 0x7f17b81976a0 2024-03-12T04:06:47.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.111+0000 7f17aeffd700 1 -- 172.21.15.42:0/1275653905 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f17a8037030 con 0x7f17b81976a0 2024-03-12T04:06:47.113 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:06:47.113 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-03-12T04:06:47.113 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:06:47.114 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-03-12T04:06:47.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17a40391e0 msgr2=0x7f17a403b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17a40391e0 0x7f17a403b6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b5c0 tx=0x7f17b40058e0 comp rx=0 tx=0).stop 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b81976a0 msgr2=0x7f17b819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b81976a0 0x7f17b819ba30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b780 tx=0x7f17a800bb40 comp rx=0 tx=0).stop 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 shutdown_connections 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17a40391e0 0x7f17a403b6a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17b819bf70 0x7f17b819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b8102790 0x7f17b8197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 --2- 172.21.15.42:0/1275653905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17b81976a0 0x7f17b819ba30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 >> 172.21.15.42:0/1275653905 conn(0x7f17b80fc070 msgr2=0x7f17b8103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 shutdown_connections 2024-03-12T04:06:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:47.113+0000 7f17bfcc6700 1 -- 172.21.15.42:0/1275653905 wait complete. 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.b calling monitor election 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.b calling monitor election 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.c calling monitor election 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.a calling monitor election 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: fsmap 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: mgrmap e13: x(active, since 61s) 2024-03-12T04:06:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: Cluster is now healthy 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: overall HEALTH_OK 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:47.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:47.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: mon.b calling monitor election 2024-03-12T04:06:47.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: mon.b calling monitor election 2024-03-12T04:06:47.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: mon.c calling monitor election 2024-03-12T04:06:47.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: mon.a calling monitor election 2024-03-12T04:06:47.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: fsmap 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: mgrmap e13: x(active, since 61s) 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: Cluster is now healthy 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: overall HEALTH_OK 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:47.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:47.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:47.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:47.751 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-12T04:06:47.751 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:06:47.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T04:06:47.780 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:06:47.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:47.848 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:06:47.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T04:06:47.883 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:06:47.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:47.955 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:06:47.955 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T04:06:47.989 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:06:47.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:06:48.061 INFO:tasks.cephadm:Adding mgr.x on smithi042 2024-03-12T04:06:48.061 INFO:tasks.cephadm:Adding mgr.y on smithi119 2024-03-12T04:06:48.061 INFO:tasks.cephadm:Adding mgr.z on smithi136 2024-03-12T04:06:48.061 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch apply mgr '3;smithi042=x;smithi119=y;smithi136=z' 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: Reconfiguring mon.a (unknown last config time)... 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: Reconfiguring daemon mon.a on smithi042 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1275653905' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:48.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: Reconfiguring mon.a (unknown last config time)... 2024-03-12T04:06:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: Reconfiguring daemon mon.a on smithi042 2024-03-12T04:06:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1275653905' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:48.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:06:48.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: Reconfiguring mon.a (unknown last config time)... 2024-03-12T04:06:48.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: Reconfiguring daemon mon.a on smithi042 2024-03-12T04:06:48.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:48.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1275653905' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:48.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:06:48.335 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:06:49.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:49.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:49.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: Reconfiguring mon.b (monmap changed)... 2024-03-12T04:06:49.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:49.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:49.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:49.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: Reconfiguring daemon mon.b on smithi119 2024-03-12T04:06:49.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:49 smithi136 ceph-mon[35571]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:49.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.841+0000 7fbc643a0700 1 -- 172.21.15.136:0/3478745342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c100690 msgr2=0x7fbc5c100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:49.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.841+0000 7fbc643a0700 1 --2- 172.21.15.136:0/3478745342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c100690 0x7fbc5c100ab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009910 tx=0x7fbc4c009c20 comp rx=0 tx=0).stop 2024-03-12T04:06:49.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.841+0000 7fbc643a0700 1 -- 172.21.15.136:0/3478745342 shutdown_connections 2024-03-12T04:06:49.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.841+0000 7fbc643a0700 1 --2- 172.21.15.136:0/3478745342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c100690 0x7fbc5c100ab0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:49.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.841+0000 7fbc643a0700 1 -- 172.21.15.136:0/3478745342 >> 172.21.15.136:0/3478745342 conn(0x7fbc5c0fbc50 msgr2=0x7fbc5c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:49.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.841+0000 7fbc643a0700 1 -- 172.21.15.136:0/3478745342 shutdown_connections 2024-03-12T04:06:49.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.841+0000 7fbc643a0700 1 -- 172.21.15.136:0/3478745342 wait complete. 2024-03-12T04:06:49.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 Processor -- start 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 -- start start 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5c100690 0x7fbc5c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c199480 0x7fbc5c19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c199900 0x7fbc5c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1a11d0 con 0x7fbc5c199480 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1a1340 con 0x7fbc5c199900 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc643a0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1a1500 con 0x7fbc5c100690 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc6213c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5c100690 0x7fbc5c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc6193b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c199480 0x7fbc5c19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc6213c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5c100690 0x7fbc5c198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:44236/0 (socket says 172.21.15.136:44236) 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.842+0000 7fbc6213c700 1 -- 172.21.15.136:0/854497802 learned_addr learned my addr 172.21.15.136:0/854497802 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6293d700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c199900 0x7fbc5c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6213c700 1 -- 172.21.15.136:0/854497802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c199900 msgr2=0x7fbc5c1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6213c700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c199900 0x7fbc5c1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:49.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6213c700 1 -- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c199480 msgr2=0x7fbc5c19e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6213c700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c199480 0x7fbc5c19e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6213c700 1 -- 172.21.15.136:0/854497802 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0095f0 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6293d700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c199900 0x7fbc5c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6193b700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c199480 0x7fbc5c19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc6213c700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5c100690 0x7fbc5c198f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000c00 tx=0x7fbc4c00bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc537fe700 1 -- 172.21.15.136:0/854497802 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c01d070 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c1a1780 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc537fe700 1 -- 172.21.15.136:0/854497802 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbc4c003d40 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc537fe700 1 -- 172.21.15.136:0/854497802 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00fd20 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.843+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c1a1cf0 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.844+0000 7fbc537fe700 1 -- 172.21.15.136:0/854497802 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fbc4c02b440 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.844+0000 7fbc537fe700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbc480391e0 0x7fbc4803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.844+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c192e70 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.844+0000 7fbc537fe700 1 -- 172.21.15.136:0/854497802 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbc4c04c8f0 con 0x7fbc5c100690 2024-03-12T04:06:49.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.844+0000 7fbc6193b700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbc480391e0 0x7fbc4803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:49.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.845+0000 7fbc6193b700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbc480391e0 0x7fbc4803b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ec0 tx=0x7fbc58009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:49.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:49.849+0000 7fbc537fe700 1 -- 172.21.15.136:0/854497802 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc4c026070 con 0x7fbc5c100690 2024-03-12T04:06:49.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:49.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:49.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: Reconfiguring mon.b (monmap changed)... 2024-03-12T04:06:49.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:49.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:49.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:49.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: Reconfiguring daemon mon.b on smithi119 2024-03-12T04:06:49.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:49 smithi119 ceph-mon[36119]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:50.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.057+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi119=y;smithi136=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc5c02d800 con 0x7fbc480391e0 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: Reconfiguring mon.b (monmap changed)... 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: Reconfiguring daemon mon.b on smithi119 2024-03-12T04:06:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:49 smithi042 ceph-mon[32260]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:50.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.063+0000 7fbc537fe700 1 -- 172.21.15.136:0/854497802 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbc5c02d800 con 0x7fbc480391e0 2024-03-12T04:06:50.067 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mgr update... 2024-03-12T04:06:50.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbc480391e0 msgr2=0x7fbc4803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:50.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbc480391e0 0x7fbc4803b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ec0 tx=0x7fbc58009210 comp rx=0 tx=0).stop 2024-03-12T04:06:50.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5c100690 msgr2=0x7fbc5c198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5c100690 0x7fbc5c198f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000c00 tx=0x7fbc4c00bcd0 comp rx=0 tx=0).stop 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 shutdown_connections 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbc480391e0 0x7fbc4803b6a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c199900 0x7fbc5c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5c100690 0x7fbc5c198f40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 --2- 172.21.15.136:0/854497802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc5c199480 0x7fbc5c19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 >> 172.21.15.136:0/854497802 conn(0x7fbc5c0fbc50 msgr2=0x7fbc5c18ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 shutdown_connections 2024-03-12T04:06:50.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:06:50.065+0000 7fbc643a0700 1 -- 172.21.15.136:0/854497802 wait complete. 2024-03-12T04:06:50.683 DEBUG:teuthology.orchestra.run.smithi119:mgr.y> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.y.service 2024-03-12T04:06:50.686 DEBUG:teuthology.orchestra.run.smithi136:mgr.z> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.z.service 2024-03-12T04:06:50.725 INFO:tasks.cephadm:Deploying OSDs... 2024-03-12T04:06:50.726 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:06:50.726 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T04:06:50.744 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T04:06:50.745 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-03-12T04:06:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T04:06:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:50.803 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-12T04:06:50.803 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:50.804 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:50.804 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-03-12 04:05:55.202172562 +0000 2024-03-12T04:06:50.804 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-03-12 03:58:47.792588808 +0000 2024-03-12T04:06:50.804 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-03-12 03:58:47.792588808 +0000 2024-03-12T04:06:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-03-12 03:58:47.792588808 +0000 2024-03-12T04:06:50.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T04:06:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-03-12T04:06:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-03-12T04:06:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.00012549 s, 4.1 MB/s 2024-03-12T04:06:50.875 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T04:06:50.934 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-03-12T04:06:50.991 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T04:06:50.991 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:50.991 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-12T04:06:50.991 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:50.991 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:50.992 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-03-12 04:05:55.246172188 +0000 2024-03-12T04:06:50.992 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-03-12 03:58:48.134585716 +0000 2024-03-12T04:06:50.992 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-03-12 03:58:48.134585716 +0000 2024-03-12T04:06:50.992 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-03-12 03:58:48.134585716 +0000 2024-03-12T04:06:50.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T04:06:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-03-12T04:06:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-03-12T04:06:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000125427 s, 4.1 MB/s 2024-03-12T04:06:51.062 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T04:06:51.120 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-03-12 04:05:55.279171908 +0000 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-03-12 03:58:48.464582732 +0000 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-03-12 03:58:48.464582732 +0000 2024-03-12T04:06:51.177 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-03-12 03:58:48.464582732 +0000 2024-03-12T04:06:51.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi119=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: Saving service mgr spec with placement smithi042=x;smithi119=y;smithi136=z;count:3 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: Reconfiguring mon.c (monmap changed)... 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: Reconfiguring daemon mon.c on smithi136 2024-03-12T04:06:51.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:51 smithi042 ceph-mon[32260]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-03-12T04:06:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-03-12T04:06:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000115185 s, 4.4 MB/s 2024-03-12T04:06:51.246 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T04:06:51.303 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-03-12 04:05:55.313171620 +0000 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-03-12 03:58:48.786579820 +0000 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-03-12 03:58:48.786579820 +0000 2024-03-12T04:06:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-03-12 03:58:48.786579820 +0000 2024-03-12T04:06:51.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T04:06:51.426 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-03-12T04:06:51.426 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-03-12T04:06:51.426 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000116069 s, 4.4 MB/s 2024-03-12T04:06:51.427 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T04:06:51.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi119=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: Saving service mgr spec with placement smithi042=x;smithi119=y;smithi136=z;count:3 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: Reconfiguring mon.c (monmap changed)... 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: Reconfiguring daemon mon.c on smithi136 2024-03-12T04:06:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:51 smithi119 ceph-mon[36119]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi119=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: Saving service mgr spec with placement smithi042=x;smithi119=y;smithi136=z;count:3 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: Reconfiguring mon.c (monmap changed)... 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T04:06:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T04:06:51.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:51.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: Reconfiguring daemon mon.c on smithi136 2024-03-12T04:06:51.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:51 smithi136 ceph-mon[35571]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:51.483 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-12T04:06:51.483 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T04:06:51.502 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T04:06:51.502 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-03-12T04:06:51.559 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-12 04:06:38.755372775 +0000 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-12 03:58:44.264566350 +0000 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-12 03:58:44.264566350 +0000 2024-03-12T04:06:51.560 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-12 03:58:44.264566350 +0000 2024-03-12T04:06:51.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T04:06:51.630 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-12T04:06:51.631 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-12T04:06:51.631 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000300859 s, 1.7 MB/s 2024-03-12T04:06:51.632 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T04:06:51.690 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-12 04:06:38.755372775 +0000 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-12 03:58:44.587558373 +0000 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-12 03:58:44.587558373 +0000 2024-03-12T04:06:51.747 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-12 03:58:44.587558373 +0000 2024-03-12T04:06:51.747 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T04:06:51.815 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-12T04:06:51.816 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-12T04:06:51.816 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000210795 s, 2.4 MB/s 2024-03-12T04:06:51.817 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T04:06:51.874 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-03-12T04:06:51.930 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T04:06:51.930 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:51.930 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-12T04:06:51.930 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:51.930 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:51.930 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-12 04:06:38.756372751 +0000 2024-03-12T04:06:51.930 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-12 03:58:44.927549975 +0000 2024-03-12T04:06:51.931 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-12 03:58:44.927549975 +0000 2024-03-12T04:06:51.931 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-12 03:58:44.927549975 +0000 2024-03-12T04:06:51.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T04:06:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-12T04:06:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-12T04:06:51.998 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00021063 s, 2.4 MB/s 2024-03-12T04:06:51.999 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T04:06:52.056 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-12 04:06:38.756372751 +0000 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-12 03:58:45.273541429 +0000 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-12 03:58:45.273541429 +0000 2024-03-12T04:06:52.112 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-12 03:58:45.273541429 +0000 2024-03-12T04:06:52.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T04:06:52.178 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-12T04:06:52.178 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-12T04:06:52.178 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000236185 s, 2.2 MB/s 2024-03-12T04:06:52.179 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T04:06:52.235 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-12T04:06:52.235 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T04:06:52.254 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T04:06:52.254 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-03-12T04:06:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T04:06:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:52.311 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-12T04:06:52.311 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:52.311 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:52.311 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-12 04:06:28.807164092 +0000 2024-03-12T04:06:52.311 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-12 03:58:44.264654008 +0000 2024-03-12T04:06:52.311 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-12 03:58:44.264654008 +0000 2024-03-12T04:06:52.311 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-03-12 03:58:44.264654008 +0000 2024-03-12T04:06:52.311 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T04:06:52.378 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-12T04:06:52.378 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-12T04:06:52.378 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000202426 s, 2.5 MB/s 2024-03-12T04:06:52.380 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T04:06:52.438 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-03-12T04:06:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-12 04:06:28.808164070 +0000 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-12 03:58:44.599646167 +0000 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-12 03:58:44.599646167 +0000 2024-03-12T04:06:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-03-12 03:58:44.599646167 +0000 2024-03-12T04:06:52.493 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T04:06:52.558 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-12T04:06:52.559 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-12T04:06:52.559 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000199709 s, 2.6 MB/s 2024-03-12T04:06:52.560 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T04:06:52.615 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-03-12T04:06:52.671 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-12 04:06:28.808164070 +0000 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-12 03:58:44.929638442 +0000 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-12 03:58:44.929638442 +0000 2024-03-12T04:06:52.672 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-03-12 03:58:44.929638442 +0000 2024-03-12T04:06:52.672 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T04:06:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:52.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T04:06:52.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:52.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:52.734 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-12T04:06:52.734 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-12T04:06:52.734 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000191209 s, 2.7 MB/s 2024-03-12T04:06:52.735 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T04:06:52.792 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-03-12T04:06:52.847 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T04:06:52.847 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T04:06:52.847 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-12T04:06:52.847 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:06:52.848 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T04:06:52.848 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-12 04:06:28.808164070 +0000 2024-03-12T04:06:52.848 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-12 03:58:45.255630811 +0000 2024-03-12T04:06:52.848 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-12 03:58:45.255630811 +0000 2024-03-12T04:06:52.848 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-03-12 03:58:45.255630811 +0000 2024-03-12T04:06:52.848 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T04:06:52.913 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-12T04:06:52.913 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-12T04:06:52.913 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000182357 s, 2.8 MB/s 2024-03-12T04:06:52.915 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:52.971 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2024-03-12T04:06:52.971 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-12T04:06:52.999 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.999 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:52.999 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:53.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:53.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:53.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:53.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:53.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T04:06:53.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:53.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:53.219 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:53.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:53 smithi119 ceph-mon[36119]: Deploying daemon mgr.z on smithi136 2024-03-12T04:06:53.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:53 smithi119 ceph-mon[36119]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:53.952 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:53 smithi136 ceph-mon[35571]: Deploying daemon mgr.z on smithi136 2024-03-12T04:06:53.952 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:53 smithi136 ceph-mon[35571]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:53 smithi042 ceph-mon[32260]: Deploying daemon mgr.z on smithi136 2024-03-12T04:06:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:53 smithi042 ceph-mon[32260]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:54.207 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 systemd[1]: Starting Ceph mgr.z for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:06:54.486 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 podman[37408]: 2024-03-12 04:06:54.287767875 +0000 UTC m=+0.017714843 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:06:54.486 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 podman[37408]: 2024-03-12 04:06:54.391683677 +0000 UTC m=+0.121630632 container create 3682a4875a169f42e3368c4ab2fc151f7530bf4d87592527c4aaa4971a225edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-12T04:06:54.957 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 podman[37408]: 2024-03-12 04:06:54.5854151 +0000 UTC m=+0.315362055 container init 3682a4875a169f42e3368c4ab2fc151f7530bf4d87592527c4aaa4971a225edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-12T04:06:54.957 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 podman[37408]: 2024-03-12 04:06:54.587884685 +0000 UTC m=+0.317831640 container start 3682a4875a169f42e3368c4ab2fc151f7530bf4d87592527c4aaa4971a225edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-12T04:06:54.958 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 ceph-mgr[37427]: -- 172.21.15.136:0/1381587446 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573c03c81a0 con 0x5573c039f000 2024-03-12T04:06:54.958 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 bash[37408]: 3682a4875a169f42e3368c4ab2fc151f7530bf4d87592527c4aaa4971a225edb 2024-03-12T04:06:54.958 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:54 smithi136 systemd[1]: Started Ceph mgr.z for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:06:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-12T04:06:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-12T04:06:55.080 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-03-12T04:06:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-03-12T04:06:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00849905 s, 1.2 GB/s 2024-03-12T04:06:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-03-12T04:06:55.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:55.125+0000 7faf2217c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:06:55.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:55.218+0000 7faf2217c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:06:55.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2024-03-12T04:06:55.894 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:06:55.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: Deploying daemon mgr.y on smithi119 2024-03-12T04:06:55.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:55 smithi119 ceph-mon[36119]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: Deploying daemon mgr.y on smithi119 2024-03-12T04:06:56.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:55 smithi042 ceph-mon[32260]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:56.087 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:55.853+0000 7faf2217c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:06:56.087 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:55.945+0000 7faf2217c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:06:56.087 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.085+0000 7faf2217c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: Deploying daemon mgr.y on smithi119 2024-03-12T04:06:56.088 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:55 smithi136 ceph-mon[35571]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:56.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.180+0000 7faf2217c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:06:56.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.261+0000 7faf2217c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:06:56.818 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.523+0000 7faf2217c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:06:56.818 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.632+0000 7faf2217c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:06:56.818 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.718+0000 7faf2217c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:06:56.818 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.816+0000 7faf2217c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:06:56.863 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:56 smithi119 systemd[1]: Starting Ceph mgr.y for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:06:57.140 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:56 smithi119 ceph-mon[36119]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:57.140 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:56 smithi119 podman[37480]: 2024-03-12 04:06:56.861171543 +0000 UTC m=+0.113813697 container create 35294ab596e7fe167187d96baaaa40970c72691e01ca980d0e3929e53edf8566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-12T04:06:57.140 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:56 smithi119 podman[37480]: 2024-03-12 04:06:56.764675121 +0000 UTC m=+0.017317285 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:06:57.140 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:57 smithi119 podman[37480]: 2024-03-12 04:06:57.08038089 +0000 UTC m=+0.333023060 container init 35294ab596e7fe167187d96baaaa40970c72691e01ca980d0e3929e53edf8566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-12T04:06:57.140 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:57 smithi119 podman[37480]: 2024-03-12 04:06:57.082802321 +0000 UTC m=+0.335444477 container start 35294ab596e7fe167187d96baaaa40970c72691e01ca980d0e3929e53edf8566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:06:57.140 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:57 smithi119 ceph-mgr[37499]: -- 172.21.15.119:0/842000644 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558363c2e1a0 con 0x558363c04800 2024-03-12T04:06:57.140 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:57 smithi119 bash[37480]: 35294ab596e7fe167187d96baaaa40970c72691e01ca980d0e3929e53edf8566 2024-03-12T04:06:57.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-mon[35571]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:57.167 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:56.997+0000 7faf2217c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:06:57.226 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:56 smithi042 ceph-mon[32260]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 -- 172.21.15.42:0/2407116315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884101d90 msgr2=0x7f18841021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2407116315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884101d90 0x7f18841021f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18780099c0 tx=0x7f1878009cd0 comp rx=0 tx=0).stop 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 -- 172.21.15.42:0/2407116315 shutdown_connections 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2407116315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1884102730 0x7f1884108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2407116315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1884100b90 0x7f1884100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2407116315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884101d90 0x7f18841021f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 -- 172.21.15.42:0/2407116315 >> 172.21.15.42:0/2407116315 conn(0x7f18840fc130 msgr2=0x7f18840fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 -- 172.21.15.42:0/2407116315 shutdown_connections 2024-03-12T04:06:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.340+0000 7f1889a8c700 1 -- 172.21.15.42:0/2407116315 wait complete. 2024-03-12T04:06:57.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 Processor -- start 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 -- start start 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884100b90 0x7f1884104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1884101d90 0x7f1884103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1884102730 0x7f18841036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f18837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884100b90 0x7f1884104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1884103e30 con 0x7f1884100b90 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1884103fa0 con 0x7f1884101d90 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1889a8c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18841a1bb0 con 0x7f1884102730 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f18837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884100b90 0x7f1884104b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52020/0 (socket says 172.21.15.42:52020) 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f18837fe700 1 -- 172.21.15.42:0/2626080798 learned_addr learned my addr 172.21.15.42:0/2626080798 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1883fff700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1884102730 0x7f18841036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:57.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.341+0000 7f1882ffd700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1884101d90 0x7f1884103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f18837fe700 1 -- 172.21.15.42:0/2626080798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1884102730 msgr2=0x7f18841036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f18837fe700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1884102730 0x7f18841036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f18837fe700 1 -- 172.21.15.42:0/2626080798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1884101d90 msgr2=0x7f1884103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f18837fe700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1884101d90 0x7f1884103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f18837fe700 1 -- 172.21.15.42:0/2626080798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780095f0 con 0x7f1884100b90 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f18837fe700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884100b90 0x7f1884104b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f187400e970 tx=0x7f187400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874009790 con 0x7f1884100b90 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f187400c7e0 con 0x7f1884100b90 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18841a1e90 con 0x7f1884100b90 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874003c40 con 0x7f1884100b90 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.342+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18841a22a0 con 0x7f1884100b90 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.343+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f1874018430 con 0x7f1884100b90 2024-03-12T04:06:57.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.343+0000 7f1880ff9700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f186c038f90 0x7f186c03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:06:57.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.343+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188410c380 con 0x7f1884100b90 2024-03-12T04:06:57.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.343+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1874014070 con 0x7f1884100b90 2024-03-12T04:06:57.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.343+0000 7f1882ffd700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f186c038f90 0x7f186c03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:06:57.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.344+0000 7f1882ffd700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f186c038f90 0x7f186c03b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f187800b580 tx=0x7f1878005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:06:57.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.348+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f187400f990 con 0x7f1884100b90 2024-03-12T04:06:57.425 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:57 smithi119 systemd[1]: Started Ceph mgr.y for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:06:57.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:06:57.567+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f18840fe360 con 0x7f186c038f90 2024-03-12T04:06:57.695 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:57 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:57.656+0000 7fd73e826200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:06:58.196 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:57 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:57.754+0000 7fd73e826200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:06:58.275 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:58.041+0000 7faf2217c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:06:58.275 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:58.179+0000 7faf2217c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:06:58.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:06:58.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:58.440 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:06:58.441 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:58.543 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:58.449+0000 7fd73e826200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:06:58.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:58.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:06:58.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:06:58.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:06:58.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:06:58.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:58.819 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:58.524+0000 7faf2217c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:06:58.820 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:58.615+0000 7faf2217c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:06:58.820 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:58.817+0000 7faf2217c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:06:58.877 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:58.543+0000 7fd73e826200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:06:58.877 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:58.684+0000 7fd73e826200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:06:58.877 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:58.790+0000 7fd73e826200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:06:58.877 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:58.874+0000 7fd73e826200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:06:59.139 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:59.137+0000 7fd73e826200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:06:59.207 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:59.004+0000 7faf2217c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:06:59.426 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:59.245+0000 7fd73e826200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:06:59.427 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:59.331+0000 7fd73e826200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:06:59.427 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:59.424+0000 7fd73e826200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:06:59.547 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:59.286+0000 7faf2217c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:06:59.548 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:59.545+0000 7faf2217c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:06:59.695 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:06:59.597+0000 7fd73e826200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:06:59.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:59.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:59.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:59.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:59.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:59.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:06:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:59.957 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:06:59.778+0000 7faf2217c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:06:59.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:06:59.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:06:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T04:07:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:06:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:00.833 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:07:00.558+0000 7faf2217c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:07:00.833 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:00.558+0000 7fd73e826200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:07:00.834 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:00.687+0000 7fd73e826200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:07:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:00 smithi042 ceph-mon[32260]: Reconfiguring mgr.x (unknown last config time)... 2024-03-12T04:07:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:00 smithi042 ceph-mon[32260]: Reconfiguring daemon mgr.x on smithi042 2024-03-12T04:07:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:00 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3125076628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e5b58b-4170-4df0-a9b2-5abe5a61884c"}]: dispatch 2024-03-12T04:07:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:00 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3125076628' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66e5b58b-4170-4df0-a9b2-5abe5a61884c"}]': finished 2024-03-12T04:07:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:00 smithi042 ceph-mon[32260]: osdmap e5: 1 total, 0 up, 1 in 2024-03-12T04:07:01.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:01.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-mon[36119]: Reconfiguring mgr.x (unknown last config time)... 2024-03-12T04:07:01.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-mon[36119]: Reconfiguring daemon mgr.x on smithi042 2024-03-12T04:07:01.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:01.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3125076628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e5b58b-4170-4df0-a9b2-5abe5a61884c"}]: dispatch 2024-03-12T04:07:01.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3125076628' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66e5b58b-4170-4df0-a9b2-5abe5a61884c"}]': finished 2024-03-12T04:07:01.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-mon[36119]: osdmap e5: 1 total, 0 up, 1 in 2024-03-12T04:07:01.107 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:00 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:01.107 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:01.016+0000 7fd73e826200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:07:01.107 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:01.103+0000 7fd73e826200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:07:01.123 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-mon[35571]: Reconfiguring mgr.x (unknown last config time)... 2024-03-12T04:07:01.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-mon[35571]: Reconfiguring daemon mgr.x on smithi042 2024-03-12T04:07:01.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:01.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3125076628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e5b58b-4170-4df0-a9b2-5abe5a61884c"}]: dispatch 2024-03-12T04:07:01.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3125076628' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66e5b58b-4170-4df0-a9b2-5abe5a61884c"}]': finished 2024-03-12T04:07:01.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-mon[35571]: osdmap e5: 1 total, 0 up, 1 in 2024-03-12T04:07:01.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:00 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:01.430 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:07:01.121+0000 7faf2217c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:07:01.430 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:07:01.217+0000 7faf2217c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:07:01.430 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:07:01.427+0000 7faf2217c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:07:01.445 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:01.296+0000 7fd73e826200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:07:01.758 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:01.476+0000 7fd73e826200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:07:01.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:01.842+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 14) v1 ==== 91024+0+0 (secure 0 0 0) 0x7f1874021430 con 0x7f1884100b90 2024-03-12T04:07:02.014 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:01.756+0000 7fd73e826200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:07:02.014 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:02 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:02.011+0000 7fd73e826200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:07:02.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-mon[36119]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:02.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2926602122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:02.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-mon[36119]: Standby manager daemon z started 2024-03-12T04:07:02.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T04:07:02.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:07:02.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T04:07:02.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:01 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:07:02.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-mon[35571]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:02.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2926602122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:02.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-mon[35571]: Standby manager daemon z started 2024-03-12T04:07:02.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T04:07:02.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:07:02.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T04:07:02.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:01 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:07:02.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:01 smithi042 ceph-mon[32260]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:02.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:01 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2926602122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:02.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:01 smithi042 ceph-mon[32260]: Standby manager daemon z started 2024-03-12T04:07:02.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:01 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T04:07:02.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:01 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:07:02.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:01 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T04:07:02.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:01 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/3644212238' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:07:02.445 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:02 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:02.244+0000 7fd73e826200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:07:03.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:02 smithi119 ceph-mon[36119]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-12T04:07:03.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T04:07:03.196 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:03 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:03.027+0000 7fd73e826200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:07:03.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:02 smithi136 ceph-mon[35571]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-12T04:07:03.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T04:07:03.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:02 smithi042 ceph-mon[32260]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-12T04:07:03.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T04:07:03.845 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:03 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:03.588+0000 7fd73e826200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:07:03.845 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:03 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:03.686+0000 7fd73e826200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:07:04.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:03 smithi119 ceph-mon[36119]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:04.196 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:07:03 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:07:03.899+0000 7fd73e826200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:07:04.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:03 smithi136 ceph-mon[35571]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:04.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:03 smithi042 ceph-mon[32260]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:04.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:04.853+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f187401cd10 con 0x7f1884100b90 2024-03-12T04:07:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:04 smithi042 ceph-mon[32260]: Standby manager daemon y started 2024-03-12T04:07:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:04 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T04:07:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:04 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:07:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:04 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T04:07:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:04 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:07:05.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:04 smithi119 ceph-mon[36119]: Standby manager daemon y started 2024-03-12T04:07:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:04 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T04:07:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:04 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:07:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:04 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T04:07:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:04 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:07:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:04 smithi136 ceph-mon[35571]: Standby manager daemon y started 2024-03-12T04:07:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:04 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T04:07:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:04 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:07:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:04 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T04:07:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:04 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/2881906412' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:07:06.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:05 smithi042 ceph-mon[32260]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:06.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:05 smithi042 ceph-mon[32260]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-12T04:07:06.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T04:07:06.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:05 smithi119 ceph-mon[36119]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:06.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:05 smithi119 ceph-mon[36119]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-12T04:07:06.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T04:07:06.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:05 smithi136 ceph-mon[35571]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:06.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:05 smithi136 ceph-mon[35571]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-12T04:07:06.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T04:07:07.272 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:07.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:07.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:07.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:07 smithi042 ceph-mon[32260]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:07 smithi119 ceph-mon[36119]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:07 smithi136 ceph-mon[35571]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:08.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:08 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T04:07:08.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:08 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:08.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:08 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T04:07:08.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:08 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:09.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:08 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T04:07:09.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:08 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:09.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:09 smithi042 ceph-mon[32260]: Deploying daemon osd.0 on smithi042 2024-03-12T04:07:09.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:09 smithi042 ceph-mon[32260]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:09.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:09 smithi119 ceph-mon[36119]: Deploying daemon osd.0 on smithi042 2024-03-12T04:07:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:09 smithi119 ceph-mon[36119]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:09.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:09 smithi136 ceph-mon[35571]: Deploying daemon osd.0 on smithi042 2024-03-12T04:07:09.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:09 smithi136 ceph-mon[35571]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:10 smithi042 ceph-mon[32260]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:11.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:10 smithi119 ceph-mon[36119]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:11.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:10 smithi136 ceph-mon[35571]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:12 smithi042 ceph-mon[32260]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:13.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:12 smithi119 ceph-mon[36119]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:12 smithi136 ceph-mon[35571]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:14 smithi042 ceph-mon[32260]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:15.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:15.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:15.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:15.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:14 smithi119 ceph-mon[36119]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:15.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:15.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:15.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:15.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:14 smithi136 ceph-mon[35571]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:16 smithi042 ceph-mon[32260]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:16 smithi119 ceph-mon[36119]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:16 smithi136 ceph-mon[35571]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:17.831 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2024-03-12T04:07:17.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.828+0000 7f1880ff9700 1 -- 172.21.15.42:0/2626080798 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f18840fe360 con 0x7f186c038f90 2024-03-12T04:07:17.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f186c038f90 msgr2=0x7f186c03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:17.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f186c038f90 0x7f186c03b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f187800b580 tx=0x7f1878005fb0 comp rx=0 tx=0).stop 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884100b90 msgr2=0x7f1884104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884100b90 0x7f1884104b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f187400e970 tx=0x7f187400ec80 comp rx=0 tx=0).stop 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 shutdown_connections 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f186c038f90 0x7f186c03b450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1884102730 0x7f18841036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1884100b90 0x7f1884104b20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 --2- 172.21.15.42:0/2626080798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1884101d90 0x7f1884103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 >> 172.21.15.42:0/2626080798 conn(0x7f18840fc130 msgr2=0x7f18840fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 shutdown_connections 2024-03-12T04:07:17.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:07:17.830+0000 7f1889a8c700 1 -- 172.21.15.42:0/2626080798 wait complete. 2024-03-12T04:07:18.435 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.0.service 2024-03-12T04:07:18.438 INFO:tasks.cephadm:Deploying osd.1 on smithi119 with /dev/vg_nvme/lv_4... 2024-03-12T04:07:18.438 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-12T04:07:18.684 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:07:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:18 smithi042 ceph-mon[32260]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T04:07:19.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:19.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:19.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:19.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:18 smithi119 ceph-mon[36119]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T04:07:19.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:19.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:19.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:19.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:18 smithi136 ceph-mon[35571]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T04:07:19.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:19 smithi042 ceph-mon[32260]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T04:07:19.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:19 smithi042 ceph-mon[32260]: osdmap e6: 1 total, 0 up, 1 in 2024-03-12T04:07:19.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:19 smithi042 ceph-mon[32260]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:07:19.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:19 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:19.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:19 smithi119 ceph-mon[36119]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T04:07:19.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:19 smithi119 ceph-mon[36119]: osdmap e6: 1 total, 0 up, 1 in 2024-03-12T04:07:19.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:19 smithi119 ceph-mon[36119]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:07:19.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:19 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:19.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:19 smithi119 ceph-mon[36119]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:20.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:19 smithi042 ceph-mon[32260]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:20.055 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:07:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0[44527]: 2024-03-12T04:07:19.797+0000 7f3ff42fa700 -1 osd.0 0 waiting for initial osdmap 2024-03-12T04:07:20.055 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:07:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0[44527]: 2024-03-12T04:07:19.803+0000 7f3ff00ed700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:07:20.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:19 smithi136 ceph-mon[35571]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T04:07:20.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:19 smithi136 ceph-mon[35571]: osdmap e6: 1 total, 0 up, 1 in 2024-03-12T04:07:20.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:19 smithi136 ceph-mon[35571]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:07:20.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:19 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:20.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:19 smithi136 ceph-mon[35571]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:20.482 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-12T04:07:20.482 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-12T04:07:20.482 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-12T04:07:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-12T04:07:20.483 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013041 s, 804 MB/s 2024-03-12T04:07:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-12T04:07:21.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:20 smithi042 ceph-mon[32260]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:07:21.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:20 smithi042 ceph-mon[32260]: osdmap e7: 1 total, 0 up, 1 in 2024-03-12T04:07:21.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:20 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:20 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.080 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:20 smithi119 ceph-mon[36119]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:07:21.080 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:20 smithi119 ceph-mon[36119]: osdmap e7: 1 total, 0 up, 1 in 2024-03-12T04:07:21.080 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:20 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.080 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:20 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2024-03-12T04:07:21.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:20 smithi136 ceph-mon[35571]: from='osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:07:21.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:20 smithi136 ceph-mon[35571]: osdmap e7: 1 total, 0 up, 1 in 2024-03-12T04:07:21.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:20 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:20 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:07:21.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] boot 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: osdmap e8: 1 total, 1 up, 1 in 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] boot 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: osdmap e8: 1 total, 1 up, 1 in 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:07:22.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:22.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:22.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:07:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: osd.0 [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] boot 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: osdmap e8: 1 total, 1 up, 1 in 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:22.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:22.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.815+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/2225966195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 msgr2=0x7f0d281003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:22.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.815+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/2225966195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281003d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009990 tx=0x7f0d18009ca0 comp rx=0 tx=0).stop 2024-03-12T04:07:22.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/2225966195 shutdown_connections 2024-03-12T04:07:22.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/2225966195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d28103930 0x7f0d2810a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:22.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/2225966195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281003d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:22.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/2225966195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d28102f90 0x7f0d281033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:22.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/2225966195 >> 172.21.15.119:0/2225966195 conn(0x7f0d280fb910 msgr2=0x7f0d280fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:07:22.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/2225966195 shutdown_connections 2024-03-12T04:07:22.819 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:22 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi042 2024-03-12T04:07:22.819 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:22 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi042 to 17093M 2024-03-12T04:07:22.819 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:22 smithi119 ceph-mon[36119]: osdmap e9: 1 total, 1 up, 1 in 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/2225966195 wait complete. 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 Processor -- start 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.816+0000 7f0d2cfb4700 1 -- start start 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2cfb4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2cfb4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d28102f90 0x7f0d28100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2cfb4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d28103930 0x7f0d28101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2cfb4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d28101c00 con 0x7f0d28103930 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2cfb4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d28101d70 con 0x7f0d280fffb0 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2cfb4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d281a1c90 con 0x7f0d28102f90 2024-03-12T04:07:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60596/0 (socket says 172.21.15.119:60596) 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 -- 172.21.15.119:0/4034138244 learned_addr learned my addr 172.21.15.119:0/4034138244 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d26d9d700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d28103930 0x7f0d28101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d25d9b700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d28102f90 0x7f0d28100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 -- 172.21.15.119:0/4034138244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d28102f90 msgr2=0x7f0d28100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d28102f90 0x7f0d28100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 -- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d28103930 msgr2=0x7f0d28101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d28103930 0x7f0d28101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 -- 172.21.15.119:0/4034138244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180095f0 con 0x7f0d280fffb0 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d25d9b700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d28102f90 0x7f0d28100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2659c700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281028f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d18003820 tx=0x7f0d18005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d177fe700 1 -- 172.21.15.119:0/4034138244 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1801d070 con 0x7f0d280fffb0 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d177fe700 1 -- 172.21.15.119:0/4034138244 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0d18004140 con 0x7f0d280fffb0 2024-03-12T04:07:22.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.817+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d281a1f10 con 0x7f0d280fffb0 2024-03-12T04:07:22.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.818+0000 7f0d177fe700 1 -- 172.21.15.119:0/4034138244 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18022d10 con 0x7f0d280fffb0 2024-03-12T04:07:22.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.818+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d281a22c0 con 0x7f0d280fffb0 2024-03-12T04:07:22.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.819+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d28068930 con 0x7f0d280fffb0 2024-03-12T04:07:22.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.820+0000 7f0d177fe700 1 -- 172.21.15.119:0/4034138244 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f0d18022470 con 0x7f0d280fffb0 2024-03-12T04:07:22.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.820+0000 7f0d177fe700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0d100a1ab0 0x7f0d100a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:22.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.820+0000 7f0d177fe700 1 -- 172.21.15.119:0/4034138244 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f0d180ccf90 con 0x7f0d280fffb0 2024-03-12T04:07:22.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.820+0000 7f0d25d9b700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0d100a1ab0 0x7f0d100a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:22.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.821+0000 7f0d25d9b700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0d100a1ab0 0x7f0d100a3f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c005fd0 tx=0x7f0d1c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:07:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:22.825+0000 7f0d177fe700 1 -- 172.21.15.119:0/4034138244 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d18097dc0 con 0x7f0d280fffb0 2024-03-12T04:07:23.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:23.025+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d2802d950 con 0x7f0d100a1ab0 2024-03-12T04:07:23.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:22 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi042 2024-03-12T04:07:23.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:22 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi042 to 17093M 2024-03-12T04:07:23.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:22 smithi042 ceph-mon[32260]: osdmap e9: 1 total, 1 up, 1 in 2024-03-12T04:07:23.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:22 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi042 2024-03-12T04:07:23.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:22 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi042 to 17093M 2024-03-12T04:07:23.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:22 smithi136 ceph-mon[35571]: osdmap e9: 1 total, 1 up, 1 in 2024-03-12T04:07:23.893 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:23 smithi119 ceph-mon[36119]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:23.893 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:07:23.893 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:07:23.893 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:24.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:23 smithi136 ceph-mon[35571]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:24.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:07:24.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:07:24.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:24.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:23 smithi042 ceph-mon[32260]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:24.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:07:24.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:07:24.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:25.123 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:24 smithi136 ceph-mon[35571]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:07:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:24 smithi119 ceph-mon[36119]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:07:25.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:24 smithi042 ceph-mon[32260]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:07:26.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:25 smithi119 ceph-mon[36119]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:26.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:25 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/3531350339' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]: dispatch 2024-03-12T04:07:26.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:25 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]: dispatch 2024-03-12T04:07:26.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:25 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]': finished 2024-03-12T04:07:26.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:25 smithi119 ceph-mon[36119]: osdmap e10: 2 total, 1 up, 2 in 2024-03-12T04:07:26.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:25 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:25 smithi136 ceph-mon[35571]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:25 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/3531350339' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]: dispatch 2024-03-12T04:07:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:25 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]: dispatch 2024-03-12T04:07:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:25 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]': finished 2024-03-12T04:07:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:25 smithi136 ceph-mon[35571]: osdmap e10: 2 total, 1 up, 2 in 2024-03-12T04:07:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:25 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:26.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:25 smithi042 ceph-mon[32260]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:26.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:25 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/3531350339' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]: dispatch 2024-03-12T04:07:26.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:25 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]: dispatch 2024-03-12T04:07:26.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:25 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e1a8d54-070c-49ef-a2a6-f2fe66c292ce"}]': finished 2024-03-12T04:07:26.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:25 smithi042 ceph-mon[32260]: osdmap e10: 2 total, 1 up, 2 in 2024-03-12T04:07:26.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:25 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:27.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:26 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/2712775752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:27.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:26 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/2712775752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:27.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:26 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/2712775752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:28.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:27 smithi119 ceph-mon[36119]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:27 smithi136 ceph-mon[35571]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:28.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:27 smithi042 ceph-mon[32260]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:29.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:28 smithi119 ceph-mon[36119]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:29.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:28 smithi136 ceph-mon[35571]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:29.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:28 smithi042 ceph-mon[32260]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:30 smithi042 ceph-mon[32260]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:31.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:30 smithi119 ceph-mon[36119]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:30 smithi136 ceph-mon[35571]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:32.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T04:07:32.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:33.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T04:07:33.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T04:07:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:33.899 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:33 smithi119 ceph-mon[36119]: Deploying daemon osd.1 on smithi119 2024-03-12T04:07:33.899 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:33 smithi119 ceph-mon[36119]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:33 smithi042 ceph-mon[32260]: Deploying daemon osd.1 on smithi119 2024-03-12T04:07:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:33 smithi042 ceph-mon[32260]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:34.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:33 smithi136 ceph-mon[35571]: Deploying daemon osd.1 on smithi119 2024-03-12T04:07:34.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:33 smithi136 ceph-mon[35571]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:35.181 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:34 smithi119 ceph-mon[36119]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:34 smithi136 ceph-mon[35571]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:34 smithi042 ceph-mon[32260]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:37.154 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:36 smithi119 ceph-mon[36119]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:37.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:36 smithi136 ceph-mon[35571]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:37.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:36 smithi042 ceph-mon[32260]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:38 smithi119 ceph-mon[36119]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:38 smithi136 ceph-mon[35571]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:38 smithi042 ceph-mon[32260]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.344+0000 7f0d177fe700 1 -- 172.21.15.119:0/4034138244 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0d2802d950 con 0x7f0d100a1ab0 2024-03-12T04:07:41.347 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 1 on host 'smithi119' 2024-03-12T04:07:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0d100a1ab0 msgr2=0x7f0d100a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0d100a1ab0 0x7f0d100a3f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c005fd0 tx=0x7f0d1c005ee0 comp rx=0 tx=0).stop 2024-03-12T04:07:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 msgr2=0x7f0d281028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281028f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d18003820 tx=0x7f0d18005ef0 comp rx=0 tx=0).stop 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 shutdown_connections 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0d100a1ab0 0x7f0d100a3f70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d28103930 0x7f0d28101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280fffb0 0x7f0d281028f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 --2- 172.21.15.119:0/4034138244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d28102f90 0x7f0d28100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 >> 172.21.15.119:0/4034138244 conn(0x7f0d280fb910 msgr2=0x7f0d28106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 shutdown_connections 2024-03-12T04:07:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:07:41.346+0000 7f0d2cfb4700 1 -- 172.21.15.119:0/4034138244 wait complete. 2024-03-12T04:07:41.637 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:41.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:41.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:41.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:07:41.988 DEBUG:teuthology.orchestra.run.smithi119:osd.1> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.1.service 2024-03-12T04:07:41.991 INFO:tasks.cephadm:Deploying osd.2 on smithi136 with /dev/vg_nvme/lv_4... 2024-03-12T04:07:41.991 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-12T04:07:42.234 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:07:42.678 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:42 smithi136 ceph-mon[35571]: from='osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T04:07:42.678 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:42 smithi136 ceph-mon[35571]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T04:07:42.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:42 smithi119 ceph-mon[36119]: from='osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T04:07:42.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:42 smithi119 ceph-mon[36119]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T04:07:42.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:42 smithi042 ceph-mon[32260]: from='osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T04:07:42.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:42 smithi042 ceph-mon[32260]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T04:07:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:43 smithi042 ceph-mon[32260]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T04:07:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:43 smithi042 ceph-mon[32260]: from='osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:07:43.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:43 smithi042 ceph-mon[32260]: osdmap e11: 2 total, 1 up, 2 in 2024-03-12T04:07:43.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:43.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:43 smithi042 ceph-mon[32260]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:07:43.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:43 smithi042 ceph-mon[32260]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:43.817 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-mon[36119]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T04:07:43.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-mon[36119]: from='osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:07:43.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-mon[36119]: osdmap e11: 2 total, 1 up, 2 in 2024-03-12T04:07:43.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:43.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-mon[36119]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:07:43.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-mon[36119]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:43.818 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1[40703]: 2024-03-12T04:07:43.546+0000 7f6731831700 -1 osd.1 0 waiting for initial osdmap 2024-03-12T04:07:43.818 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:07:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1[40703]: 2024-03-12T04:07:43.550+0000 7f672d624700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:07:43.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:43 smithi136 ceph-mon[35571]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T04:07:43.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:43 smithi136 ceph-mon[35571]: from='osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:07:43.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:43 smithi136 ceph-mon[35571]: osdmap e11: 2 total, 1 up, 2 in 2024-03-12T04:07:43.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:43.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:43 smithi136 ceph-mon[35571]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:07:43.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:43 smithi136 ceph-mon[35571]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:44.070 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-12T04:07:44.070 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-12T04:07:44.071 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-03-12T04:07:44.071 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-03-12T04:07:44.071 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00959924 s, 1.1 GB/s 2024-03-12T04:07:44.071 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-03-12T04:07:44.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:44 smithi119 ceph-mon[36119]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:07:44.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:44 smithi119 ceph-mon[36119]: osdmap e12: 2 total, 1 up, 2 in 2024-03-12T04:07:44.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:44.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:44.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:44 smithi136 ceph-mon[35571]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:07:44.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:44 smithi136 ceph-mon[35571]: osdmap e12: 2 total, 1 up, 2 in 2024-03-12T04:07:44.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:44.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:44.672 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-03-12T04:07:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:44 smithi042 ceph-mon[32260]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:07:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:44 smithi042 ceph-mon[32260]: osdmap e12: 2 total, 1 up, 2 in 2024-03-12T04:07:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:44.921 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:07:45.676 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] boot 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: osdmap e13: 2 total, 2 up, 2 in 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] boot 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: osdmap e13: 2 total, 2 up, 2 in 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:45.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: osd.1 [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] boot 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: osdmap e13: 2 total, 2 up, 2 in 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:07:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 -- 172.21.15.136:0/2047721718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8102740 msgr2=0x7f67c8108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 --2- 172.21.15.136:0/2047721718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8102740 0x7f67c8108ff0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f67c000b0a0 tx=0x7f67c000b3b0 comp rx=0 tx=0).stop 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 -- 172.21.15.136:0/2047721718 shutdown_connections 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 --2- 172.21.15.136:0/2047721718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8102740 0x7f67c8108ff0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 --2- 172.21.15.136:0/2047721718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67c8100ba0 0x7f67c8100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 --2- 172.21.15.136:0/2047721718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67c8101da0 0x7f67c8102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 -- 172.21.15.136:0/2047721718 >> 172.21.15.136:0/2047721718 conn(0x7f67c80fc140 msgr2=0x7f67c80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 -- 172.21.15.136:0/2047721718 shutdown_connections 2024-03-12T04:07:46.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.438+0000 7f67ce374700 1 -- 172.21.15.136:0/2047721718 wait complete. 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 Processor -- start 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 -- start start 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8100ba0 0x7f67c8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67c8101da0 0x7f67c8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67c8102740 0x7f67c819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819f770 con 0x7f67c8102740 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819f8b0 con 0x7f67c8101da0 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67ce374700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819fa70 con 0x7f67c8100ba0 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67c7fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8100ba0 0x7f67c8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67c7fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8100ba0 0x7f67c8197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:40882/0 (socket says 172.21.15.136:40882) 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67c7fff700 1 -- 172.21.15.136:0/2113008649 learned_addr learned my addr 172.21.15.136:0/2113008649 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67c77fe700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67c8101da0 0x7f67c8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.439+0000 7f67cc911700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67c8102740 0x7f67c819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c7fff700 1 -- 172.21.15.136:0/2113008649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67c8101da0 msgr2=0x7f67c8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c7fff700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67c8101da0 0x7f67c8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:46.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c7fff700 1 -- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67c8102740 msgr2=0x7f67c819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c7fff700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67c8102740 0x7f67c819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c7fff700 1 -- 172.21.15.136:0/2113008649 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c0009d00 con 0x7f67c8100ba0 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c7fff700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8100ba0 0x7f67c8197c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c990 tx=0x7f67b800cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c57fa700 1 -- 172.21.15.136:0/2113008649 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8007b00 con 0x7f67c8100ba0 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c57fa700 1 -- 172.21.15.136:0/2113008649 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f67b8007c60 con 0x7f67c8100ba0 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c819fd50 con 0x7f67c8100ba0 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67c57fa700 1 -- 172.21.15.136:0/2113008649 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8018660 con 0x7f67c8100ba0 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.440+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c81a01e0 con 0x7f67c8100ba0 2024-03-12T04:07:46.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.441+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c8068930 con 0x7f67c8100ba0 2024-03-12T04:07:46.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.442+0000 7f67c57fa700 1 -- 172.21.15.136:0/2113008649 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f67b800f450 con 0x7f67c8100ba0 2024-03-12T04:07:46.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.443+0000 7f67c57fa700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f67b00a1ab0 0x7f67b00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:07:46.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.443+0000 7f67c57fa700 1 -- 172.21.15.136:0/2113008649 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f67b80cbc10 con 0x7f67c8100ba0 2024-03-12T04:07:46.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.443+0000 7f67c77fe700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f67b00a1ab0 0x7f67b00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:07:46.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.444+0000 7f67c77fe700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f67b00a1ab0 0x7f67b00a3f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67bc005950 tx=0x7f67bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:07:46.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.447+0000 7f67c57fa700 1 -- 172.21.15.136:0/2113008649 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67b80966e0 con 0x7f67c8100ba0 2024-03-12T04:07:46.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:07:46.646+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f67c802d9b0 con 0x7f67b00a1ab0 2024-03-12T04:07:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:46 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi119 2024-03-12T04:07:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:46 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi119 to 17088M 2024-03-12T04:07:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:46 smithi042 ceph-mon[32260]: osdmap e14: 2 total, 2 up, 2 in 2024-03-12T04:07:46.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:46 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi119 2024-03-12T04:07:46.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:46 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi119 to 17088M 2024-03-12T04:07:46.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:46 smithi119 ceph-mon[36119]: osdmap e14: 2 total, 2 up, 2 in 2024-03-12T04:07:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:46 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi119 2024-03-12T04:07:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:46 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi119 to 17088M 2024-03-12T04:07:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:46 smithi136 ceph-mon[35571]: osdmap e14: 2 total, 2 up, 2 in 2024-03-12T04:07:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:47 smithi042 ceph-mon[32260]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:07:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:07:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:07:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:47 smithi042 ceph-mon[32260]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:47.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:47 smithi136 ceph-mon[35571]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:07:47.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:07:47.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:07:47.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:47.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:47 smithi136 ceph-mon[35571]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:47.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:47 smithi119 ceph-mon[36119]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:07:47.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:07:47.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:07:47.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:47.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:47 smithi119 ceph-mon[36119]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:49.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:48 smithi119 ceph-mon[36119]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:49.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:48 smithi136 ceph-mon[35571]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:48 smithi042 ceph-mon[32260]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:50.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:49 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/190035286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]: dispatch 2024-03-12T04:07:50.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:49 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]: dispatch 2024-03-12T04:07:50.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:49 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]': finished 2024-03-12T04:07:50.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:49 smithi119 ceph-mon[36119]: osdmap e15: 3 total, 2 up, 3 in 2024-03-12T04:07:50.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:07:50.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:49 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/3854786706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:50.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:49 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/190035286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]: dispatch 2024-03-12T04:07:50.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:49 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]: dispatch 2024-03-12T04:07:50.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:49 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]': finished 2024-03-12T04:07:50.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:49 smithi136 ceph-mon[35571]: osdmap e15: 3 total, 2 up, 3 in 2024-03-12T04:07:50.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:07:50.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:49 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/3854786706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:50.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:49 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/190035286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]: dispatch 2024-03-12T04:07:50.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:49 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]: dispatch 2024-03-12T04:07:50.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:49 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb3ebf6a-d807-4ead-89b0-bd81dcf650e5"}]': finished 2024-03-12T04:07:50.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:49 smithi042 ceph-mon[32260]: osdmap e15: 3 total, 2 up, 3 in 2024-03-12T04:07:50.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:07:50.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:49 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/3854786706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:07:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:50 smithi119 ceph-mon[36119]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:50 smithi136 ceph-mon[35571]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:50 smithi042 ceph-mon[32260]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:53.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:52 smithi119 ceph-mon[36119]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:53.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:52 smithi136 ceph-mon[35571]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:53.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:52 smithi042 ceph-mon[32260]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:55.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:54 smithi136 ceph-mon[35571]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:55.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:54 smithi119 ceph-mon[36119]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:55.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:54 smithi042 ceph-mon[32260]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:56.609 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:56 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T04:07:56.610 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:56 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:56.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:56 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T04:07:56.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:56 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:56.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:56 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T04:07:56.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:56 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:07:57.573 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:57 smithi136 ceph-mon[35571]: Deploying daemon osd.2 on smithi136 2024-03-12T04:07:57.573 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:57 smithi136 ceph-mon[35571]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:57.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:57 smithi119 ceph-mon[36119]: Deploying daemon osd.2 on smithi136 2024-03-12T04:07:57.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:57 smithi119 ceph-mon[36119]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:57.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:57 smithi042 ceph-mon[32260]: Deploying daemon osd.2 on smithi136 2024-03-12T04:07:57.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:57 smithi042 ceph-mon[32260]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:59.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:07:58 smithi136 ceph-mon[35571]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:59.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:07:58 smithi119 ceph-mon[36119]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:07:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:07:58 smithi042 ceph-mon[32260]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:01.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:00 smithi136 ceph-mon[35571]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:01.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:00 smithi119 ceph-mon[36119]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:01.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:00 smithi042 ceph-mon[32260]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:03.025 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:03.025 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:03.025 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:03.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:03.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:03.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:03.906 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:03 smithi136 ceph-mon[35571]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:04.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:03 smithi042 ceph-mon[32260]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:04.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:03 smithi119 ceph-mon[36119]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:05.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.514+0000 7f67c57fa700 1 -- 172.21.15.136:0/2113008649 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f67c802d9b0 con 0x7f67b00a1ab0 2024-03-12T04:08:05.517 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 2 on host 'smithi136' 2024-03-12T04:08:05.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f67b00a1ab0 msgr2=0x7f67b00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:05.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f67b00a1ab0 0x7f67b00a3f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67bc005950 tx=0x7f67bc00b410 comp rx=0 tx=0).stop 2024-03-12T04:08:05.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8100ba0 msgr2=0x7f67c8197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:05.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8100ba0 0x7f67c8197c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c990 tx=0x7f67b800cca0 comp rx=0 tx=0).stop 2024-03-12T04:08:05.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 shutdown_connections 2024-03-12T04:08:05.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f67b00a1ab0 0x7f67b00a3f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:05.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67c8102740 0x7f67c819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:05.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c8100ba0 0x7f67c8197c20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:05.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 --2- 172.21.15.136:0/2113008649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67c8101da0 0x7f67c8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:05.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 >> 172.21.15.136:0/2113008649 conn(0x7f67c80fc140 msgr2=0x7f67c81050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:08:05.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 shutdown_connections 2024-03-12T04:08:05.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:08:05.517+0000 7f67ce374700 1 -- 172.21.15.136:0/2113008649 wait complete. 2024-03-12T04:08:05.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:05.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:05.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:05.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:05 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:05.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:05 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:06.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:06.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:06.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:06.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:06.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:06.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:06.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:06.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:06.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:05 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:06.163 DEBUG:teuthology.orchestra.run.smithi136:osd.2> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.2.service 2024-03-12T04:08:06.165 INFO:tasks.cephadm:Deploying osd.3 on smithi042 with /dev/vg_nvme/lv_3... 2024-03-12T04:08:06.165 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-12T04:08:06.417 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:08:06.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:06 smithi042 ceph-mon[32260]: from='osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T04:08:06.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:06 smithi042 ceph-mon[32260]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T04:08:06.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:06 smithi119 ceph-mon[36119]: from='osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T04:08:06.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:06 smithi119 ceph-mon[36119]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T04:08:06.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:06 smithi136 ceph-mon[35571]: from='osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T04:08:06.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:06 smithi136 ceph-mon[35571]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T04:08:07.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:07 smithi119 ceph-mon[36119]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T04:08:07.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:07 smithi119 ceph-mon[36119]: osdmap e16: 3 total, 2 up, 3 in 2024-03-12T04:08:07.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:07.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:07 smithi119 ceph-mon[36119]: from='osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:08:07.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:07 smithi119 ceph-mon[36119]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:08:07.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:07 smithi119 ceph-mon[36119]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:07.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-mon[35571]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T04:08:07.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-mon[35571]: osdmap e16: 3 total, 2 up, 3 in 2024-03-12T04:08:07.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:07.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-mon[35571]: from='osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:08:07.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-mon[35571]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:08:07.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-mon[35571]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:07.958 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2[40714]: 2024-03-12T04:08:07.600+0000 7fa1a0e55700 -1 osd.2 0 waiting for initial osdmap 2024-03-12T04:08:07.958 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2[40714]: 2024-03-12T04:08:07.605+0000 7fa19cc48700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:08:07.968 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:07 smithi042 ceph-mon[32260]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T04:08:07.968 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:07 smithi042 ceph-mon[32260]: osdmap e16: 3 total, 2 up, 3 in 2024-03-12T04:08:07.968 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:07.968 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:07 smithi042 ceph-mon[32260]: from='osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:08:07.968 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:07 smithi042 ceph-mon[32260]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:08:07.969 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:07 smithi042 ceph-mon[32260]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-12T04:08:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-12T04:08:08.705 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-03-12T04:08:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-03-12T04:08:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122677 s, 855 MB/s 2024-03-12T04:08:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-03-12T04:08:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:08 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:08:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:08 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:08:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:08 smithi136 ceph-mon[35571]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:08:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:08 smithi136 ceph-mon[35571]: osdmap e17: 3 total, 2 up, 3 in 2024-03-12T04:08:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:08 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:08 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:08.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:08 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:08:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:08 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:08:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:08 smithi042 ceph-mon[32260]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:08:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:08 smithi042 ceph-mon[32260]: osdmap e17: 3 total, 2 up, 3 in 2024-03-12T04:08:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:08 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:08 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:08.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:08 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:08:08.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:08 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:08:08.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:08 smithi119 ceph-mon[36119]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:08:08.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:08 smithi119 ceph-mon[36119]: osdmap e17: 3 total, 2 up, 3 in 2024-03-12T04:08:08.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:08 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:08.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:08 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:09.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2024-03-12T04:08:09.583 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] boot 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: osdmap e18: 3 total, 3 up, 3 in 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:09.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:09.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] boot 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: osdmap e18: 3 total, 3 up, 3 in 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:09.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:09.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:09.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: osd.2 [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] boot 2024-03-12T04:08:09.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: osdmap e18: 3 total, 3 up, 3 in 2024-03-12T04:08:09.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:08:09.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T04:08:09.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:08:09.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:09.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:09.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:09.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:09.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:10.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:10 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi136 2024-03-12T04:08:10.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:10 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi136 to 17088M 2024-03-12T04:08:10.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T04:08:10.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:10 smithi042 ceph-mon[32260]: osdmap e19: 3 total, 3 up, 3 in 2024-03-12T04:08:10.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:08:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:10 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi136 2024-03-12T04:08:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:10 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi136 to 17088M 2024-03-12T04:08:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T04:08:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:10 smithi119 ceph-mon[36119]: osdmap e19: 3 total, 3 up, 3 in 2024-03-12T04:08:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:08:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:10 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi136 2024-03-12T04:08:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:10 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi136 to 17088M 2024-03-12T04:08:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T04:08:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:10 smithi136 ceph-mon[35571]: osdmap e19: 3 total, 3 up, 3 in 2024-03-12T04:08:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:08:10.994 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:10 smithi042 sudo[49075]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T04:08:10.994 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:10 smithi042 sudo[49075]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:10.995 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:10 smithi042 sudo[49075]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:10.995 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:10 smithi042 sudo[49075]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:11.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 -- 172.21.15.42:0/3994179363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39681050d0 msgr2=0x7f39681074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 --2- 172.21.15.42:0/3994179363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39681050d0 0x7f39681074c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f395c009990 tx=0x7f395c009ca0 comp rx=0 tx=0).stop 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 -- 172.21.15.42:0/3994179363 shutdown_connections 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 --2- 172.21.15.42:0/3994179363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3968107a00 0x7f3968109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 --2- 172.21.15.42:0/3994179363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681027a0 0x7f3968104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 --2- 172.21.15.42:0/3994179363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39681050d0 0x7f39681074c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 -- 172.21.15.42:0/3994179363 >> 172.21.15.42:0/3994179363 conn(0x7f39680fc170 msgr2=0x7f39680fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 -- 172.21.15.42:0/3994179363 shutdown_connections 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.291+0000 7f396f2a8700 1 -- 172.21.15.42:0/3994179363 wait complete. 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 Processor -- start 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 -- start start 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39681027a0 0x7f3968199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681050d0 0x7f396819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:11.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3968107a00 0x7f39681a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39681a19a0 con 0x7f3968107a00 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39681a1ae0 con 0x7f39681050d0 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396f2a8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39681a1c70 con 0x7f39681027a0 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396d845700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3968107a00 0x7f39681a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396d845700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3968107a00 0x7f39681a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50134/0 (socket says 172.21.15.42:50134) 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396d845700 1 -- 172.21.15.42:0/1626943077 learned_addr learned my addr 172.21.15.42:0/1626943077 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396c843700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681050d0 0x7f396819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.292+0000 7f396d044700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39681027a0 0x7f3968199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396d845700 1 -- 172.21.15.42:0/1626943077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39681027a0 msgr2=0x7f3968199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396d845700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39681027a0 0x7f3968199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396d845700 1 -- 172.21.15.42:0/1626943077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681050d0 msgr2=0x7f396819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396d845700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681050d0 0x7f396819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:11.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396d845700 1 -- 172.21.15.42:0/1626943077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c0095f0 con 0x7f3968107a00 2024-03-12T04:08:11.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396d845700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3968107a00 0x7f39681a13d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f396000b480 tx=0x7f396000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:08:11.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960011830 con 0x7f3968107a00 2024-03-12T04:08:11.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f396000be50 con 0x7f3968107a00 2024-03-12T04:08:11.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39681a1f50 con 0x7f3968107a00 2024-03-12T04:08:11.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960010d60 con 0x7f3968107a00 2024-03-12T04:08:11.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.293+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39681a23c0 con 0x7f3968107a00 2024-03-12T04:08:11.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.294+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3968109580 con 0x7f3968107a00 2024-03-12T04:08:11.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.295+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f3960011990 con 0x7f3968107a00 2024-03-12T04:08:11.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.296+0000 7f39667fc700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39500a1a30 0x7f39500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:11.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.296+0000 7f396d044700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39500a1a30 0x7f39500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:11.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.296+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f39600cb8b0 con 0x7f3968107a00 2024-03-12T04:08:11.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.296+0000 7f396d044700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39500a1a30 0x7f39500a3ef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3958009550 tx=0x7f39580091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:08:11.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.301+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3960095ed0 con 0x7f3968107a00 2024-03-12T04:08:11.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:11.510+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f396802d8d0 con 0x7f39500a1a30 2024-03-12T04:08:11.512 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:11 smithi042 sudo[49085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T04:08:11.512 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:11 smithi042 sudo[49085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:11.512 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:11 smithi042 sudo[49085]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:11.512 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:08:11 smithi042 sudo[49085]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:11.776 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44830]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T04:08:11.776 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:11.776 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44830]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:11.776 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44830]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:11.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T04:08:11.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:11 smithi119 ceph-mon[36119]: osdmap e20: 3 total, 3 up, 3 in 2024-03-12T04:08:11.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:11 smithi119 ceph-mon[36119]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:11.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:11.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:08:11.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:08:11.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T04:08:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:11 smithi042 ceph-mon[32260]: osdmap e20: 3 total, 3 up, 3 in 2024-03-12T04:08:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:11 smithi042 ceph-mon[32260]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:08:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:08:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T04:08:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:11 smithi136 ceph-mon[35571]: osdmap e20: 3 total, 3 up, 3 in 2024-03-12T04:08:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:11 smithi136 ceph-mon[35571]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:08:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:08:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:12.195 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44833]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T04:08:12.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:12.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44833]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:12.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:08:11 smithi119 sudo[44833]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:12.297 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44772]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T04:08:12.297 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44772]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:12.297 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44772]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:12.297 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44772]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:12.707 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44775]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T04:08:12.707 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44775]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:12.707 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44775]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:12.707 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:08:12 smithi136 sudo[44775]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:12.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49248]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T04:08:12.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49248]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:12.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49248]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:12.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49248]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:12.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:12.862+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f39600cdce0 con 0x7f3968107a00 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49252]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49252]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 sudo[49252]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 ceph-mon[32260]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 ceph-mon[32260]: osdmap e21: 3 total, 3 up, 3 in 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:12 smithi136 ceph-mon[35571]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:12 smithi136 ceph-mon[35571]: osdmap e21: 3 total, 3 up, 3 in 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44778]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44778]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44778]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:13.116 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44778]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:13.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:12 smithi119 ceph-mon[36119]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:08:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:12 smithi119 ceph-mon[36119]: osdmap e21: 3 total, 3 up, 3 in 2024-03-12T04:08:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T04:08:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44781]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 sudo[44781]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:13.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44836]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T04:08:13.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:13.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44836]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:13.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44836]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:14.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T04:08:14.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T04:08:14.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44839]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T04:08:14.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 sudo[44839]: pam_unix(sudo:session): session closed for user root 2024-03-12T04:08:14.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:13 smithi119 ceph-mon[36119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:13 smithi136 ceph-mon[35571]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:08:14.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:13 smithi042 ceph-mon[32260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T04:08:15.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:14 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1578717988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcfd2c44-146b-4495-92e4-ad2e3c85d16d"}]: dispatch 2024-03-12T04:08:15.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:14 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1578717988' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcfd2c44-146b-4495-92e4-ad2e3c85d16d"}]': finished 2024-03-12T04:08:15.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:14 smithi119 ceph-mon[36119]: osdmap e22: 4 total, 3 up, 4 in 2024-03-12T04:08:15.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:15.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:14 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1578717988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcfd2c44-146b-4495-92e4-ad2e3c85d16d"}]: dispatch 2024-03-12T04:08:15.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:14 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1578717988' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcfd2c44-146b-4495-92e4-ad2e3c85d16d"}]': finished 2024-03-12T04:08:15.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:14 smithi136 ceph-mon[35571]: osdmap e22: 4 total, 3 up, 4 in 2024-03-12T04:08:15.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:15.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:14 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1578717988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcfd2c44-146b-4495-92e4-ad2e3c85d16d"}]: dispatch 2024-03-12T04:08:15.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:14 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1578717988' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcfd2c44-146b-4495-92e4-ad2e3c85d16d"}]': finished 2024-03-12T04:08:15.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:14 smithi042 ceph-mon[32260]: osdmap e22: 4 total, 3 up, 4 in 2024-03-12T04:08:15.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:16.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:15 smithi119 ceph-mon[36119]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:16.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:15 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3472083039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:08:16.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:15 smithi136 ceph-mon[35571]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:16.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:15 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3472083039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:08:16.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:15 smithi042 ceph-mon[32260]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:16.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:15 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3472083039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:08:18.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:17 smithi119 ceph-mon[36119]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:18.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:17 smithi136 ceph-mon[35571]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:18.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:17 smithi042 ceph-mon[32260]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:20.167 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:19 smithi042 ceph-mon[32260]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:20.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:19 smithi119 ceph-mon[36119]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:20.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:19 smithi136 ceph-mon[35571]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:22.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:21 smithi119 ceph-mon[36119]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:22.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T04:08:22.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:21 smithi136 ceph-mon[35571]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T04:08:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:22.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:21 smithi042 ceph-mon[32260]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:22.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T04:08:22.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:23.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:22 smithi119 ceph-mon[36119]: Deploying daemon osd.3 on smithi042 2024-03-12T04:08:23.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:22 smithi136 ceph-mon[35571]: Deploying daemon osd.3 on smithi042 2024-03-12T04:08:23.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:22 smithi042 ceph-mon[32260]: Deploying daemon osd.3 on smithi042 2024-03-12T04:08:24.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:23 smithi042 ceph-mon[32260]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:24.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:23 smithi119 ceph-mon[36119]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:24.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:23 smithi136 ceph-mon[35571]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:25.057 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:24 smithi042 ceph-mon[32260]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:25.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:24 smithi119 ceph-mon[36119]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:25.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:24 smithi136 ceph-mon[35571]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:27.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:26 smithi119 ceph-mon[36119]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:27.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:26 smithi136 ceph-mon[35571]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:27.211 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:26 smithi042 ceph-mon[32260]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:28.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:28 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:28.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:28 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:28.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:28 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:28.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:28 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:28.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:28 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:28.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:28 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:28 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:28 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:28 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:29.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:29 smithi042 ceph-mon[32260]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:29.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:29 smithi119 ceph-mon[36119]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:29.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:29 smithi136 ceph-mon[35571]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:31.118 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 3 on host 'smithi042' 2024-03-12T04:08:31.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.115+0000 7f39667fc700 1 -- 172.21.15.42:0/1626943077 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f396802d8d0 con 0x7f39500a1a30 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39500a1a30 msgr2=0x7f39500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39500a1a30 0x7f39500a3ef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3958009550 tx=0x7f39580091f0 comp rx=0 tx=0).stop 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3968107a00 msgr2=0x7f39681a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3968107a00 0x7f39681a13d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f396000b480 tx=0x7f396000b790 comp rx=0 tx=0).stop 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 shutdown_connections 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39500a1a30 0x7f39500a3ef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3968107a00 0x7f39681a13d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39681027a0 0x7f3968199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 --2- 172.21.15.42:0/1626943077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681050d0 0x7f396819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.118+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 >> 172.21.15.42:0/1626943077 conn(0x7f39680fc170 msgr2=0x7f3968101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:08:31.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.119+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 shutdown_connections 2024-03-12T04:08:31.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:08:31.119+0000 7f396f2a8700 1 -- 172.21.15.42:0/1626943077 wait complete. 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:31.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:31.793 DEBUG:teuthology.orchestra.run.smithi042:osd.3> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.3.service 2024-03-12T04:08:31.795 INFO:tasks.cephadm:Deploying osd.4 on smithi119 with /dev/vg_nvme/lv_3... 2024-03-12T04:08:31.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-12T04:08:32.045 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:08:32.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:32 smithi042 ceph-mon[32260]: from='osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T04:08:32.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:32 smithi042 ceph-mon[32260]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T04:08:32.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:32 smithi119 ceph-mon[36119]: from='osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T04:08:32.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:32 smithi119 ceph-mon[36119]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T04:08:32.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:32 smithi136 ceph-mon[35571]: from='osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T04:08:32.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:32 smithi136 ceph-mon[35571]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T04:08:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-mon[32260]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T04:08:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-mon[32260]: osdmap e23: 4 total, 3 up, 4 in 2024-03-12T04:08:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-mon[32260]: from='osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:08:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-mon[32260]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:08:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-mon[32260]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:33.422 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:33 smithi119 ceph-mon[36119]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T04:08:33.422 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:33 smithi119 ceph-mon[36119]: osdmap e23: 4 total, 3 up, 4 in 2024-03-12T04:08:33.423 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:33 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:33.423 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:33 smithi119 ceph-mon[36119]: from='osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:08:33.423 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:33 smithi119 ceph-mon[36119]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:08:33.423 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:33 smithi119 ceph-mon[36119]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:33 smithi136 ceph-mon[35571]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T04:08:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:33 smithi136 ceph-mon[35571]: osdmap e23: 4 total, 3 up, 4 in 2024-03-12T04:08:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:33 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:33 smithi136 ceph-mon[35571]: from='osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:08:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:33 smithi136 ceph-mon[35571]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:08:33.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:33 smithi136 ceph-mon[35571]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-12T04:08:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-12T04:08:33.980 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-12T04:08:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-12T04:08:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112812 s, 929 MB/s 2024-03-12T04:08:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-12T04:08:34.066 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:08:33.648+0000 7f65d29aa700 -1 osd.3 0 waiting for initial osdmap 2024-03-12T04:08:34.066 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:08:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:08:33.654+0000 7f65ca795700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:08:34.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:34 smithi119 ceph-mon[36119]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:08:34.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:34 smithi119 ceph-mon[36119]: osdmap e24: 4 total, 3 up, 4 in 2024-03-12T04:08:34.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:34.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:34.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:34 smithi119 ceph-mon[36119]: from='osd.3 ' entity='osd.3' 2024-03-12T04:08:34.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:34 smithi136 ceph-mon[35571]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:08:34.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:34 smithi136 ceph-mon[35571]: osdmap e24: 4 total, 3 up, 4 in 2024-03-12T04:08:34.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:34.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:34.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:34 smithi136 ceph-mon[35571]: from='osd.3 ' entity='osd.3' 2024-03-12T04:08:34.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:34 smithi042 ceph-mon[32260]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:08:34.541 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:34 smithi042 ceph-mon[32260]: osdmap e24: 4 total, 3 up, 4 in 2024-03-12T04:08:34.541 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:34.541 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:34.541 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:34 smithi042 ceph-mon[32260]: from='osd.3 ' entity='osd.3' 2024-03-12T04:08:34.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2024-03-12T04:08:34.826 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] boot 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: osdmap e25: 4 total, 4 up, 4 in 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:35.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: osdmap e26: 4 total, 4 up, 4 in 2024-03-12T04:08:35.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:35.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:35.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:08:35.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:08:35.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] boot 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: osdmap e25: 4 total, 4 up, 4 in 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: osdmap e26: 4 total, 4 up, 4 in 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:35.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:08:35.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:08:35.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: osd.3 [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] boot 2024-03-12T04:08:35.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: osdmap e25: 4 total, 4 up, 4 in 2024-03-12T04:08:35.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: osdmap e26: 4 total, 4 up, 4 in 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:35.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:36.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:36 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi042 2024-03-12T04:08:36.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:36 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi042 to 8546M 2024-03-12T04:08:36.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:36 smithi119 ceph-mon[36119]: osdmap e27: 4 total, 4 up, 4 in 2024-03-12T04:08:36.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:36 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi042 2024-03-12T04:08:36.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:36 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi042 to 8546M 2024-03-12T04:08:36.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:36 smithi136 ceph-mon[35571]: osdmap e27: 4 total, 4 up, 4 in 2024-03-12T04:08:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 -- 172.21.15.119:0/1511815642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c102730 msgr2=0x7fee2c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 --2- 172.21.15.119:0/1511815642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c102730 0x7fee2c104b20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009990 tx=0x7fee1c009ca0 comp rx=0 tx=0).stop 2024-03-12T04:08:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 -- 172.21.15.119:0/1511815642 shutdown_connections 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 --2- 172.21.15.119:0/1511815642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee2c107990 0x7fee2c109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 --2- 172.21.15.119:0/1511815642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c102730 0x7fee2c104b20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 --2- 172.21.15.119:0/1511815642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee2c105060 0x7fee2c107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 -- 172.21.15.119:0/1511815642 >> 172.21.15.119:0/1511815642 conn(0x7fee2c0fc160 msgr2=0x7fee2c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 -- 172.21.15.119:0/1511815642 shutdown_connections 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 -- 172.21.15.119:0/1511815642 wait complete. 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 Processor -- start 2024-03-12T04:08:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.477+0000 7fee3274c700 1 -- start start 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee3274c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee2c102730 0x7fee2c102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee3274c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee2c105060 0x7fee2c1006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee3274c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c107990 0x7fee2c100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee3274c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1013a0 con 0x7fee2c102730 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee3274c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c101510 con 0x7fee2c107990 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee3274c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1a1cd0 con 0x7fee2c105060 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c107990 0x7fee2c100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c107990 0x7fee2c100c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53590/0 (socket says 172.21.15.119:53590) 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 -- 172.21.15.119:0/4012018301 learned_addr learned my addr 172.21.15.119:0/4012018301 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee2b7fe700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee2c105060 0x7fee2c1006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee2bfff700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee2c102730 0x7fee2c102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 -- 172.21.15.119:0/4012018301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee2c105060 msgr2=0x7fee2c1006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee2c105060 0x7fee2c1006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 -- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee2c102730 msgr2=0x7fee2c102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee2c102730 0x7fee2c102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 -- 172.21.15.119:0/4012018301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c0095f0 con 0x7fee2c107990 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee30ce9700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c107990 0x7fee2c100c20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee2400b9e0 tx=0x7fee2400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee297fa700 1 -- 172.21.15.119:0/4012018301 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400d8e0 con 0x7fee2c107990 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee297fa700 1 -- 172.21.15.119:0/4012018301 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fee24009880 con 0x7fee2c107990 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c1a1fb0 con 0x7fee2c107990 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.478+0000 7fee297fa700 1 -- 172.21.15.119:0/4012018301 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee240143d0 con 0x7fee2c107990 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.479+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c1a24f0 con 0x7fee2c107990 2024-03-12T04:08:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.479+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c0fdce0 con 0x7fee2c107990 2024-03-12T04:08:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.481+0000 7fee297fa700 1 -- 172.21.15.119:0/4012018301 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fee240156b0 con 0x7fee2c107990 2024-03-12T04:08:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.481+0000 7fee297fa700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fee140a1b40 0x7fee140a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:08:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.481+0000 7fee297fa700 1 -- 172.21.15.119:0/4012018301 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fee24011070 con 0x7fee2c107990 2024-03-12T04:08:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.481+0000 7fee2bfff700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fee140a1b40 0x7fee140a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:08:36.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.482+0000 7fee2bfff700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fee140a1b40 0x7fee140a4000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00b5c0 tx=0x7fee1c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:08:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.485+0000 7fee297fa700 1 -- 172.21.15.119:0/4012018301 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee240962b0 con 0x7fee2c107990 2024-03-12T04:08:36.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:36 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi042 2024-03-12T04:08:36.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:36 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi042 to 8546M 2024-03-12T04:08:36.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:36 smithi042 ceph-mon[32260]: osdmap e27: 4 total, 4 up, 4 in 2024-03-12T04:08:36.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:36.685+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fee2c02d8e0 con 0x7fee140a1b40 2024-03-12T04:08:37.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:37 smithi119 ceph-mon[36119]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:08:37.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:08:37.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:08:37.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:37.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:37 smithi119 ceph-mon[36119]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:37.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:37 smithi119 ceph-mon[36119]: osdmap e28: 4 total, 4 up, 4 in 2024-03-12T04:08:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:37 smithi136 ceph-mon[35571]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:08:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:08:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:08:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:37 smithi136 ceph-mon[35571]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:37 smithi136 ceph-mon[35571]: osdmap e28: 4 total, 4 up, 4 in 2024-03-12T04:08:37.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:37 smithi042 ceph-mon[32260]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:08:37.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:08:37.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:08:37.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:37.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:37 smithi042 ceph-mon[32260]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:37.468 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:37 smithi042 ceph-mon[32260]: osdmap e28: 4 total, 4 up, 4 in 2024-03-12T04:08:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:38 smithi042 ceph-mon[32260]: osdmap e29: 4 total, 4 up, 4 in 2024-03-12T04:08:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:38 smithi042 ceph-mon[32260]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:39.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:38 smithi119 ceph-mon[36119]: osdmap e29: 4 total, 4 up, 4 in 2024-03-12T04:08:39.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:38 smithi119 ceph-mon[36119]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:39.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:38 smithi136 ceph-mon[35571]: osdmap e29: 4 total, 4 up, 4 in 2024-03-12T04:08:39.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:38 smithi136 ceph-mon[35571]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:39 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/2618767274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]: dispatch 2024-03-12T04:08:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:39 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]: dispatch 2024-03-12T04:08:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:39 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]': finished 2024-03-12T04:08:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:39 smithi042 ceph-mon[32260]: osdmap e30: 5 total, 4 up, 5 in 2024-03-12T04:08:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:39 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:39 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/3391132561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:08:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:39 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/2618767274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]: dispatch 2024-03-12T04:08:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:39 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]: dispatch 2024-03-12T04:08:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:39 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]': finished 2024-03-12T04:08:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:39 smithi119 ceph-mon[36119]: osdmap e30: 5 total, 4 up, 5 in 2024-03-12T04:08:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:39 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:39 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/3391132561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:08:40.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:39 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/2618767274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]: dispatch 2024-03-12T04:08:40.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:39 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]: dispatch 2024-03-12T04:08:40.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:39 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97a77fed-dfbe-45a5-8048-e1424bfee89f"}]': finished 2024-03-12T04:08:40.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:39 smithi136 ceph-mon[35571]: osdmap e30: 5 total, 4 up, 5 in 2024-03-12T04:08:40.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:39 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:40.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:39 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/3391132561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:08:41.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:40 smithi042 ceph-mon[32260]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-12T04:08:41.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:40 smithi119 ceph-mon[36119]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-12T04:08:41.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:40 smithi136 ceph-mon[35571]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-12T04:08:43.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:42 smithi119 ceph-mon[36119]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T04:08:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:42 smithi136 ceph-mon[35571]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T04:08:43.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:42 smithi042 ceph-mon[32260]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T04:08:45.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:44 smithi119 ceph-mon[36119]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-12T04:08:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:44 smithi136 ceph-mon[35571]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-12T04:08:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:44 smithi042 ceph-mon[32260]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-12T04:08:46.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T04:08:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:46 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T04:08:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:46 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T04:08:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:46 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:47.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:47 smithi119 ceph-mon[36119]: Deploying daemon osd.4 on smithi119 2024-03-12T04:08:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:47 smithi119 ceph-mon[36119]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-12T04:08:47.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:47 smithi136 ceph-mon[35571]: Deploying daemon osd.4 on smithi119 2024-03-12T04:08:47.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:47 smithi136 ceph-mon[35571]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-12T04:08:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:47 smithi042 ceph-mon[32260]: Deploying daemon osd.4 on smithi119 2024-03-12T04:08:48.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:47 smithi042 ceph-mon[32260]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-12T04:08:49.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:48 smithi042 ceph-mon[32260]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:08:49.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:48 smithi119 ceph-mon[36119]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:08:49.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:48 smithi136 ceph-mon[35571]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:08:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:50 smithi119 ceph-mon[36119]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-12T04:08:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:50 smithi136 ceph-mon[35571]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-12T04:08:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:50 smithi042 ceph-mon[32260]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-12T04:08:53.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:53.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:53.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:53.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:53 smithi119 ceph-mon[36119]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:53 smithi136 ceph-mon[35571]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:53 smithi042 ceph-mon[32260]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:55.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:54 smithi119 ceph-mon[36119]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:55.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:54 smithi136 ceph-mon[35571]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:55.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:54 smithi042 ceph-mon[32260]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:55.655 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 4 on host 'smithi119' 2024-03-12T04:08:55.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.652+0000 7fee297fa700 1 -- 172.21.15.119:0/4012018301 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fee2c02d8e0 con 0x7fee140a1b40 2024-03-12T04:08:55.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fee140a1b40 msgr2=0x7fee140a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fee140a1b40 0x7fee140a4000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00b5c0 tx=0x7fee1c005fd0 comp rx=0 tx=0).stop 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c107990 msgr2=0x7fee2c100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c107990 0x7fee2c100c20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee2400b9e0 tx=0x7fee2400bcf0 comp rx=0 tx=0).stop 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 shutdown_connections 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fee140a1b40 0x7fee140a4000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c107990 0x7fee2c100c20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee2c102730 0x7fee2c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 --2- 172.21.15.119:0/4012018301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee2c105060 0x7fee2c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 >> 172.21.15.119:0/4012018301 conn(0x7fee2c0fc160 msgr2=0x7fee2c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.654+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 shutdown_connections 2024-03-12T04:08:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:08:55.655+0000 7fee3274c700 1 -- 172.21.15.119:0/4012018301 wait complete. 2024-03-12T04:08:56.310 DEBUG:teuthology.orchestra.run.smithi119:osd.4> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.4.service 2024-03-12T04:08:56.312 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_3... 2024-03-12T04:08:56.312 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-12T04:08:56.563 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:08:56.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:56 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:56 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:56 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:56 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:56 smithi136 ceph-mon[35571]: from='osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T04:08:56.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:56 smithi136 ceph-mon[35571]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T04:08:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:56 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:56 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:56 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:56 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:56 smithi119 ceph-mon[36119]: from='osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T04:08:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:56 smithi119 ceph-mon[36119]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T04:08:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:56 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:56 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:56 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:56 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:56 smithi042 ceph-mon[32260]: from='osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T04:08:57.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:56 smithi042 ceph-mon[32260]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T04:08:57.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:57 smithi136 ceph-mon[35571]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T04:08:57.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:57 smithi136 ceph-mon[35571]: osdmap e31: 5 total, 4 up, 5 in 2024-03-12T04:08:57.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:57 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:57.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:57 smithi136 ceph-mon[35571]: from='osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:08:57.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:57 smithi136 ceph-mon[35571]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:08:57.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:57 smithi136 ceph-mon[35571]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:58.001 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-mon[36119]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T04:08:58.001 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-mon[36119]: osdmap e31: 5 total, 4 up, 5 in 2024-03-12T04:08:58.001 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:58.001 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-mon[36119]: from='osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:08:58.001 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-mon[36119]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:08:58.001 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-mon[36119]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:58.001 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:08:57.726+0000 7fd3e0891700 -1 osd.4 0 waiting for initial osdmap 2024-03-12T04:08:58.001 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:08:57 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:08:57.731+0000 7fd3dc684700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:08:58.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:57 smithi042 ceph-mon[32260]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T04:08:58.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:57 smithi042 ceph-mon[32260]: osdmap e31: 5 total, 4 up, 5 in 2024-03-12T04:08:58.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:57 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:58.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:57 smithi042 ceph-mon[32260]: from='osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:08:58.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:57 smithi042 ceph-mon[32260]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:08:58.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:57 smithi042 ceph-mon[32260]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:58.521 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-12T04:08:58.522 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-12T04:08:58.522 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-03-12T04:08:58.522 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-03-12T04:08:58.522 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00946371 s, 1.1 GB/s 2024-03-12T04:08:58.522 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 2024-03-12T04:08:58.522 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-03-12T04:08:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:58 smithi119 ceph-mon[36119]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:08:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:58 smithi119 ceph-mon[36119]: osdmap e32: 5 total, 4 up, 5 in 2024-03-12T04:08:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:59.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:58 smithi042 ceph-mon[32260]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:08:59.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:58 smithi042 ceph-mon[32260]: osdmap e32: 5 total, 4 up, 5 in 2024-03-12T04:08:59.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:59.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:59.136 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:58 smithi136 ceph-mon[35571]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:08:59.136 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:58 smithi136 ceph-mon[35571]: osdmap e32: 5 total, 4 up, 5 in 2024-03-12T04:08:59.136 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:59.136 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:59.162 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-03-12T04:08:59.399 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:08:59.789 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] boot 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: osdmap e33: 5 total, 5 up, 5 in 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:59.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:59.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:08:59.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:08:59.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:08:59.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:08:59.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:08:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] boot 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: osdmap e33: 5 total, 5 up, 5 in 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:00.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:08:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: osd.4 [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] boot 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: osdmap e33: 5 total, 5 up, 5 in 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:00.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:00.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:08:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:00.854 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:00 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi119 2024-03-12T04:09:00.854 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:00 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi119 to 8544M 2024-03-12T04:09:00.854 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:00 smithi136 ceph-mon[35571]: osdmap e34: 5 total, 5 up, 5 in 2024-03-12T04:09:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:00 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi119 2024-03-12T04:09:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:00 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi119 to 8544M 2024-03-12T04:09:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:00 smithi042 ceph-mon[32260]: osdmap e34: 5 total, 5 up, 5 in 2024-03-12T04:09:01.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 -- 172.21.15.136:0/1012770858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 msgr2=0x7fd6c41020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:01.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 --2- 172.21.15.136:0/1012770858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c41020e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009990 tx=0x7fd6c0009ca0 comp rx=0 tx=0).stop 2024-03-12T04:09:01.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 -- 172.21.15.136:0/1012770858 shutdown_connections 2024-03-12T04:09:01.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 --2- 172.21.15.136:0/1012770858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6c4103860 0x7fd6c410a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:01.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 --2- 172.21.15.136:0/1012770858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c41020e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 --2- 172.21.15.136:0/1012770858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4102ec0 0x7fd6c4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 -- 172.21.15.136:0/1012770858 >> 172.21.15.136:0/1012770858 conn(0x7fd6c40fd280 msgr2=0x7fd6c40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 -- 172.21.15.136:0/1012770858 shutdown_connections 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 -- 172.21.15.136:0/1012770858 wait complete. 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.079+0000 7fd6cb445700 1 Processor -- start 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6cb445700 1 -- start start 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6cb445700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c40759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6cb445700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4102ec0 0x7fd6c4074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6cb445700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6c4103860 0x7fd6c40745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6cb445700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c4074d40 con 0x7fd6c4103860 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6cb445700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c4074eb0 con 0x7fd6c4102ec0 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6cb445700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c41a1c60 con 0x7fd6c4101cc0 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c40759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c40759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:43570/0 (socket says 172.21.15.136:43570) 2024-03-12T04:09:01.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 -- 172.21.15.136:0/2436857956 learned_addr learned my addr 172.21.15.136:0/2436857956 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c89e0700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4102ec0 0x7fd6c4074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c99e2700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6c4103860 0x7fd6c40745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 -- 172.21.15.136:0/2436857956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4102ec0 msgr2=0x7fd6c4074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4102ec0 0x7fd6c4074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 -- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6c4103860 msgr2=0x7fd6c40745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6c4103860 0x7fd6c40745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 -- 172.21.15.136:0/2436857956 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00095f0 con 0x7fd6c4101cc0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.080+0000 7fd6c91e1700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c40759c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00036e0 tx=0x7fd6c0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.081+0000 7fd6ba7fc700 1 -- 172.21.15.136:0/2436857956 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c001d070 con 0x7fd6c4101cc0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.081+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c41a1e90 con 0x7fd6c4101cc0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.081+0000 7fd6ba7fc700 1 -- 172.21.15.136:0/2436857956 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6c00040d0 con 0x7fd6c4101cc0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.081+0000 7fd6ba7fc700 1 -- 172.21.15.136:0/2436857956 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0022de0 con 0x7fd6c4101cc0 2024-03-12T04:09:01.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.081+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c41a2240 con 0x7fd6c4101cc0 2024-03-12T04:09:01.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.082+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c41091a0 con 0x7fd6c4101cc0 2024-03-12T04:09:01.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.083+0000 7fd6ba7fc700 1 -- 172.21.15.136:0/2436857956 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd6c0022570 con 0x7fd6c4101cc0 2024-03-12T04:09:01.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.083+0000 7fd6ba7fc700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6b00a1ac0 0x7fd6b00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:01.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.083+0000 7fd6ba7fc700 1 -- 172.21.15.136:0/2436857956 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fd6c00cd640 con 0x7fd6c4101cc0 2024-03-12T04:09:01.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.084+0000 7fd6c89e0700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6b00a1ac0 0x7fd6b00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:01.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.084+0000 7fd6c89e0700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6b00a1ac0 0x7fd6b00a3f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4007c60 tx=0x7fd6b40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:09:01.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.088+0000 7fd6ba7fc700 1 -- 172.21.15.136:0/2436857956 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6c0097a70 con 0x7fd6c4101cc0 2024-03-12T04:09:01.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:00 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi119 2024-03-12T04:09:01.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:00 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi119 to 8544M 2024-03-12T04:09:01.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:00 smithi119 ceph-mon[36119]: osdmap e34: 5 total, 5 up, 5 in 2024-03-12T04:09:01.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:01.289+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6c402d870 con 0x7fd6b00a1ac0 2024-03-12T04:09:02.049 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:01 smithi136 ceph-mon[35571]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:02.049 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:01 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:02.049 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:01 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:02.049 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:01 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:02.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:01 smithi042 ceph-mon[32260]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:02.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:01 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:02.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:01 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:02.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:01 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:02.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:01 smithi119 ceph-mon[36119]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:02.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:01 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:02.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:01 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:02.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:01 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:02 smithi042 ceph-mon[32260]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:03.154 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:02 smithi136 ceph-mon[35571]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:03.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:02 smithi119 ceph-mon[36119]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:04.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:03 smithi042 ceph-mon[32260]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:04.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:03 smithi119 ceph-mon[36119]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:04.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:03 smithi136 ceph-mon[35571]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/3477852757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]: dispatch 2024-03-12T04:09:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:04 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]: dispatch 2024-03-12T04:09:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:04 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]': finished 2024-03-12T04:09:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:04 smithi136 ceph-mon[35571]: osdmap e35: 6 total, 5 up, 6 in 2024-03-12T04:09:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:04 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/3477852757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]: dispatch 2024-03-12T04:09:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:04 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]: dispatch 2024-03-12T04:09:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:04 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]': finished 2024-03-12T04:09:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:04 smithi042 ceph-mon[32260]: osdmap e35: 6 total, 5 up, 6 in 2024-03-12T04:09:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:04 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:05.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/3477852757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]: dispatch 2024-03-12T04:09:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:04 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]: dispatch 2024-03-12T04:09:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:04 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dc50d1-ecd2-4abc-af46-3ad102d339be"}]': finished 2024-03-12T04:09:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:04 smithi119 ceph-mon[36119]: osdmap e35: 6 total, 5 up, 6 in 2024-03-12T04:09:05.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:04 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:06.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:05 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/646319780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:06.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:05 smithi042 ceph-mon[32260]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:06.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:05 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/646319780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:06.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:05 smithi119 ceph-mon[36119]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:06.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:05 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/646319780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:06.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:05 smithi136 ceph-mon[35571]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:07.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:06 smithi119 ceph-mon[36119]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:07.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:06 smithi136 ceph-mon[35571]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:07.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:06 smithi042 ceph-mon[32260]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:09.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:08 smithi119 ceph-mon[36119]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:09.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:08 smithi136 ceph-mon[35571]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:09.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:08 smithi042 ceph-mon[32260]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:11.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:10 smithi119 ceph-mon[36119]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:11.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:10 smithi136 ceph-mon[35571]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:11.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:10 smithi042 ceph-mon[32260]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T04:09:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:11 smithi119 ceph-mon[36119]: Deploying daemon osd.5 on smithi136 2024-03-12T04:09:12.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T04:09:12.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:12.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:11 smithi136 ceph-mon[35571]: Deploying daemon osd.5 on smithi136 2024-03-12T04:09:12.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T04:09:12.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:12.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:11 smithi042 ceph-mon[32260]: Deploying daemon osd.5 on smithi136 2024-03-12T04:09:13.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:12 smithi119 ceph-mon[36119]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:12 smithi136 ceph-mon[35571]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:12 smithi042 ceph-mon[32260]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:15.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:14 smithi119 ceph-mon[36119]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:15.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:14 smithi136 ceph-mon[35571]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:15.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:14 smithi042 ceph-mon[32260]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:17.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:16 smithi119 ceph-mon[36119]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:16 smithi136 ceph-mon[35571]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:17.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:16 smithi042 ceph-mon[32260]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:18.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:18.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:18.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:18.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:18.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:18.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:18.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:18.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:18.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:19 smithi136 ceph-mon[35571]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:19 smithi042 ceph-mon[32260]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:19.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:19 smithi119 ceph-mon[36119]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:20.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.711+0000 7fd6ba7fc700 1 -- 172.21.15.136:0/2436857956 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd6c402d870 con 0x7fd6b00a1ac0 2024-03-12T04:09:20.713 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-03-12T04:09:20.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6b00a1ac0 msgr2=0x7fd6b00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6b00a1ac0 0x7fd6b00a3f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4007c60 tx=0x7fd6b40073d0 comp rx=0 tx=0).stop 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 msgr2=0x7fd6c40759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c40759c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00036e0 tx=0x7fd6c0005740 comp rx=0 tx=0).stop 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 shutdown_connections 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6b00a1ac0 0x7fd6b00a3f80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6c4103860 0x7fd6c40745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c4101cc0 0x7fd6c40759c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.714+0000 7fd6cb445700 1 --2- 172.21.15.136:0/2436857956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4102ec0 0x7fd6c4074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.715+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 >> 172.21.15.136:0/2436857956 conn(0x7fd6c40fd280 msgr2=0x7fd6c41062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.715+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 shutdown_connections 2024-03-12T04:09:20.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:09:20.715+0000 7fd6cb445700 1 -- 172.21.15.136:0/2436857956 wait complete. 2024-03-12T04:09:21.410 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.5.service 2024-03-12T04:09:21.413 INFO:tasks.cephadm:Deploying osd.6 on smithi042 with /dev/vg_nvme/lv_2... 2024-03-12T04:09:21.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-12T04:09:21.665 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:09:21.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:21 smithi042 ceph-mon[32260]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:21.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:21 smithi042 ceph-mon[32260]: from='osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T04:09:21.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:21 smithi042 ceph-mon[32260]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T04:09:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:21 smithi119 ceph-mon[36119]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:21 smithi119 ceph-mon[36119]: from='osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T04:09:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:21 smithi119 ceph-mon[36119]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T04:09:21.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:21.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:21 smithi136 ceph-mon[35571]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:21 smithi136 ceph-mon[35571]: from='osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T04:09:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:21 smithi136 ceph-mon[35571]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T04:09:22.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:22 smithi136 ceph-mon[35571]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T04:09:22.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:22 smithi136 ceph-mon[35571]: osdmap e36: 6 total, 5 up, 6 in 2024-03-12T04:09:22.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:22 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:22.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:22 smithi136 ceph-mon[35571]: from='osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:09:22.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:22 smithi136 ceph-mon[35571]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:09:23.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:22 smithi042 ceph-mon[32260]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T04:09:23.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:22 smithi042 ceph-mon[32260]: osdmap e36: 6 total, 5 up, 6 in 2024-03-12T04:09:23.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:23.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:22 smithi042 ceph-mon[32260]: from='osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:09:23.048 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:22 smithi042 ceph-mon[32260]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:09:23.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:22 smithi119 ceph-mon[36119]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T04:09:23.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:22 smithi119 ceph-mon[36119]: osdmap e36: 6 total, 5 up, 6 in 2024-03-12T04:09:23.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:22 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:23.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:22 smithi119 ceph-mon[36119]: from='osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:09:23.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:22 smithi119 ceph-mon[36119]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:09:23.707 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:09:23.241+0000 7fe9d3bee700 -1 osd.5 0 waiting for initial osdmap 2024-03-12T04:09:23.708 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:09:23.250+0000 7fe9cb9d9700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:09:23.770 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-12T04:09:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-12T04:09:23.771 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-03-12T04:09:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-03-12T04:09:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00894348 s, 1.2 GB/s 2024-03-12T04:09:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-03-12T04:09:24.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:23 smithi042 ceph-mon[32260]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:09:24.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:23 smithi042 ceph-mon[32260]: osdmap e37: 6 total, 5 up, 6 in 2024-03-12T04:09:24.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:23 smithi042 ceph-mon[32260]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:24.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:23 smithi042 ceph-mon[32260]: from='osd.5 ' entity='osd.5' 2024-03-12T04:09:24.083 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-mon[35571]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:09:24.083 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-mon[35571]: osdmap e37: 6 total, 5 up, 6 in 2024-03-12T04:09:24.084 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.084 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.084 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-mon[35571]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:24.084 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:23 smithi136 ceph-mon[35571]: from='osd.5 ' entity='osd.5' 2024-03-12T04:09:24.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:23 smithi119 ceph-mon[36119]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:09:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:23 smithi119 ceph-mon[36119]: osdmap e37: 6 total, 5 up, 6 in 2024-03-12T04:09:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:23 smithi119 ceph-mon[36119]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-12T04:09:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:23 smithi119 ceph-mon[36119]: from='osd.5 ' entity='osd.5' 2024-03-12T04:09:24.669 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2024-03-12T04:09:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] boot 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: osdmap e38: 6 total, 6 up, 6 in 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:24.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:24.994 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:24.994 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:24.994 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:24.994 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:24.994 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:09:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:09:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] boot 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: osdmap e38: 6 total, 6 up, 6 in 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:25.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:25.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:25.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:25.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:25.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:09:25.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: osd.5 [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] boot 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: osdmap e38: 6 total, 6 up, 6 in 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:25.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:25 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi136 2024-03-12T04:09:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:25 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi136 to 8544M 2024-03-12T04:09:26.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:25 smithi042 ceph-mon[32260]: osdmap e39: 6 total, 6 up, 6 in 2024-03-12T04:09:26.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:25 smithi042 ceph-mon[32260]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:26.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:25 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi136 2024-03-12T04:09:26.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:25 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi136 to 8544M 2024-03-12T04:09:26.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:25 smithi119 ceph-mon[36119]: osdmap e39: 6 total, 6 up, 6 in 2024-03-12T04:09:26.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:25 smithi119 ceph-mon[36119]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:25 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi136 2024-03-12T04:09:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:25 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi136 to 8544M 2024-03-12T04:09:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:25 smithi136 ceph-mon[35571]: osdmap e39: 6 total, 6 up, 6 in 2024-03-12T04:09:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:25 smithi136 ceph-mon[35571]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:26.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 -- 172.21.15.42:0/1922185310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 msgr2=0x7f179810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:26.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 --2- 172.21.15.42:0/1922185310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f179810a0f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1790009fc0 tx=0x7f179000b330 comp rx=0 tx=0).stop 2024-03-12T04:09:26.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 -- 172.21.15.42:0/1922185310 shutdown_connections 2024-03-12T04:09:26.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 --2- 172.21.15.42:0/1922185310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f179810a0f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 --2- 172.21.15.42:0/1922185310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1798101ca0 0x7f17981020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 --2- 172.21.15.42:0/1922185310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1798102ea0 0x7f1798103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 -- 172.21.15.42:0/1922185310 >> 172.21.15.42:0/1922185310 conn(0x7f17980fd280 msgr2=0x7f17980ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:09:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 -- 172.21.15.42:0/1922185310 shutdown_connections 2024-03-12T04:09:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.749+0000 7f179d6c8700 1 -- 172.21.15.42:0/1922185310 wait complete. 2024-03-12T04:09:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 Processor -- start 2024-03-12T04:09:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 -- start start 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1798101ca0 0x7f1798075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1798102ea0 0x7f1798074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f17980745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1798074d20 con 0x7f1798103840 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1798074e90 con 0x7f1798101ca0 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f179d6c8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981a1cd0 con 0x7f1798102ea0 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f17977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f17980745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f17977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f17980745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58158/0 (socket says 172.21.15.42:58158) 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f17977fe700 1 -- 172.21.15.42:0/2027681480 learned_addr learned my addr 172.21.15.42:0/2027681480 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f17967fc700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1798102ea0 0x7f1798074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.750+0000 7f1796ffd700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1798101ca0 0x7f1798075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f17977fe700 1 -- 172.21.15.42:0/2027681480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1798102ea0 msgr2=0x7f1798074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f17977fe700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1798102ea0 0x7f1798074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f17977fe700 1 -- 172.21.15.42:0/2027681480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1798101ca0 msgr2=0x7f1798075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f17977fe700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1798101ca0 0x7f1798075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f17977fe700 1 -- 172.21.15.42:0/2027681480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0095f0 con 0x7f1798103840 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f17967fc700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1798102ea0 0x7f1798074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f17977fe700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f17980745a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f179000d010 tx=0x7f179000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f177ffff700 1 -- 172.21.15.42:0/2027681480 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000e040 con 0x7f1798103840 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f177ffff700 1 -- 172.21.15.42:0/2027681480 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1790009350 con 0x7f1798103840 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f177ffff700 1 -- 172.21.15.42:0/2027681480 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179001b6b0 con 0x7f1798103840 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1790009c40 con 0x7f1798103840 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.751+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17981a21d0 con 0x7f1798103840 2024-03-12T04:09:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.752+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17981090e0 con 0x7f1798103840 2024-03-12T04:09:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.753+0000 7f177ffff700 1 -- 172.21.15.42:0/2027681480 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f179001b810 con 0x7f1798103840 2024-03-12T04:09:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.754+0000 7f177ffff700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17800a1ac0 0x7f17800a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.754+0000 7f177ffff700 1 -- 172.21.15.42:0/2027681480 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f17900ce160 con 0x7f1798103840 2024-03-12T04:09:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.754+0000 7f1796ffd700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17800a1ac0 0x7f17800a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.754+0000 7f1796ffd700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17800a1ac0 0x7f17800a3f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1798102d00 tx=0x7f1788009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:09:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.757+0000 7f177ffff700 1 -- 172.21.15.42:0/2027681480 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1790098200 con 0x7f1798103840 2024-03-12T04:09:26.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:26.971+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f179802d870 con 0x7f17800a1ac0 2024-03-12T04:09:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:26 smithi042 ceph-mon[32260]: osdmap e40: 6 total, 6 up, 6 in 2024-03-12T04:09:27.115 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:26 smithi119 ceph-mon[36119]: osdmap e40: 6 total, 6 up, 6 in 2024-03-12T04:09:27.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:26 smithi136 ceph-mon[35571]: osdmap e40: 6 total, 6 up, 6 in 2024-03-12T04:09:27.870 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:27 smithi042 ceph-mon[32260]: osdmap e41: 6 total, 6 up, 6 in 2024-03-12T04:09:27.870 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:27 smithi042 ceph-mon[32260]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:27.870 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:27.870 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:27.871 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:28.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:27 smithi119 ceph-mon[36119]: osdmap e41: 6 total, 6 up, 6 in 2024-03-12T04:09:28.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:27 smithi119 ceph-mon[36119]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:28.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:28.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:28.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:27 smithi136 ceph-mon[35571]: osdmap e41: 6 total, 6 up, 6 in 2024-03-12T04:09:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:27 smithi136 ceph-mon[35571]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:29.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:28 smithi042 ceph-mon[32260]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:29.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:28 smithi042 ceph-mon[32260]: osdmap e42: 6 total, 6 up, 6 in 2024-03-12T04:09:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:28 smithi119 ceph-mon[36119]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:28 smithi119 ceph-mon[36119]: osdmap e42: 6 total, 6 up, 6 in 2024-03-12T04:09:29.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:28 smithi136 ceph-mon[35571]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:29.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:28 smithi136 ceph-mon[35571]: osdmap e42: 6 total, 6 up, 6 in 2024-03-12T04:09:30.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:29 smithi119 ceph-mon[36119]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:30.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:29 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3740140819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99e7959d-3257-4c03-b302-c98052f009e0"}]: dispatch 2024-03-12T04:09:30.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:29 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3740140819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99e7959d-3257-4c03-b302-c98052f009e0"}]': finished 2024-03-12T04:09:30.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:29 smithi119 ceph-mon[36119]: osdmap e43: 7 total, 6 up, 7 in 2024-03-12T04:09:30.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:29 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:29 smithi136 ceph-mon[35571]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:29 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3740140819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99e7959d-3257-4c03-b302-c98052f009e0"}]: dispatch 2024-03-12T04:09:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:29 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3740140819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99e7959d-3257-4c03-b302-c98052f009e0"}]': finished 2024-03-12T04:09:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:29 smithi136 ceph-mon[35571]: osdmap e43: 7 total, 6 up, 7 in 2024-03-12T04:09:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:29 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:29 smithi042 ceph-mon[32260]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:29 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3740140819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99e7959d-3257-4c03-b302-c98052f009e0"}]: dispatch 2024-03-12T04:09:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:29 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3740140819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99e7959d-3257-4c03-b302-c98052f009e0"}]': finished 2024-03-12T04:09:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:29 smithi042 ceph-mon[32260]: osdmap e43: 7 total, 6 up, 7 in 2024-03-12T04:09:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:29 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:31.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:30 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/436847363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:30 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/436847363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:30 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/436847363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:32.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:31 smithi119 ceph-mon[36119]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-12T04:09:32.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:31 smithi136 ceph-mon[35571]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-12T04:09:32.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:31 smithi042 ceph-mon[32260]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-12T04:09:33.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:32 smithi119 ceph-mon[36119]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-12T04:09:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:32 smithi136 ceph-mon[35571]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-12T04:09:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:32 smithi042 ceph-mon[32260]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-12T04:09:35.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:34 smithi119 ceph-mon[36119]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T04:09:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:34 smithi136 ceph-mon[35571]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T04:09:35.268 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:34 smithi042 ceph-mon[32260]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T04:09:37.158 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:36 smithi042 ceph-mon[32260]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-12T04:09:37.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:36 smithi119 ceph-mon[36119]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-12T04:09:37.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:36 smithi136 ceph-mon[35571]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-12T04:09:38.065 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T04:09:38.065 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:38.065 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:37 smithi042 ceph-mon[32260]: Deploying daemon osd.6 on smithi042 2024-03-12T04:09:38.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T04:09:38.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:38.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:37 smithi119 ceph-mon[36119]: Deploying daemon osd.6 on smithi042 2024-03-12T04:09:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T04:09:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:37 smithi136 ceph-mon[35571]: Deploying daemon osd.6 on smithi042 2024-03-12T04:09:39.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:38 smithi119 ceph-mon[36119]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:09:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:38 smithi136 ceph-mon[35571]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:09:39.279 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:38 smithi042 ceph-mon[32260]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:09:41.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:40 smithi042 ceph-mon[32260]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-12T04:09:41.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:40 smithi119 ceph-mon[36119]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-12T04:09:41.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:40 smithi136 ceph-mon[35571]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-12T04:09:43.043 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:42 smithi042 ceph-mon[32260]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:43.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:42 smithi119 ceph-mon[36119]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:42 smithi136 ceph-mon[35571]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:44.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:45.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:45 smithi042 ceph-mon[32260]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:45.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:45 smithi119 ceph-mon[36119]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:45.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:45 smithi136 ceph-mon[35571]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:47.184 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 6 on host 'smithi042' 2024-03-12T04:09:47.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.181+0000 7f177ffff700 1 -- 172.21.15.42:0/2027681480 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f179802d870 con 0x7f17800a1ac0 2024-03-12T04:09:47.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17800a1ac0 msgr2=0x7f17800a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:47.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17800a1ac0 0x7f17800a3f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1798102d00 tx=0x7f1788009260 comp rx=0 tx=0).stop 2024-03-12T04:09:47.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 msgr2=0x7f17980745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:47.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f17980745a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f179000d010 tx=0x7f179000bc00 comp rx=0 tx=0).stop 2024-03-12T04:09:47.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 shutdown_connections 2024-03-12T04:09:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17800a1ac0 0x7f17800a3f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1798103840 0x7f17980745a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1798101ca0 0x7f1798075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 --2- 172.21.15.42:0/2027681480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1798102ea0 0x7f1798074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.183+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 >> 172.21.15.42:0/2027681480 conn(0x7f17980fd280 msgr2=0x7f1798106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:09:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.184+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 shutdown_connections 2024-03-12T04:09:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:09:47.184+0000 7f179d6c8700 1 -- 172.21.15.42:0/2027681480 wait complete. 2024-03-12T04:09:47.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T04:09:47.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T04:09:47.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:47.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T04:09:47.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:09:48.006 DEBUG:teuthology.orchestra.run.smithi042:osd.6> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.6.service 2024-03-12T04:09:48.008 INFO:tasks.cephadm:Deploying osd.7 on smithi119 with /dev/vg_nvme/lv_2... 2024-03-12T04:09:48.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-12T04:09:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:09:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:49 smithi042 ceph-mon[32260]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T04:09:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:49 smithi042 ceph-mon[32260]: osdmap e44: 7 total, 6 up, 7 in 2024-03-12T04:09:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:49 smithi042 ceph-mon[32260]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:09:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:49 smithi042 ceph-mon[32260]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:49.417 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:49 smithi119 ceph-mon[36119]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T04:09:49.418 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:49 smithi119 ceph-mon[36119]: osdmap e44: 7 total, 6 up, 7 in 2024-03-12T04:09:49.418 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:49 smithi119 ceph-mon[36119]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:09:49.418 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:49.418 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:49 smithi119 ceph-mon[36119]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:49 smithi136 ceph-mon[35571]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T04:09:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:49 smithi136 ceph-mon[35571]: osdmap e44: 7 total, 6 up, 7 in 2024-03-12T04:09:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:49 smithi136 ceph-mon[35571]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:09:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:49 smithi136 ceph-mon[35571]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:49.792 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:09:49 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:09:49.615+0000 7f3632c12700 -1 osd.6 0 waiting for initial osdmap 2024-03-12T04:09:49.792 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:09:49 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:09:49.625+0000 7f362a1fc700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:09:50.301 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-12T04:09:50.302 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-12T04:09:50.302 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-12T04:09:50.302 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-12T04:09:50.302 INFO:teuthology.orchestra.run.smithi119.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129185 s, 812 MB/s 2024-03-12T04:09:50.302 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-12T04:09:50.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:50 smithi119 ceph-mon[36119]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:09:50.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:50 smithi119 ceph-mon[36119]: osdmap e45: 7 total, 6 up, 7 in 2024-03-12T04:09:50.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:50 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:50.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:50 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:50.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:50 smithi119 ceph-mon[36119]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' 2024-03-12T04:09:50.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:50 smithi136 ceph-mon[35571]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:09:50.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:50 smithi136 ceph-mon[35571]: osdmap e45: 7 total, 6 up, 7 in 2024-03-12T04:09:50.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:50 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:50.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:50 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:50.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:50 smithi136 ceph-mon[35571]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' 2024-03-12T04:09:50.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:50 smithi042 ceph-mon[32260]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:09:50.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:50 smithi042 ceph-mon[32260]: osdmap e45: 7 total, 6 up, 7 in 2024-03-12T04:09:50.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:50 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:50.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:50 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:50.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:50 smithi042 ceph-mon[32260]: from='osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893]' entity='osd.6' 2024-03-12T04:09:50.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2024-03-12T04:09:51.191 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:09:51.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:51 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:09:51.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:51 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:09:51.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:51.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:51 smithi119 ceph-mon[36119]: osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] boot 2024-03-12T04:09:51.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:51 smithi119 ceph-mon[36119]: osdmap e46: 7 total, 7 up, 7 in 2024-03-12T04:09:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:51 smithi119 ceph-mon[36119]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:51 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:09:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:51 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:09:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:51 smithi136 ceph-mon[35571]: osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] boot 2024-03-12T04:09:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:51 smithi136 ceph-mon[35571]: osdmap e46: 7 total, 7 up, 7 in 2024-03-12T04:09:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:51.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:51 smithi136 ceph-mon[35571]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:51.549 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:51 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:09:51.549 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:51 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:09:51.549 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:51.549 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:51 smithi042 ceph-mon[32260]: osd.6 [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] boot 2024-03-12T04:09:51.549 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:51 smithi042 ceph-mon[32260]: osdmap e46: 7 total, 7 up, 7 in 2024-03-12T04:09:51.549 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:09:51.549 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:51 smithi042 ceph-mon[32260]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-12T04:09:52.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi042 2024-03-12T04:09:52.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi042 to 5697M 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:52 smithi042 ceph-mon[32260]: osdmap e47: 7 total, 7 up, 7 in 2024-03-12T04:09:52.630 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi042 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi042 to 5697M 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:52 smithi119 ceph-mon[36119]: osdmap e47: 7 total, 7 up, 7 in 2024-03-12T04:09:52.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi042 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi042 to 5697M 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:09:52.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:52 smithi136 ceph-mon[35571]: osdmap e47: 7 total, 7 up, 7 in 2024-03-12T04:09:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 -- 172.21.15.119:0/2052217583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0101da0 msgr2=0x7f99c0102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 --2- 172.21.15.119:0/2052217583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0101da0 0x7f99c0102200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f99b40099c0 tx=0x7f99b4009cd0 comp rx=0 tx=0).stop 2024-03-12T04:09:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 -- 172.21.15.119:0/2052217583 shutdown_connections 2024-03-12T04:09:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 --2- 172.21.15.119:0/2052217583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99c0102740 0x7f99c0108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 --2- 172.21.15.119:0/2052217583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99c0100ba0 0x7f99c0100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 --2- 172.21.15.119:0/2052217583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0101da0 0x7f99c0102200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 -- 172.21.15.119:0/2052217583 >> 172.21.15.119:0/2052217583 conn(0x7f99c00fc140 msgr2=0x7f99c00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:09:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 -- 172.21.15.119:0/2052217583 shutdown_connections 2024-03-12T04:09:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.871+0000 7f99c638d700 1 -- 172.21.15.119:0/2052217583 wait complete. 2024-03-12T04:09:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 Processor -- start 2024-03-12T04:09:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 -- start start 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99c0100ba0 0x7f99c0197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99c0101da0 0x7f99c0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0102740 0x7f99c019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019f740 con 0x7f99c0100ba0 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019f880 con 0x7f99c0102740 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c638d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019fa10 con 0x7f99c0101da0 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.872+0000 7f99c492a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0102740 0x7f99c019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0102740 0x7f99c019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44590/0 (socket says 172.21.15.119:44590) 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 -- 172.21.15.119:0/1514767978 learned_addr learned my addr 172.21.15.119:0/1514767978 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99bf7fe700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99c0101da0 0x7f99c0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99bffff700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99c0100ba0 0x7f99c0197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 -- 172.21.15.119:0/1514767978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99c0101da0 msgr2=0x7f99c0198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99c0101da0 0x7f99c0198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 -- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99c0100ba0 msgr2=0x7f99c0197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99c0100ba0 0x7f99c0197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 -- 172.21.15.119:0/1514767978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40095f0 con 0x7f99c0102740 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99bffff700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99c0100ba0 0x7f99c0197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c492a700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0102740 0x7f99c019f170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007ee0 tx=0x7f99b800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99bd7fa700 1 -- 172.21.15.119:0/1514767978 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8017070 con 0x7f99c0102740 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c019fcf0 con 0x7f99c0102740 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99bd7fa700 1 -- 172.21.15.119:0/1514767978 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99b800f040 con 0x7f99c0102740 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99bd7fa700 1 -- 172.21.15.119:0/1514767978 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8005730 con 0x7f99c0102740 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.873+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c01a01a0 con 0x7f99c0102740 2024-03-12T04:09:52.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.874+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0068930 con 0x7f99c0102740 2024-03-12T04:09:52.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.875+0000 7f99bd7fa700 1 -- 172.21.15.119:0/1514767978 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f99b8005890 con 0x7f99c0102740 2024-03-12T04:09:52.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.876+0000 7f99bd7fa700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99a80a1ac0 0x7f99a80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:09:52.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.876+0000 7f99bd7fa700 1 -- 172.21.15.119:0/1514767978 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f99b80cc5b0 con 0x7f99c0102740 2024-03-12T04:09:52.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.876+0000 7f99bffff700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99a80a1ac0 0x7f99a80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:09:52.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.877+0000 7f99bffff700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99a80a1ac0 0x7f99a80a3f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005fd0 tx=0x7f99b0005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:09:52.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:52.879+0000 7f99bd7fa700 1 -- 172.21.15.119:0/1514767978 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99b80964d0 con 0x7f99c0102740 2024-03-12T04:09:53.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:09:53.081+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c006a060 con 0x7f99a80a1ac0 2024-03-12T04:09:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:53 smithi042 ceph-mon[32260]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:53.693 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:53 smithi119 ceph-mon[36119]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:53.693 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:53.693 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:53.693 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:53 smithi136 ceph-mon[35571]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:09:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:09:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:09:54.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:54 smithi042 ceph-mon[32260]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:54.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:54 smithi119 ceph-mon[36119]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:54 smithi136 ceph-mon[35571]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:09:55.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:55 smithi042 ceph-mon[32260]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:55.608 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:55 smithi136 ceph-mon[35571]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:55.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:55 smithi119 ceph-mon[36119]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:56.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:56 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/1697969706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]: dispatch 2024-03-12T04:09:56.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:56 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]: dispatch 2024-03-12T04:09:56.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:56 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]': finished 2024-03-12T04:09:56.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:56 smithi119 ceph-mon[36119]: osdmap e48: 8 total, 7 up, 8 in 2024-03-12T04:09:56.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:56 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:09:56.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:56 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/1697969706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]: dispatch 2024-03-12T04:09:56.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:56 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]: dispatch 2024-03-12T04:09:56.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:56 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]': finished 2024-03-12T04:09:56.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:56 smithi042 ceph-mon[32260]: osdmap e48: 8 total, 7 up, 8 in 2024-03-12T04:09:56.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:56 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:09:56.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:56 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/1697969706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]: dispatch 2024-03-12T04:09:56.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:56 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]: dispatch 2024-03-12T04:09:56.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:56 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c430bbfa-80fb-485a-a186-ee581a9a8196"}]': finished 2024-03-12T04:09:56.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:56 smithi136 ceph-mon[35571]: osdmap e48: 8 total, 7 up, 8 in 2024-03-12T04:09:56.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:56 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:09:57.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:57 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/455594057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:57.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:57 smithi042 ceph-mon[32260]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:57.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:57 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/455594057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:57.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:57 smithi119 ceph-mon[36119]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:57.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:57 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/455594057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:09:57.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:57 smithi136 ceph-mon[35571]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:59.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:09:58 smithi119 ceph-mon[36119]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:59.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:09:58 smithi136 ceph-mon[35571]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:09:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:09:58 smithi042 ceph-mon[32260]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:00.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:00 smithi042 ceph-mon[32260]: overall HEALTH_OK 2024-03-12T04:10:00.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:00 smithi119 ceph-mon[36119]: overall HEALTH_OK 2024-03-12T04:10:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:00 smithi136 ceph-mon[35571]: overall HEALTH_OK 2024-03-12T04:10:01.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:01 smithi042 ceph-mon[32260]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:01.428 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:01 smithi119 ceph-mon[36119]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:01 smithi136 ceph-mon[35571]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:03.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:02 smithi119 ceph-mon[36119]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:03.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:02 smithi136 ceph-mon[35571]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:03.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:02 smithi042 ceph-mon[32260]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:04.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:03 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T04:10:04.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:03 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:04.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:03 smithi119 ceph-mon[36119]: Deploying daemon osd.7 on smithi119 2024-03-12T04:10:04.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:03 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T04:10:04.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:03 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:04.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:03 smithi136 ceph-mon[35571]: Deploying daemon osd.7 on smithi119 2024-03-12T04:10:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:03 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T04:10:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:03 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:03 smithi042 ceph-mon[32260]: Deploying daemon osd.7 on smithi119 2024-03-12T04:10:05.171 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:04 smithi119 ceph-mon[36119]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:04 smithi136 ceph-mon[35571]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:05.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:04 smithi042 ceph-mon[32260]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:07.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:06 smithi119 ceph-mon[36119]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:07.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:06 smithi136 ceph-mon[35571]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:07.209 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:06 smithi042 ceph-mon[32260]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:10.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:09 smithi042 ceph-mon[32260]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:09 smithi119 ceph-mon[36119]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:09 smithi136 ceph-mon[35571]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:11.134 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:10 smithi119 ceph-mon[36119]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:11.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:10 smithi136 ceph-mon[35571]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:11.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:10 smithi042 ceph-mon[32260]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:12.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.700+0000 7f99bd7fa700 1 -- 172.21.15.119:0/1514767978 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f99c006a060 con 0x7f99a80a1ac0 2024-03-12T04:10:12.703 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 7 on host 'smithi119' 2024-03-12T04:10:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.702+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99a80a1ac0 msgr2=0x7f99a80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.702+0000 7f99c638d700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99a80a1ac0 0x7f99a80a3f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005fd0 tx=0x7f99b0005f00 comp rx=0 tx=0).stop 2024-03-12T04:10:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.702+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0102740 msgr2=0x7f99c019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.702+0000 7f99c638d700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0102740 0x7f99c019f170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007ee0 tx=0x7f99b800d3b0 comp rx=0 tx=0).stop 2024-03-12T04:10:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 shutdown_connections 2024-03-12T04:10:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99a80a1ac0 0x7f99a80a3f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c0102740 0x7f99c019f170 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:12.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99c0100ba0 0x7f99c0197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:12.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 --2- 172.21.15.119:0/1514767978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99c0101da0 0x7f99c0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:12.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 >> 172.21.15.119:0/1514767978 conn(0x7f99c00fc140 msgr2=0x7f99c0069950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:10:12.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 shutdown_connections 2024-03-12T04:10:12.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:10:12.703+0000 7f99c638d700 1 -- 172.21.15.119:0/1514767978 wait complete. 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T04:10:13.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:12 smithi119 ceph-mon[36119]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:13.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T04:10:13.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:12 smithi136 ceph-mon[35571]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T04:10:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:12 smithi042 ceph-mon[32260]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:13.324 DEBUG:teuthology.orchestra.run.smithi119:osd.7> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.7.service 2024-03-12T04:10:13.326 INFO:tasks.cephadm:Deploying osd.8 on smithi136 with /dev/vg_nvme/lv_2... 2024-03-12T04:10:13.326 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-12T04:10:13.575 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:10:14.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:14 smithi119 ceph-mon[36119]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T04:10:14.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:14 smithi119 ceph-mon[36119]: osdmap e49: 8 total, 7 up, 8 in 2024-03-12T04:10:14.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:14 smithi119 ceph-mon[36119]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:10:14.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:14 smithi136 ceph-mon[35571]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T04:10:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:14 smithi136 ceph-mon[35571]: osdmap e49: 8 total, 7 up, 8 in 2024-03-12T04:10:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:14 smithi136 ceph-mon[35571]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:10:14.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:14 smithi042 ceph-mon[32260]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T04:10:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:14 smithi042 ceph-mon[32260]: osdmap e49: 8 total, 7 up, 8 in 2024-03-12T04:10:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:14 smithi042 ceph-mon[32260]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:10:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.446 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:10:15.099+0000 7fed5cb1e700 -1 osd.7 0 waiting for initial osdmap 2024-03-12T04:10:15.446 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:10:15.112+0000 7fed54909700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:10:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-12T04:10:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-12T04:10:15.741 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-03-12T04:10:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-03-12T04:10:15.741 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00904546 s, 1.2 GB/s 2024-03-12T04:10:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-03-12T04:10:15.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:10:15.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: osdmap e50: 8 total, 7 up, 8 in 2024-03-12T04:10:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: osdmap e51: 8 total, 7 up, 8 in 2024-03-12T04:10:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:15 smithi119 ceph-mon[36119]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' 2024-03-12T04:10:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:10:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: osdmap e50: 8 total, 7 up, 8 in 2024-03-12T04:10:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: osdmap e51: 8 total, 7 up, 8 in 2024-03-12T04:10:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:15.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:15 smithi136 ceph-mon[35571]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' 2024-03-12T04:10:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:10:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: osdmap e50: 8 total, 7 up, 8 in 2024-03-12T04:10:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-12T04:10:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: osdmap e51: 8 total, 7 up, 8 in 2024-03-12T04:10:16.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:16.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:15 smithi042 ceph-mon[32260]: from='osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628]' entity='osd.7' 2024-03-12T04:10:16.465 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-03-12T04:10:16.706 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:10:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:10:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:10:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] boot 2024-03-12T04:10:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: osdmap e52: 8 total, 8 up, 8 in 2024-03-12T04:10:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:16 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] boot 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: osdmap e52: 8 total, 8 up, 8 in 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:16 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:10:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:10:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: osd.7 [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] boot 2024-03-12T04:10:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: osdmap e52: 8 total, 8 up, 8 in 2024-03-12T04:10:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:16 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:17.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:17 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi119 2024-03-12T04:10:17.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:17 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi119 to 5696M 2024-03-12T04:10:17.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:17 smithi136 ceph-mon[35571]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:17.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:17 smithi136 ceph-mon[35571]: osdmap e53: 8 total, 8 up, 8 in 2024-03-12T04:10:17.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:17 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi119 2024-03-12T04:10:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:17 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi119 to 5696M 2024-03-12T04:10:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:17 smithi119 ceph-mon[36119]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:17 smithi119 ceph-mon[36119]: osdmap e53: 8 total, 8 up, 8 in 2024-03-12T04:10:18.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:17 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi119 2024-03-12T04:10:18.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:17 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi119 to 5696M 2024-03-12T04:10:18.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:17 smithi042 ceph-mon[32260]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:18.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:17 smithi042 ceph-mon[32260]: osdmap e53: 8 total, 8 up, 8 in 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 -- 172.21.15.136:0/1216741476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 msgr2=0x7f1c38103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/1216741476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38103300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009990 tx=0x7f1c24009ca0 comp rx=0 tx=0).stop 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 -- 172.21.15.136:0/1216741476 shutdown_connections 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/1216741476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38103840 0x7f1c3810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/1216741476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c38101ca0 0x7f1c381020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/1216741476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38103300 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 -- 172.21.15.136:0/1216741476 >> 172.21.15.136:0/1216741476 conn(0x7f1c380fd280 msgr2=0x7f1c380ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 -- 172.21.15.136:0/1216741476 shutdown_connections 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.317+0000 7f1c3eed8700 1 -- 172.21.15.136:0/1216741476 wait complete. 2024-03-12T04:10:18.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 Processor -- start 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 -- start start 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c38101ca0 0x7f1c38107a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38107f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38103840 0x7f1c381045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38104d50 con 0x7f1c38103840 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38104ec0 con 0x7f1c38101ca0 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3eed8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38105080 con 0x7f1c38102ea0 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c37fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c37fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:52078/0 (socket says 172.21.15.136:52078) 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c37fff700 1 -- 172.21.15.136:0/3627919685 learned_addr learned my addr 172.21.15.136:0/3627919685 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3d475700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38103840 0x7f1c381045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.318+0000 7f1c3cc74700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c38101ca0 0x7f1c38107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c37fff700 1 -- 172.21.15.136:0/3627919685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c38101ca0 msgr2=0x7f1c38107a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:18.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c37fff700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c38101ca0 0x7f1c38107a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c37fff700 1 -- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38103840 msgr2=0x7f1c381045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c37fff700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38103840 0x7f1c381045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c37fff700 1 -- 172.21.15.136:0/3627919685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c240095f0 con 0x7f1c38102ea0 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c3cc74700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c38101ca0 0x7f1c38107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c37fff700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38107f70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c240098e0 tx=0x7f1c240057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c35ffb700 1 -- 172.21.15.136:0/3627919685 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2401d070 con 0x7f1c38102ea0 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c35ffb700 1 -- 172.21.15.136:0/3627919685 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c240041d0 con 0x7f1c38102ea0 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c381a6180 con 0x7f1c38102ea0 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c35ffb700 1 -- 172.21.15.136:0/3627919685 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400f460 con 0x7f1c38102ea0 2024-03-12T04:10:18.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.319+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c381a6590 con 0x7f1c38102ea0 2024-03-12T04:10:18.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.320+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c38068930 con 0x7f1c38102ea0 2024-03-12T04:10:18.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.321+0000 7f1c35ffb700 1 -- 172.21.15.136:0/3627919685 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1c24022470 con 0x7f1c38102ea0 2024-03-12T04:10:18.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.322+0000 7f1c35ffb700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1c280a1b40 0x7f1c280a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:18.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.322+0000 7f1c35ffb700 1 -- 172.21.15.136:0/3627919685 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f1c240ce3c0 con 0x7f1c38102ea0 2024-03-12T04:10:18.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.322+0000 7f1c3cc74700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1c280a1b40 0x7f1c280a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:18.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.323+0000 7f1c3cc74700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1c280a1b40 0x7f1c280a4000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005fd0 tx=0x7f1c2c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:10:18.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.325+0000 7f1c35ffb700 1 -- 172.21.15.136:0/3627919685 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c24098230 con 0x7f1c38102ea0 2024-03-12T04:10:18.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:18.529+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c380755c0 con 0x7f1c280a1b40 2024-03-12T04:10:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:18 smithi136 ceph-mon[35571]: osdmap e54: 8 total, 8 up, 8 in 2024-03-12T04:10:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:18 smithi136 ceph-mon[35571]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:10:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:10:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:10:19.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:18 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:19.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:18 smithi136 ceph-mon[35571]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:18 smithi042 ceph-mon[32260]: osdmap e54: 8 total, 8 up, 8 in 2024-03-12T04:10:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:18 smithi042 ceph-mon[32260]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:10:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:10:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:10:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:18 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:18 smithi042 ceph-mon[32260]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:18 smithi119 ceph-mon[36119]: osdmap e54: 8 total, 8 up, 8 in 2024-03-12T04:10:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:18 smithi119 ceph-mon[36119]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:10:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:10:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:10:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:18 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:18 smithi119 ceph-mon[36119]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:20.148 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:20 smithi136 ceph-mon[35571]: osdmap e55: 8 total, 8 up, 8 in 2024-03-12T04:10:20.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:20 smithi042 ceph-mon[32260]: osdmap e55: 8 total, 8 up, 8 in 2024-03-12T04:10:20.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:20 smithi119 ceph-mon[36119]: osdmap e55: 8 total, 8 up, 8 in 2024-03-12T04:10:21.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:21 smithi042 ceph-mon[32260]: osdmap e56: 8 total, 8 up, 8 in 2024-03-12T04:10:21.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:21 smithi042 ceph-mon[32260]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:21.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:21 smithi119 ceph-mon[36119]: osdmap e56: 8 total, 8 up, 8 in 2024-03-12T04:10:21.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:21 smithi119 ceph-mon[36119]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:21.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:21 smithi136 ceph-mon[35571]: osdmap e56: 8 total, 8 up, 8 in 2024-03-12T04:10:21.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:21 smithi136 ceph-mon[35571]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/1680727933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]: dispatch 2024-03-12T04:10:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:22 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]: dispatch 2024-03-12T04:10:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:22 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]': finished 2024-03-12T04:10:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:22 smithi042 ceph-mon[32260]: osdmap e57: 9 total, 8 up, 9 in 2024-03-12T04:10:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:22 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/3893761860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:10:22.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:22 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/1680727933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]: dispatch 2024-03-12T04:10:22.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:22 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]: dispatch 2024-03-12T04:10:22.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:22 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]': finished 2024-03-12T04:10:22.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:22 smithi119 ceph-mon[36119]: osdmap e57: 9 total, 8 up, 9 in 2024-03-12T04:10:22.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:22 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:22.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:22 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/3893761860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:10:22.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:22 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/1680727933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]: dispatch 2024-03-12T04:10:22.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:22 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]: dispatch 2024-03-12T04:10:22.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:22 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b72976-753c-4cee-9be5-867ebd0479ef"}]': finished 2024-03-12T04:10:22.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:22 smithi136 ceph-mon[35571]: osdmap e57: 9 total, 8 up, 9 in 2024-03-12T04:10:22.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:22 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:22.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:22 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/3893761860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:10:23.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:23 smithi042 ceph-mon[32260]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:23.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:23 smithi119 ceph-mon[36119]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:23 smithi136 ceph-mon[35571]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:25.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:24 smithi119 ceph-mon[36119]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:25.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:24 smithi136 ceph-mon[35571]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:25.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:24 smithi042 ceph-mon[32260]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:27.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:26 smithi119 ceph-mon[36119]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:27.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:26 smithi136 ceph-mon[35571]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:27.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:26 smithi042 ceph-mon[32260]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:29.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:28 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-12T04:10:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:28 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:29.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:28 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-12T04:10:29.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:28 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:29.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:28 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-12T04:10:29.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:28 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:30.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:29 smithi119 ceph-mon[36119]: Deploying daemon osd.8 on smithi136 2024-03-12T04:10:30.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:29 smithi119 ceph-mon[36119]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:29 smithi136 ceph-mon[35571]: Deploying daemon osd.8 on smithi136 2024-03-12T04:10:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:29 smithi136 ceph-mon[35571]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:29 smithi042 ceph-mon[32260]: Deploying daemon osd.8 on smithi136 2024-03-12T04:10:30.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:29 smithi042 ceph-mon[32260]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:31.045 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:30 smithi136 ceph-mon[35571]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:31.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:30 smithi119 ceph-mon[36119]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:31.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:30 smithi042 ceph-mon[32260]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:32 smithi136 ceph-mon[35571]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:32 smithi042 ceph-mon[32260]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:33.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:32 smithi119 ceph-mon[36119]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:35.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:35.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:35.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:35.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:35 smithi119 ceph-mon[36119]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:35.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:35.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:35.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:35.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:35 smithi136 ceph-mon[35571]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:35 smithi042 ceph-mon[32260]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:37.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:36 smithi136 ceph-mon[35571]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:37.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:36 smithi042 ceph-mon[32260]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:37.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:36 smithi119 ceph-mon[36119]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:38.598 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 8 on host 'smithi136' 2024-03-12T04:10:38.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.596+0000 7f1c35ffb700 1 -- 172.21.15.136:0/3627919685 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1c380755c0 con 0x7f1c280a1b40 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1c280a1b40 msgr2=0x7f1c280a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1c280a1b40 0x7f1c280a4000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005fd0 tx=0x7f1c2c005e40 comp rx=0 tx=0).stop 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 msgr2=0x7f1c38107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38107f70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c240098e0 tx=0x7f1c240057f0 comp rx=0 tx=0).stop 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 shutdown_connections 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1c280a1b40 0x7f1c280a4000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38103840 0x7f1c381045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c38101ca0 0x7f1c38107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.598+0000 7f1c3eed8700 1 --2- 172.21.15.136:0/3627919685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c38102ea0 0x7f1c38107f70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.599+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 >> 172.21.15.136:0/3627919685 conn(0x7f1c380fd280 msgr2=0x7f1c38073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.599+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 shutdown_connections 2024-03-12T04:10:38.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:10:38.599+0000 7f1c3eed8700 1 -- 172.21.15.136:0/3627919685 wait complete. 2024-03-12T04:10:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T04:10:39.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T04:10:39.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:39.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:39.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T04:10:39.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T04:10:39.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:39.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:10:39.359 DEBUG:teuthology.orchestra.run.smithi136:osd.8> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.8.service 2024-03-12T04:10:39.362 INFO:tasks.cephadm:Deploying osd.9 on smithi042 with /dev/vg_nvme/lv_1... 2024-03-12T04:10:39.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-12T04:10:39.616 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:10:39.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:39 smithi136 ceph-mon[35571]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-12T04:10:39.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:39 smithi136 ceph-mon[35571]: osdmap e58: 9 total, 8 up, 9 in 2024-03-12T04:10:39.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:39 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:39.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:39 smithi136 ceph-mon[35571]: from='osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:10:39.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:39 smithi136 ceph-mon[35571]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:10:39.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:39 smithi136 ceph-mon[35571]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:40.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:39 smithi042 ceph-mon[32260]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-12T04:10:40.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:39 smithi042 ceph-mon[32260]: osdmap e58: 9 total, 8 up, 9 in 2024-03-12T04:10:40.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:39 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:40.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:39 smithi042 ceph-mon[32260]: from='osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:10:40.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:39 smithi042 ceph-mon[32260]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:10:40.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:39 smithi042 ceph-mon[32260]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:40.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:39 smithi119 ceph-mon[36119]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-12T04:10:40.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:39 smithi119 ceph-mon[36119]: osdmap e58: 9 total, 8 up, 9 in 2024-03-12T04:10:40.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:39 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:40.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:39 smithi119 ceph-mon[36119]: from='osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:10:40.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:39 smithi119 ceph-mon[36119]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:10:40.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:39 smithi119 ceph-mon[36119]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:40.572 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:10:40.261+0000 7f93d299d700 -1 osd.8 0 waiting for initial osdmap 2024-03-12T04:10:40.572 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:10:40.273+0000 7f93ca788700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:10:41.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-mon[35571]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:10:41.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-mon[35571]: osdmap e59: 9 total, 8 up, 9 in 2024-03-12T04:10:41.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-mon[35571]: osdmap e60: 9 total, 8 up, 9 in 2024-03-12T04:10:41.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:40 smithi136 ceph-mon[35571]: from='osd.8 ' entity='osd.8' 2024-03-12T04:10:41.181 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:40 smithi042 ceph-mon[32260]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:10:41.181 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:40 smithi042 ceph-mon[32260]: osdmap e59: 9 total, 8 up, 9 in 2024-03-12T04:10:41.181 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:40 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.181 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:40 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.182 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:40 smithi042 ceph-mon[32260]: osdmap e60: 9 total, 8 up, 9 in 2024-03-12T04:10:41.182 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:40 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.182 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:40 smithi042 ceph-mon[32260]: from='osd.8 ' entity='osd.8' 2024-03-12T04:10:41.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:40 smithi119 ceph-mon[36119]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:10:41.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:40 smithi119 ceph-mon[36119]: osdmap e59: 9 total, 8 up, 9 in 2024-03-12T04:10:41.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:40 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:40 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:40 smithi119 ceph-mon[36119]: osdmap e60: 9 total, 8 up, 9 in 2024-03-12T04:10:41.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:40 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:40 smithi119 ceph-mon[36119]: from='osd.8 ' entity='osd.8' 2024-03-12T04:10:41.904 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-12T04:10:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-12T04:10:41.905 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-03-12T04:10:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-03-12T04:10:41.905 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00841749 s, 1.2 GB/s 2024-03-12T04:10:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-03-12T04:10:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:41 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:10:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:41 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:10:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:41 smithi136 ceph-mon[35571]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:41 smithi136 ceph-mon[35571]: osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] boot 2024-03-12T04:10:41.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:41 smithi136 ceph-mon[35571]: osdmap e61: 9 total, 9 up, 9 in 2024-03-12T04:10:41.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:42.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:41 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:10:42.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:41 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:10:42.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:42.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:41 smithi119 ceph-mon[36119]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:42.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:41 smithi119 ceph-mon[36119]: osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] boot 2024-03-12T04:10:42.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:41 smithi119 ceph-mon[36119]: osdmap e61: 9 total, 9 up, 9 in 2024-03-12T04:10:42.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:42.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:41 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:10:42.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:41 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:10:42.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:42.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:41 smithi042 ceph-mon[32260]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-12T04:10:42.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:41 smithi042 ceph-mon[32260]: osd.8 [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] boot 2024-03-12T04:10:42.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:41 smithi042 ceph-mon[32260]: osdmap e61: 9 total, 9 up, 9 in 2024-03-12T04:10:42.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:10:42.580 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_1 2024-03-12T04:10:42.821 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:10:43.275 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: osdmap e62: 9 total, 9 up, 9 in 2024-03-12T04:10:43.275 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi136 2024-03-12T04:10:43.275 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi136 to 5696M 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.276 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:43 smithi042 ceph-mon[32260]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: osdmap e62: 9 total, 9 up, 9 in 2024-03-12T04:10:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi136 2024-03-12T04:10:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi136 to 5696M 2024-03-12T04:10:43.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:43.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:43.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:43 smithi119 ceph-mon[36119]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:43.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: osdmap e62: 9 total, 9 up, 9 in 2024-03-12T04:10:43.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi136 2024-03-12T04:10:43.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi136 to 5696M 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:10:43.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:43 smithi136 ceph-mon[35571]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:44.288 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:44 smithi042 ceph-mon[32260]: osdmap e63: 9 total, 9 up, 9 in 2024-03-12T04:10:44.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:44 smithi119 ceph-mon[36119]: osdmap e63: 9 total, 9 up, 9 in 2024-03-12T04:10:44.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:44 smithi136 ceph-mon[35571]: osdmap e63: 9 total, 9 up, 9 in 2024-03-12T04:10:44.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.495+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/3857649214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 msgr2=0x7f4aa0103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:44.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.495+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/3857649214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0103300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4a940099c0 tx=0x7f4a94009cd0 comp rx=0 tx=0).stop 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.495+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/3857649214 shutdown_connections 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/3857649214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa0103840 0x7f4aa010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/3857649214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aa0101ca0 0x7f4aa01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/3857649214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0103300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/3857649214 >> 172.21.15.42:0/3857649214 conn(0x7f4aa00fd280 msgr2=0x7f4aa00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/3857649214 shutdown_connections 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/3857649214 wait complete. 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 Processor -- start 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.496+0000 7f4aa5ceb700 1 -- start start 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4aa5ceb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aa0101ca0 0x7f4aa0075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4aa5ceb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4aa5ceb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa0103840 0x7f4aa00745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4aa5ceb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0074d20 con 0x7f4aa0102ea0 2024-03-12T04:10:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4aa5ceb700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0074e90 con 0x7f4aa0101ca0 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4aa5ceb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01a1be0 con 0x7f4aa0103840 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aa0101ca0 0x7f4aa0075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60382/0 (socket says 172.21.15.42:60382) 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 -- 172.21.15.42:0/461803707 learned_addr learned my addr 172.21.15.42:0/461803707 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9ffff700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa0103840 0x7f4aa00745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 -- 172.21.15.42:0/461803707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa0103840 msgr2=0x7f4aa00745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa0103840 0x7f4aa00745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 -- 172.21.15.42:0/461803707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aa0101ca0 msgr2=0x7f4aa0075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aa0101ca0 0x7f4aa0075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 -- 172.21.15.42:0/461803707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a940095f0 con 0x7f4aa0102ea0 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9effd700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0074060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a94000c00 tx=0x7f4a94009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:10:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4a9cff9700 1 -- 172.21.15.42:0/461803707 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401d070 con 0x7f4aa0102ea0 2024-03-12T04:10:44.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.497+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa01a1e60 con 0x7f4aa0102ea0 2024-03-12T04:10:44.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.498+0000 7f4a9cff9700 1 -- 172.21.15.42:0/461803707 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a94003f50 con 0x7f4aa0102ea0 2024-03-12T04:10:44.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.498+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa01a2280 con 0x7f4aa0102ea0 2024-03-12T04:10:44.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.498+0000 7f4a9cff9700 1 -- 172.21.15.42:0/461803707 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94022be0 con 0x7f4aa0102ea0 2024-03-12T04:10:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.498+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c005360 con 0x7f4aa0102ea0 2024-03-12T04:10:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.500+0000 7f4a9cff9700 1 -- 172.21.15.42:0/461803707 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4a94022470 con 0x7f4aa0102ea0 2024-03-12T04:10:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.501+0000 7f4a9cff9700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a880a1ac0 0x7f4a880a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:10:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.501+0000 7f4a9f7fe700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a880a1ac0 0x7f4a880a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:10:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.501+0000 7f4a9cff9700 1 -- 172.21.15.42:0/461803707 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f4a940ce450 con 0x7f4aa0102ea0 2024-03-12T04:10:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.501+0000 7f4a9f7fe700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a880a1ac0 0x7f4a880a3f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0102d00 tx=0x7f4a9000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:10:44.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.504+0000 7f4a9cff9700 1 -- 172.21.15.42:0/461803707 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a94098000 con 0x7f4aa0102ea0 2024-03-12T04:10:44.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:10:44.720+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a8c000bf0 con 0x7f4a880a1ac0 2024-03-12T04:10:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:45 smithi042 ceph-mon[32260]: osdmap e64: 9 total, 9 up, 9 in 2024-03-12T04:10:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:45 smithi042 ceph-mon[32260]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:10:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:10:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:10:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:45 smithi042 ceph-mon[32260]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:45.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:45 smithi119 ceph-mon[36119]: osdmap e64: 9 total, 9 up, 9 in 2024-03-12T04:10:45.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:45 smithi119 ceph-mon[36119]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:10:45.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:10:45.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:10:45.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:45.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:45 smithi119 ceph-mon[36119]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:45.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:45 smithi136 ceph-mon[35571]: osdmap e64: 9 total, 9 up, 9 in 2024-03-12T04:10:45.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:45 smithi136 ceph-mon[35571]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:10:45.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:10:45.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:10:45.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:45.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:45 smithi136 ceph-mon[35571]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:46 smithi042 ceph-mon[32260]: osdmap e65: 9 total, 9 up, 9 in 2024-03-12T04:10:46.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:46 smithi119 ceph-mon[36119]: osdmap e65: 9 total, 9 up, 9 in 2024-03-12T04:10:46.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:46 smithi136 ceph-mon[35571]: osdmap e65: 9 total, 9 up, 9 in 2024-03-12T04:10:47.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:47 smithi042 ceph-mon[32260]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-12T04:10:47.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:47 smithi119 ceph-mon[36119]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-12T04:10:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:47 smithi136 ceph-mon[35571]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-12T04:10:48.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:48 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1683151226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d19ae72-048c-4680-bf7d-2b2f2bf353f7"}]: dispatch 2024-03-12T04:10:48.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:48 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1683151226' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d19ae72-048c-4680-bf7d-2b2f2bf353f7"}]': finished 2024-03-12T04:10:48.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:48 smithi042 ceph-mon[32260]: osdmap e66: 10 total, 9 up, 10 in 2024-03-12T04:10:48.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:48 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:10:48.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:48 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1683151226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d19ae72-048c-4680-bf7d-2b2f2bf353f7"}]: dispatch 2024-03-12T04:10:48.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:48 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1683151226' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d19ae72-048c-4680-bf7d-2b2f2bf353f7"}]': finished 2024-03-12T04:10:48.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:48 smithi119 ceph-mon[36119]: osdmap e66: 10 total, 9 up, 10 in 2024-03-12T04:10:48.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:48 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:10:48.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:48 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1683151226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d19ae72-048c-4680-bf7d-2b2f2bf353f7"}]: dispatch 2024-03-12T04:10:48.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:48 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1683151226' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d19ae72-048c-4680-bf7d-2b2f2bf353f7"}]': finished 2024-03-12T04:10:48.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:48 smithi136 ceph-mon[35571]: osdmap e66: 10 total, 9 up, 10 in 2024-03-12T04:10:48.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:48 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:10:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:49 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2039320175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:10:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:49 smithi042 ceph-mon[32260]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-12T04:10:49.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:49 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2039320175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:10:49.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:49 smithi119 ceph-mon[36119]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-12T04:10:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:49 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2039320175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:10:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:49 smithi136 ceph-mon[35571]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-12T04:10:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:50 smithi136 ceph-mon[35571]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-12T04:10:51.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:50 smithi042 ceph-mon[32260]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-12T04:10:51.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:50 smithi119 ceph-mon[36119]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-12T04:10:53.064 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:52 smithi042 ceph-mon[32260]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T04:10:53.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:52 smithi136 ceph-mon[35571]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T04:10:53.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:52 smithi119 ceph-mon[36119]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T04:10:55.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:54 smithi136 ceph-mon[35571]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:10:55.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:54 smithi042 ceph-mon[32260]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:10:55.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:54 smithi119 ceph-mon[36119]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-12T04:10:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-12T04:10:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:55 smithi136 ceph-mon[35571]: Deploying daemon osd.9 on smithi042 2024-03-12T04:10:56.304 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-12T04:10:56.304 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:56.304 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:55 smithi042 ceph-mon[32260]: Deploying daemon osd.9 on smithi042 2024-03-12T04:10:56.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-12T04:10:56.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:10:56.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:55 smithi119 ceph-mon[36119]: Deploying daemon osd.9 on smithi042 2024-03-12T04:10:57.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:56 smithi136 ceph-mon[35571]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:56 smithi042 ceph-mon[32260]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:57.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:56 smithi119 ceph-mon[36119]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:59.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:10:58 smithi136 ceph-mon[35571]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:10:58 smithi042 ceph-mon[32260]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:10:59.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:10:58 smithi119 ceph-mon[36119]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:01.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:00 smithi136 ceph-mon[35571]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:01.212 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:00 smithi042 ceph-mon[32260]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:01.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:00 smithi119 ceph-mon[36119]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:02.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:02.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:02.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:02.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:02.963 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:02.963 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:02.963 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:03.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:03 smithi119 ceph-mon[36119]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:03.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:03 smithi136 ceph-mon[35571]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:03.958 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:03 smithi042 ceph-mon[32260]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:05.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.195+0000 7f4a9cff9700 1 -- 172.21.15.42:0/461803707 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a8c000bf0 con 0x7f4a880a1ac0 2024-03-12T04:11:05.198 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:04 smithi042 ceph-mon[32260]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:05.198 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 9 on host 'smithi042' 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a880a1ac0 msgr2=0x7f4a880a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a880a1ac0 0x7f4a880a3f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0102d00 tx=0x7f4a9000b410 comp rx=0 tx=0).stop 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 msgr2=0x7f4aa0074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0074060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a94000c00 tx=0x7f4a94009580 comp rx=0 tx=0).stop 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 shutdown_connections 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a880a1ac0 0x7f4a880a3f80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa0103840 0x7f4aa00745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aa0101ca0 0x7f4aa0075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 --2- 172.21.15.42:0/461803707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aa0102ea0 0x7f4aa0074060 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:05.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.197+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 >> 172.21.15.42:0/461803707 conn(0x7f4aa00fd280 msgr2=0x7f4aa01062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:11:05.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.198+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 shutdown_connections 2024-03-12T04:11:05.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:11:05.198+0000 7f4aa5ceb700 1 -- 172.21.15.42:0/461803707 wait complete. 2024-03-12T04:11:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:04 smithi136 ceph-mon[35571]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:05.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:04 smithi119 ceph-mon[36119]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:05.855 DEBUG:teuthology.orchestra.run.smithi042:osd.9> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.9.service 2024-03-12T04:11:05.857 INFO:tasks.cephadm:Deploying osd.10 on smithi119 with /dev/vg_nvme/lv_1... 2024-03-12T04:11:05.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-12T04:11:06.103 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:11:06.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:06 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:06 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:06 smithi119 ceph-mon[36119]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-12T04:11:06.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:06 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:06 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:06 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:06 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:06 smithi136 ceph-mon[35571]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-12T04:11:06.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:06 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:06 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.531 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:06 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.531 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:06 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.531 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:06 smithi042 ceph-mon[32260]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-12T04:11:06.531 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:06 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:06.532 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:06 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:07.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:07 smithi119 ceph-mon[36119]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-12T04:11:07.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:07 smithi119 ceph-mon[36119]: osdmap e67: 10 total, 9 up, 10 in 2024-03-12T04:11:07.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:07 smithi119 ceph-mon[36119]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:11:07.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:07 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:07.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:07 smithi119 ceph-mon[36119]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:07 smithi136 ceph-mon[35571]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-12T04:11:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:07 smithi136 ceph-mon[35571]: osdmap e67: 10 total, 9 up, 10 in 2024-03-12T04:11:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:07 smithi136 ceph-mon[35571]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:11:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:07 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:07 smithi136 ceph-mon[35571]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:07 smithi042 ceph-mon[32260]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-12T04:11:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:07 smithi042 ceph-mon[32260]: osdmap e67: 10 total, 9 up, 10 in 2024-03-12T04:11:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:07 smithi042 ceph-mon[32260]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-03-12T04:11:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:07 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:07 smithi042 ceph-mon[32260]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:07.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:11:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:11:07.194+0000 7fef103a6700 -1 osd.9 0 waiting for initial osdmap 2024-03-12T04:11:07.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:11:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:11:07.206+0000 7fef0c199700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:11:08.129 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-12T04:11:08.130 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-12T04:11:08.130 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-12T04:11:08.130 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-12T04:11:08.130 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120306 s, 872 MB/s 2024-03-12T04:11:08.130 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-12T04:11:08.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:08 smithi042 ceph-mon[32260]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:11:08.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:08 smithi042 ceph-mon[32260]: osdmap e68: 10 total, 9 up, 10 in 2024-03-12T04:11:08.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:08 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:08.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:08 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:08.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:08 smithi136 ceph-mon[35571]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:11:08.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:08 smithi136 ceph-mon[35571]: osdmap e68: 10 total, 9 up, 10 in 2024-03-12T04:11:08.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:08 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:08.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:08 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:08.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:08 smithi119 ceph-mon[36119]: from='osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-03-12T04:11:08.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:08 smithi119 ceph-mon[36119]: osdmap e68: 10 total, 9 up, 10 in 2024-03-12T04:11:08.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:08 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:08.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:08 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:08.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_1 2024-03-12T04:11:09.027 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:11:09.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:09 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:11:09.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:09 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:11:09.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:09 smithi119 ceph-mon[36119]: osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] boot 2024-03-12T04:11:09.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:09 smithi119 ceph-mon[36119]: osdmap e69: 10 total, 10 up, 10 in 2024-03-12T04:11:09.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:09 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:09.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:09 smithi119 ceph-mon[36119]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:09 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:11:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:09 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:11:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:09 smithi136 ceph-mon[35571]: osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] boot 2024-03-12T04:11:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:09 smithi136 ceph-mon[35571]: osdmap e69: 10 total, 10 up, 10 in 2024-03-12T04:11:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:09 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:09 smithi136 ceph-mon[35571]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:09.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:09 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:11:09.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:09 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:11:09.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:09 smithi042 ceph-mon[32260]: osd.9 [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] boot 2024-03-12T04:11:09.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:09 smithi042 ceph-mon[32260]: osdmap e69: 10 total, 10 up, 10 in 2024-03-12T04:11:09.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:09 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:11:09.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:09 smithi042 ceph-mon[32260]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-12T04:11:10.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: osdmap e70: 10 total, 10 up, 10 in 2024-03-12T04:11:10.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi042 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi042 to 4273M 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:10.459 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:10.459 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.459 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:10 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:10.502 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: osdmap e70: 10 total, 10 up, 10 in 2024-03-12T04:11:10.502 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi042 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi042 to 4273M 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.503 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:10 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: osdmap e70: 10 total, 10 up, 10 in 2024-03-12T04:11:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi042 2024-03-12T04:11:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi042 to 4273M 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:10.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:10 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 -- 172.21.15.119:0/3764681451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98100ba0 msgr2=0x7f7e98100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 --2- 172.21.15.119:0/3764681451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98100ba0 0x7f7e98100fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7e880099c0 tx=0x7f7e88009cd0 comp rx=0 tx=0).stop 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 -- 172.21.15.119:0/3764681451 shutdown_connections 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 --2- 172.21.15.119:0/3764681451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98102740 0x7f7e98108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 --2- 172.21.15.119:0/3764681451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98100ba0 0x7f7e98100fc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 --2- 172.21.15.119:0/3764681451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98101da0 0x7f7e98102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 -- 172.21.15.119:0/3764681451 >> 172.21.15.119:0/3764681451 conn(0x7f7e980fc140 msgr2=0x7f7e980fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 -- 172.21.15.119:0/3764681451 shutdown_connections 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 -- 172.21.15.119:0/3764681451 wait complete. 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.896+0000 7f7e9e439700 1 Processor -- start 2024-03-12T04:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9e439700 1 -- start start 2024-03-12T04:11:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9e439700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98100ba0 0x7f7e9806a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9e439700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98101da0 0x7f7e9819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9e439700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98102740 0x7f7e9819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9e439700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9819f720 con 0x7f7e98100ba0 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9e439700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9819f890 con 0x7f7e98102740 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9e439700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9819fa50 con 0x7f7e98101da0 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e97fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98100ba0 0x7f7e9806a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9c9d6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98102740 0x7f7e9819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e97fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98100ba0 0x7f7e9806a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.119:48356/0 (socket says 172.21.15.119:48356) 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e97fff700 1 -- 172.21.15.119:0/1025862029 learned_addr learned my addr 172.21.15.119:0/1025862029 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9c9d6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98102740 0x7f7e9819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54052/0 (socket says 172.21.15.119:54052) 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e977fe700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98101da0 0x7f7e9819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9c9d6700 1 -- 172.21.15.119:0/1025862029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98101da0 msgr2=0x7f7e9819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9c9d6700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98101da0 0x7f7e9819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9c9d6700 1 -- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98100ba0 msgr2=0x7f7e9806a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9c9d6700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98100ba0 0x7f7e9806a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e9c9d6700 1 -- 172.21.15.119:0/1025862029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880095f0 con 0x7f7e98102740 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.897+0000 7f7e97fff700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98100ba0 0x7f7e9806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.898+0000 7f7e9c9d6700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98102740 0x7f7e9819ef80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7e90007f10 tx=0x7f7e9000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.898+0000 7f7e957fa700 1 -- 172.21.15.119:0/1025862029 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000ddd0 con 0x7f7e98102740 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.898+0000 7f7e957fa700 1 -- 172.21.15.119:0/1025862029 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e9000f040 con 0x7f7e98102740 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.898+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9819fd30 con 0x7f7e98102740 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.898+0000 7f7e957fa700 1 -- 172.21.15.119:0/1025862029 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90005a70 con 0x7f7e98102740 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.898+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e981a01d0 con 0x7f7e98102740 2024-03-12T04:11:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.899+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e98108020 con 0x7f7e98102740 2024-03-12T04:11:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.900+0000 7f7e957fa700 1 -- 172.21.15.119:0/1025862029 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7e90004ad0 con 0x7f7e98102740 2024-03-12T04:11:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.900+0000 7f7e957fa700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7e800a1b40 0x7f7e800a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.900+0000 7f7e957fa700 1 -- 172.21.15.119:0/1025862029 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f7e900ccbd0 con 0x7f7e98102740 2024-03-12T04:11:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.901+0000 7f7e97fff700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7e800a1b40 0x7f7e800a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.901+0000 7f7e97fff700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7e800a1b40 0x7f7e800a4000 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e880037f0 tx=0x7f7e88009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:11:10.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:10.904+0000 7f7e957fa700 1 -- 172.21.15.119:0/1025862029 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e900965e0 con 0x7f7e98102740 2024-03-12T04:11:11.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:11.104+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e9802da20 con 0x7f7e800a1b40 2024-03-12T04:11:11.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:11.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:11.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:11.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:11 smithi119 ceph-mon[36119]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:11.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:11:11.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:11:11.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:11 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:11.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:11.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:11.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:11.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:11 smithi136 ceph-mon[35571]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:11.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:11:11.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:11:11.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:11 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:11 smithi042 ceph-mon[32260]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:11:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:11:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:11 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:12.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:12 smithi119 ceph-mon[36119]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:11:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:12 smithi136 ceph-mon[35571]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:11:12.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:12 smithi042 ceph-mon[32260]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:11:13.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:13 smithi119 ceph-mon[36119]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:13.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:13 smithi136 ceph-mon[35571]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:13.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:13 smithi042 ceph-mon[32260]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:14 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/354067185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]: dispatch 2024-03-12T04:11:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:14 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]: dispatch 2024-03-12T04:11:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:14 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]': finished 2024-03-12T04:11:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:14 smithi119 ceph-mon[36119]: osdmap e71: 11 total, 10 up, 11 in 2024-03-12T04:11:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:14 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/354067185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]: dispatch 2024-03-12T04:11:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:14 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]: dispatch 2024-03-12T04:11:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:14 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]': finished 2024-03-12T04:11:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:14 smithi136 ceph-mon[35571]: osdmap e71: 11 total, 10 up, 11 in 2024-03-12T04:11:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:14 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/354067185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]: dispatch 2024-03-12T04:11:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:14 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]: dispatch 2024-03-12T04:11:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:14 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4"}]': finished 2024-03-12T04:11:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:14 smithi042 ceph-mon[32260]: osdmap e71: 11 total, 10 up, 11 in 2024-03-12T04:11:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:15.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:15 smithi119 ceph-mon[36119]: from='client.? 172.21.15.119:0/3633524283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:11:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:15 smithi119 ceph-mon[36119]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:15.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:15 smithi136 ceph-mon[35571]: from='client.? 172.21.15.119:0/3633524283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:11:15.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:15 smithi136 ceph-mon[35571]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:15.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:15 smithi042 ceph-mon[32260]: from='client.? 172.21.15.119:0/3633524283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:11:15.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:15 smithi042 ceph-mon[32260]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:16 smithi136 ceph-mon[35571]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:17.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:16 smithi042 ceph-mon[32260]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:17.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:16 smithi119 ceph-mon[36119]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:19.191 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:18 smithi119 ceph-mon[36119]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:19.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:18 smithi136 ceph-mon[35571]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:19.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:18 smithi042 ceph-mon[32260]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:21.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:20 smithi119 ceph-mon[36119]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:21.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:20 smithi136 ceph-mon[35571]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:21.285 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:20 smithi042 ceph-mon[32260]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:22.134 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-12T04:11:22.134 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:22.134 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:21 smithi119 ceph-mon[36119]: Deploying daemon osd.10 on smithi119 2024-03-12T04:11:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-12T04:11:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:22.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:21 smithi136 ceph-mon[35571]: Deploying daemon osd.10 on smithi119 2024-03-12T04:11:22.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-12T04:11:22.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:22.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:21 smithi042 ceph-mon[32260]: Deploying daemon osd.10 on smithi119 2024-03-12T04:11:23.116 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:22 smithi119 ceph-mon[36119]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:23.315 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:22 smithi042 ceph-mon[32260]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:22 smithi136 ceph-mon[35571]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:24 smithi119 ceph-mon[36119]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:25.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:24 smithi136 ceph-mon[35571]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:25.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:24 smithi042 ceph-mon[32260]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:28.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:28.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:28.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:28.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:27 smithi042 ceph-mon[32260]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:28.193 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:28.194 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:28.194 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:28.194 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:27 smithi119 ceph-mon[36119]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:28.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:27 smithi136 ceph-mon[35571]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:29.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:28 smithi119 ceph-mon[36119]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:29.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:28 smithi136 ceph-mon[35571]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:29.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:28 smithi042 ceph-mon[32260]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:30.621 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 10 on host 'smithi119' 2024-03-12T04:11:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.619+0000 7f7e957fa700 1 -- 172.21.15.119:0/1025862029 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7e9802da20 con 0x7f7e800a1b40 2024-03-12T04:11:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7e800a1b40 msgr2=0x7f7e800a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7e800a1b40 0x7f7e800a4000 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e880037f0 tx=0x7f7e88009f10 comp rx=0 tx=0).stop 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98102740 msgr2=0x7f7e9819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98102740 0x7f7e9819ef80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7e90007f10 tx=0x7f7e9000d3b0 comp rx=0 tx=0).stop 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 shutdown_connections 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7e800a1b40 0x7f7e800a4000 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e98102740 0x7f7e9819ef80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e98100ba0 0x7f7e9806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 --2- 172.21.15.119:0/1025862029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98101da0 0x7f7e9819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 >> 172.21.15.119:0/1025862029 conn(0x7f7e980fc140 msgr2=0x7f7e98105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 shutdown_connections 2024-03-12T04:11:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-12T04:11:30.621+0000 7f7e9e439700 1 -- 172.21.15.119:0/1025862029 wait complete. 2024-03-12T04:11:30.935 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:30.935 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:30.935 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:30.935 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:30.935 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:30.935 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:30.935 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:30.936 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:30.936 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T04:11:30.936 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:30 smithi119 ceph-mon[36119]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T04:11:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:31.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:31.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:31.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T04:11:31.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:30 smithi042 ceph-mon[32260]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T04:11:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:31.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:31.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T04:11:31.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:30 smithi136 ceph-mon[35571]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T04:11:31.287 DEBUG:teuthology.orchestra.run.smithi119:osd.10> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.10.service 2024-03-12T04:11:31.290 INFO:tasks.cephadm:Deploying osd.11 on smithi136 with /dev/vg_nvme/lv_1... 2024-03-12T04:11:31.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-12T04:11:31.536 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:11:31.922 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:31 smithi136 ceph-mon[35571]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:31.922 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:31 smithi136 ceph-mon[35571]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-12T04:11:31.922 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:31 smithi136 ceph-mon[35571]: osdmap e72: 11 total, 10 up, 11 in 2024-03-12T04:11:31.922 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:31 smithi136 ceph-mon[35571]: from='osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:11:31.922 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:31 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:31.922 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:31 smithi136 ceph-mon[35571]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:11:31.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:31 smithi119 ceph-mon[36119]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:31.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:31 smithi119 ceph-mon[36119]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-12T04:11:31.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:31 smithi119 ceph-mon[36119]: osdmap e72: 11 total, 10 up, 11 in 2024-03-12T04:11:31.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:31 smithi119 ceph-mon[36119]: from='osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:11:31.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:31 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:31.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:31 smithi119 ceph-mon[36119]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:11:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:31 smithi042 ceph-mon[32260]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:31 smithi042 ceph-mon[32260]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-12T04:11:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:31 smithi042 ceph-mon[32260]: osdmap e72: 11 total, 10 up, 11 in 2024-03-12T04:11:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:31 smithi042 ceph-mon[32260]: from='osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:11:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:31 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:31 smithi042 ceph-mon[32260]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-12T04:11:33.195 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:11:33.050+0000 7f0b594fe700 -1 osd.10 0 waiting for initial osdmap 2024-03-12T04:11:33.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:11:33.061+0000 7f0b512e9700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-12T04:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-12T04:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-03-12T04:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-03-12T04:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112236 s, 934 MB/s 2024-03-12T04:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 2024-03-12T04:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-03-12T04:11:33.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-mon[36119]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:11:33.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-mon[36119]: osdmap e73: 11 total, 10 up, 11 in 2024-03-12T04:11:33.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:33.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:33.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-mon[36119]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:33.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:33 smithi119 ceph-mon[36119]: from='osd.10 ' entity='osd.10' 2024-03-12T04:11:33.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:33 smithi136 ceph-mon[35571]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:11:33.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:33 smithi136 ceph-mon[35571]: osdmap e73: 11 total, 10 up, 11 in 2024-03-12T04:11:33.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:33 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:33.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:33 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:33.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:33 smithi136 ceph-mon[35571]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:33.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:33 smithi136 ceph-mon[35571]: from='osd.10 ' entity='osd.10' 2024-03-12T04:11:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:33 smithi042 ceph-mon[32260]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-12T04:11:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:33 smithi042 ceph-mon[32260]: osdmap e73: 11 total, 10 up, 11 in 2024-03-12T04:11:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:33 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:33 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:33 smithi042 ceph-mon[32260]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-03-12T04:11:34.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:33 smithi042 ceph-mon[32260]: from='osd.10 ' entity='osd.10' 2024-03-12T04:11:34.396 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-03-12T04:11:34.640 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] boot 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: osdmap e74: 11 total, 11 up, 11 in 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.896 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.897 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.897 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.897 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:34.897 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:34.897 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:34.897 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:34.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:11:34.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:11:34.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] boot 2024-03-12T04:11:34.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: osdmap e74: 11 total, 11 up, 11 in 2024-03-12T04:11:34.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:34.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:34.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:34.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:34.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:34.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:34.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:34.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:11:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:11:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: osd.10 [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] boot 2024-03-12T04:11:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: osdmap e74: 11 total, 11 up, 11 in 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:35.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:11:35.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:35.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:35 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi119 2024-03-12T04:11:35.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:35 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi119 to 4272M 2024-03-12T04:11:35.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:35 smithi136 ceph-mon[35571]: osdmap e75: 11 total, 11 up, 11 in 2024-03-12T04:11:35.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:35 smithi136 ceph-mon[35571]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:35.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:35 smithi136 ceph-mon[35571]: osdmap e76: 11 total, 11 up, 11 in 2024-03-12T04:11:35.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:35 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi119 2024-03-12T04:11:35.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:35 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi119 to 4272M 2024-03-12T04:11:35.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:35 smithi119 ceph-mon[36119]: osdmap e75: 11 total, 11 up, 11 in 2024-03-12T04:11:35.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:35 smithi119 ceph-mon[36119]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:35.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:35 smithi119 ceph-mon[36119]: osdmap e76: 11 total, 11 up, 11 in 2024-03-12T04:11:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:35 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi119 2024-03-12T04:11:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:35 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi119 to 4272M 2024-03-12T04:11:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:35 smithi042 ceph-mon[32260]: osdmap e75: 11 total, 11 up, 11 in 2024-03-12T04:11:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:35 smithi042 ceph-mon[32260]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:36.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:35 smithi042 ceph-mon[32260]: osdmap e76: 11 total, 11 up, 11 in 2024-03-12T04:11:36.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 -- 172.21.15.136:0/981017554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0101da0 msgr2=0x7f88e0102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 --2- 172.21.15.136:0/981017554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0101da0 0x7f88e0102200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009990 tx=0x7f88d0009ca0 comp rx=0 tx=0).stop 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 -- 172.21.15.136:0/981017554 shutdown_connections 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 --2- 172.21.15.136:0/981017554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e0102740 0x7f88e0108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 --2- 172.21.15.136:0/981017554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e0100ba0 0x7f88e0100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 --2- 172.21.15.136:0/981017554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0101da0 0x7f88e0102200 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 -- 172.21.15.136:0/981017554 >> 172.21.15.136:0/981017554 conn(0x7f88e00fc140 msgr2=0x7f88e00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.333+0000 7f88e63c5700 1 -- 172.21.15.136:0/981017554 shutdown_connections 2024-03-12T04:11:36.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.334+0000 7f88e63c5700 1 -- 172.21.15.136:0/981017554 wait complete. 2024-03-12T04:11:36.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.334+0000 7f88e63c5700 1 Processor -- start 2024-03-12T04:11:36.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.334+0000 7f88e63c5700 1 -- start start 2024-03-12T04:11:36.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e63c5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e0100ba0 0x7f88e0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:36.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e63c5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e0101da0 0x7f88e0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:36.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e63c5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0102740 0x7f88e019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:36.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e63c5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e019f790 con 0x7f88e0101da0 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e63c5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e019f8d0 con 0x7f88e0100ba0 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e63c5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e019fa90 con 0x7f88e0102740 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0102740 0x7f88e019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0102740 0x7f88e019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:43052/0 (socket says 172.21.15.136:43052) 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 -- 172.21.15.136:0/3082758899 learned_addr learned my addr 172.21.15.136:0/3082758899 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88df7fe700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e0101da0 0x7f88e0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88dffff700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e0100ba0 0x7f88e0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 -- 172.21.15.136:0/3082758899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e0100ba0 msgr2=0x7f88e0197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e0100ba0 0x7f88e0197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 -- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e0101da0 msgr2=0x7f88e0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e0101da0 0x7f88e0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 -- 172.21.15.136:0/3082758899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00095f0 con 0x7f88e0102740 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e4962700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0102740 0x7f88e019f1c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f88d800ada0 tx=0x7f88d800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:11:36.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88dd7fa700 1 -- 172.21.15.136:0/3082758899 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800f070 con 0x7f88e0102740 2024-03-12T04:11:36.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88dd7fa700 1 -- 172.21.15.136:0/3082758899 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88d8007b30 con 0x7f88e0102740 2024-03-12T04:11:36.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e019fd70 con 0x7f88e0102740 2024-03-12T04:11:36.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.335+0000 7f88dd7fa700 1 -- 172.21.15.136:0/3082758899 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8005280 con 0x7f88e0102740 2024-03-12T04:11:36.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.336+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e01a0240 con 0x7f88e0102740 2024-03-12T04:11:36.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.337+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e010c430 con 0x7f88e0102740 2024-03-12T04:11:36.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.338+0000 7f88dd7fa700 1 -- 172.21.15.136:0/3082758899 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f88d800c8d0 con 0x7f88e0102740 2024-03-12T04:11:36.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.338+0000 7f88dd7fa700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f88cc0a1b40 0x7f88cc0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:11:36.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.338+0000 7f88dd7fa700 1 -- 172.21.15.136:0/3082758899 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f88d80cc2c0 con 0x7f88e0102740 2024-03-12T04:11:36.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.338+0000 7f88dffff700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f88cc0a1b40 0x7f88cc0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:11:36.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.339+0000 7f88dffff700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f88cc0a1b40 0x7f88cc0a4000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f88e0199120 tx=0x7f88c8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:11:36.344 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.343+0000 7f88dd7fa700 1 -- 172.21.15.136:0/3082758899 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f88d8095b90 con 0x7f88e0102740 2024-03-12T04:11:36.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:36.543+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f88e00fe380 con 0x7f88cc0a1b40 2024-03-12T04:11:36.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:36 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:11:36.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:36 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:11:36.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:36 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:36 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:11:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:36 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:11:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:36 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:36 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T04:11:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:36 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T04:11:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:36 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:37.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:37 smithi136 ceph-mon[35571]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:11:37.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:37 smithi136 ceph-mon[35571]: osdmap e77: 11 total, 11 up, 11 in 2024-03-12T04:11:37.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:37 smithi136 ceph-mon[35571]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T04:11:37.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:37 smithi136 ceph-mon[35571]: osdmap e78: 11 total, 11 up, 11 in 2024-03-12T04:11:37.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:37 smithi119 ceph-mon[36119]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:11:37.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:37 smithi119 ceph-mon[36119]: osdmap e77: 11 total, 11 up, 11 in 2024-03-12T04:11:37.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:37 smithi119 ceph-mon[36119]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T04:11:37.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:37 smithi119 ceph-mon[36119]: osdmap e78: 11 total, 11 up, 11 in 2024-03-12T04:11:38.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:37 smithi042 ceph-mon[32260]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:11:38.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:37 smithi042 ceph-mon[32260]: osdmap e77: 11 total, 11 up, 11 in 2024-03-12T04:11:38.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:37 smithi042 ceph-mon[32260]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T04:11:38.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:37 smithi042 ceph-mon[32260]: osdmap e78: 11 total, 11 up, 11 in 2024-03-12T04:11:39.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:38 smithi119 ceph-mon[36119]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-12T04:11:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:38 smithi136 ceph-mon[35571]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-12T04:11:39.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:38 smithi042 ceph-mon[32260]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-12T04:11:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:39 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/1148625675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]: dispatch 2024-03-12T04:11:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:39 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]: dispatch 2024-03-12T04:11:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:39 smithi136 ceph-mon[35571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]': finished 2024-03-12T04:11:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:39 smithi136 ceph-mon[35571]: osdmap e79: 12 total, 11 up, 12 in 2024-03-12T04:11:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:39 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:11:40.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:39 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/1148625675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]: dispatch 2024-03-12T04:11:40.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:39 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]: dispatch 2024-03-12T04:11:40.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:39 smithi042 ceph-mon[32260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]': finished 2024-03-12T04:11:40.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:39 smithi042 ceph-mon[32260]: osdmap e79: 12 total, 11 up, 12 in 2024-03-12T04:11:40.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:39 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:11:40.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:39 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/1148625675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]: dispatch 2024-03-12T04:11:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:39 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]: dispatch 2024-03-12T04:11:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:39 smithi119 ceph-mon[36119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf7c32db-7617-4fe4-ab70-4bf8efcb0869"}]': finished 2024-03-12T04:11:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:39 smithi119 ceph-mon[36119]: osdmap e79: 12 total, 11 up, 12 in 2024-03-12T04:11:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:39 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:11:41.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:40 smithi042 ceph-mon[32260]: from='client.? 172.21.15.136:0/2203957510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:11:41.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:40 smithi042 ceph-mon[32260]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-03-12T04:11:41.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:40 smithi119 ceph-mon[36119]: from='client.? 172.21.15.136:0/2203957510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:11:41.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:40 smithi119 ceph-mon[36119]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-03-12T04:11:41.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:40 smithi136 ceph-mon[35571]: from='client.? 172.21.15.136:0/2203957510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T04:11:41.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:40 smithi136 ceph-mon[35571]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-03-12T04:11:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:42 smithi136 ceph-mon[35571]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-12T04:11:43.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:42 smithi042 ceph-mon[32260]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-12T04:11:43.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:42 smithi119 ceph-mon[36119]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-12T04:11:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:44 smithi136 ceph-mon[35571]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:45.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:44 smithi042 ceph-mon[32260]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:45.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:44 smithi119 ceph-mon[36119]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:47.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:46 smithi136 ceph-mon[35571]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:47.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:46 smithi042 ceph-mon[32260]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:47.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:46 smithi119 ceph-mon[36119]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:48.047 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-12T04:11:48.048 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:48.048 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:47 smithi136 ceph-mon[35571]: Deploying daemon osd.11 on smithi136 2024-03-12T04:11:48.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-12T04:11:48.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:48.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:47 smithi042 ceph-mon[32260]: Deploying daemon osd.11 on smithi136 2024-03-12T04:11:48.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-12T04:11:48.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:11:48.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:47 smithi119 ceph-mon[36119]: Deploying daemon osd.11 on smithi136 2024-03-12T04:11:49.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:48 smithi136 ceph-mon[35571]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:49.318 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:48 smithi042 ceph-mon[32260]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:49.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:48 smithi119 ceph-mon[36119]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:50 smithi136 ceph-mon[35571]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:50 smithi042 ceph-mon[32260]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:51.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:50 smithi119 ceph-mon[36119]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:53.137 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:52 smithi136 ceph-mon[35571]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:53.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:52 smithi042 ceph-mon[32260]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:53.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:52 smithi119 ceph-mon[36119]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:54.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:54 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:54.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:54 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:54.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:54 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:54.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:54 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:54.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:54 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:54.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:54 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:54 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:54 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:54 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:11:55.625 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:55 smithi136 ceph-mon[35571]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:55.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:55 smithi119 ceph-mon[36119]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:55.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:55 smithi042 ceph-mon[32260]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:57.168 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 11 on host 'smithi136' 2024-03-12T04:11:57.169 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.165+0000 7f88dd7fa700 1 -- 172.21.15.136:0/3082758899 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f88e00fe380 con 0x7f88cc0a1b40 2024-03-12T04:11:57.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f88cc0a1b40 msgr2=0x7f88cc0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:57.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f88cc0a1b40 0x7f88cc0a4000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f88e0199120 tx=0x7f88c8009190 comp rx=0 tx=0).stop 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0102740 msgr2=0x7f88e019f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0102740 0x7f88e019f1c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f88d800ada0 tx=0x7f88d800c330 comp rx=0 tx=0).stop 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 shutdown_connections 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f88cc0a1b40 0x7f88cc0a4000 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88e0102740 0x7f88e019f1c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e0100ba0 0x7f88e0197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 --2- 172.21.15.136:0/3082758899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e0101da0 0x7f88e0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.168+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 >> 172.21.15.136:0/3082758899 conn(0x7f88e00fc140 msgr2=0x7f88e00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.169+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 shutdown_connections 2024-03-12T04:11:57.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:11:57.169+0000 7f88e63c5700 1 -- 172.21.15.136:0/3082758899 wait complete. 2024-03-12T04:11:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:56 smithi042 ceph-mon[32260]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:57.321 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:56 smithi136 ceph-mon[35571]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:57.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:56 smithi119 ceph-mon[36119]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:57.864 DEBUG:teuthology.orchestra.run.smithi136:osd.11> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.11.service 2024-03-12T04:11:57.867 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-12T04:11:57.867 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T04:11:58.126 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:11:58.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:58 smithi042 ceph-mon[32260]: from='osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T04:11:58.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:58 smithi042 ceph-mon[32260]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T04:11:58.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:58 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:58 smithi119 ceph-mon[36119]: from='osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T04:11:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:58 smithi119 ceph-mon[36119]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T04:11:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:58 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:58 smithi136 ceph-mon[35571]: from='osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T04:11:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:58 smithi136 ceph-mon[35571]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T04:11:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:58 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:11:59.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:59 smithi119 ceph-mon[36119]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-12T04:11:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:59 smithi119 ceph-mon[36119]: osdmap e80: 12 total, 11 up, 12 in 2024-03-12T04:11:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:59 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:11:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:59 smithi119 ceph-mon[36119]: from='osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:11:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:59 smithi119 ceph-mon[36119]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:11:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:11:59 smithi119 ceph-mon[36119]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-mon[35571]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-12T04:11:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-mon[35571]: osdmap e80: 12 total, 11 up, 12 in 2024-03-12T04:11:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:11:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-mon[35571]: from='osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:11:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-mon[35571]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:11:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-mon[35571]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:59 smithi042 ceph-mon[32260]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-12T04:11:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:59 smithi042 ceph-mon[32260]: osdmap e80: 12 total, 11 up, 12 in 2024-03-12T04:11:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:59 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:11:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:59 smithi042 ceph-mon[32260]: from='osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:11:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:59 smithi042 ceph-mon[32260]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-03-12T04:11:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:11:59 smithi042 ceph-mon[32260]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:11:59.829 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:11:59.563+0000 7fb308a12700 -1 osd.11 0 waiting for initial osdmap 2024-03-12T04:11:59.829 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:11:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:11:59.578+0000 7fb3007fd700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T04:12:00.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 -- 172.21.15.42:0/3213288899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c102ea0 msgr2=0x7f6a5c103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3213288899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c102ea0 0x7f6a5c103300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a440099c0 tx=0x7f6a44009cd0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 -- 172.21.15.42:0/3213288899 shutdown_connections 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3213288899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c103840 0x7f6a5c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3213288899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c101ca0 0x7f6a5c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3213288899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c102ea0 0x7f6a5c103300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 -- 172.21.15.42:0/3213288899 >> 172.21.15.42:0/3213288899 conn(0x7f6a5c0fd280 msgr2=0x7f6a5c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 -- 172.21.15.42:0/3213288899 shutdown_connections 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.106+0000 7f6a6294c700 1 -- 172.21.15.42:0/3213288899 wait complete. 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 Processor -- start 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 -- start start 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c101ca0 0x7f6a5c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:00.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c102ea0 0x7f6a5c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c103840 0x7f6a5c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c19f710 con 0x7f6a5c101ca0 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c19f850 con 0x7f6a5c103840 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a6294c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c19f9e0 con 0x7f6a5c102ea0 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a5bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c101ca0 0x7f6a5c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a5bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c101ca0 0x7f6a5c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44936/0 (socket says 172.21.15.42:44936) 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.107+0000 7f6a5bfff700 1 -- 172.21.15.42:0/3222382429 learned_addr learned my addr 172.21.15.42:0/3222382429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a5b7fe700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c102ea0 0x7f6a5c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a60ee9700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c103840 0x7f6a5c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a5bfff700 1 -- 172.21.15.42:0/3222382429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c102ea0 msgr2=0x7f6a5c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a5bfff700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c102ea0 0x7f6a5c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a5bfff700 1 -- 172.21.15.42:0/3222382429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c103840 msgr2=0x7f6a5c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a5bfff700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c103840 0x7f6a5c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a5bfff700 1 -- 172.21.15.42:0/3222382429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440095f0 con 0x7f6a5c101ca0 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a60ee9700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c103840 0x7f6a5c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a5bfff700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c101ca0 0x7f6a5c197b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00d9b0 tx=0x7f6a4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a597fa700 1 -- 172.21.15.42:0/3222382429 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c004020 con 0x7f6a5c101ca0 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a597fa700 1 -- 172.21.15.42:0/3222382429 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a4c009730 con 0x7f6a5c101ca0 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c19fcc0 con 0x7f6a5c101ca0 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a597fa700 1 -- 172.21.15.42:0/3222382429 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c003a70 con 0x7f6a5c101ca0 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.108+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c1a01c0 con 0x7f6a5c101ca0 2024-03-12T04:12:00.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.109+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c109150 con 0x7f6a5c101ca0 2024-03-12T04:12:00.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.111+0000 7f6a597fa700 1 -- 172.21.15.42:0/3222382429 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6a4c0098a0 con 0x7f6a5c101ca0 2024-03-12T04:12:00.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.111+0000 7f6a597fa700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6a480a1b40 0x7f6a480a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:00.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.112+0000 7f6a5b7fe700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6a480a1b40 0x7f6a480a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:00.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.112+0000 7f6a597fa700 1 -- 172.21.15.42:0/3222382429 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f6a4c0ccea0 con 0x7f6a5c101ca0 2024-03-12T04:12:00.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.112+0000 7f6a5b7fe700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6a480a1b40 0x7f6a480a4000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009990 tx=0x7f6a44005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:00.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.115+0000 7f6a597fa700 1 -- 172.21.15.42:0/3222382429 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a4c096650 con 0x7f6a5c101ca0 2024-03-12T04:12:00.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.317+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f6a5c10d4d0 con 0x7f6a5c101ca0 2024-03-12T04:12:00.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.318+0000 7f6a597fa700 1 -- 172.21.15.42:0/3222382429 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f6a4c099d10 con 0x7f6a5c101ca0 2024-03-12T04:12:00.320 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6a480a1b40 msgr2=0x7f6a480a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6a480a1b40 0x7f6a480a4000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009990 tx=0x7f6a44005ee0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c101ca0 msgr2=0x7f6a5c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c101ca0 0x7f6a5c197b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00d9b0 tx=0x7f6a4c00dcc0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 shutdown_connections 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6a480a1b40 0x7f6a480a4000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c103840 0x7f6a5c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c101ca0 0x7f6a5c197b80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 --2- 172.21.15.42:0/3222382429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c102ea0 0x7f6a5c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 >> 172.21.15.42:0/3222382429 conn(0x7f6a5c0fd280 msgr2=0x7f6a5c106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 shutdown_connections 2024-03-12T04:12:00.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:00.320+0000 7f6a6294c700 1 -- 172.21.15.42:0/3222382429 wait complete. 2024-03-12T04:12:00.421 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:00 smithi042 ceph-mon[32260]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:12:00.421 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:00 smithi042 ceph-mon[32260]: osdmap e81: 12 total, 11 up, 12 in 2024-03-12T04:12:00.421 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:00.421 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:00 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:00.421 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:00 smithi042 ceph-mon[32260]: from='osd.11 ' entity='osd.11' 2024-03-12T04:12:00.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:00 smithi119 ceph-mon[36119]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:12:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:00 smithi119 ceph-mon[36119]: osdmap e81: 12 total, 11 up, 12 in 2024-03-12T04:12:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:00 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:00 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:00 smithi119 ceph-mon[36119]: from='osd.11 ' entity='osd.11' 2024-03-12T04:12:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:00 smithi136 ceph-mon[35571]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-03-12T04:12:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:00 smithi136 ceph-mon[35571]: osdmap e81: 12 total, 11 up, 12 in 2024-03-12T04:12:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:00 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:00 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:00 smithi136 ceph-mon[35571]: from='osd.11 ' entity='osd.11' 2024-03-12T04:12:01.112 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1710216693,"num_in_osds":12,"osd_in_since":1710216699,"num_remapped_pgs":0} 2024-03-12T04:12:01.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: purged_snaps scrub starts 2024-03-12T04:12:01.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: purged_snaps scrub ok 2024-03-12T04:12:01.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:01.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3222382429' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T04:12:01.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] boot 2024-03-12T04:12:01.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: osdmap e82: 12 total, 12 up, 12 in 2024-03-12T04:12:01.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:01.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:01 smithi042 ceph-mon[32260]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:12:01.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: purged_snaps scrub starts 2024-03-12T04:12:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: purged_snaps scrub ok 2024-03-12T04:12:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3222382429' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T04:12:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] boot 2024-03-12T04:12:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: osdmap e82: 12 total, 12 up, 12 in 2024-03-12T04:12:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:01 smithi119 ceph-mon[36119]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: purged_snaps scrub starts 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: purged_snaps scrub ok 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3222382429' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: osd.11 [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] boot 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: osdmap e82: 12 total, 12 up, 12 in 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:12:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:01 smithi136 ceph-mon[35571]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T04:12:02.114 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T04:12:02.359 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:12:02.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: Detected new or changed devices on smithi136 2024-03-12T04:12:02.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi136 to 4272M 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:02 smithi042 ceph-mon[32260]: osdmap e83: 12 total, 12 up, 12 in 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: Detected new or changed devices on smithi136 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi136 to 4272M 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:02.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:02.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:02.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:02.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:02.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:02 smithi119 ceph-mon[36119]: osdmap e83: 12 total, 12 up, 12 in 2024-03-12T04:12:02.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: Detected new or changed devices on smithi136 2024-03-12T04:12:02.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi136 to 4272M 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:02.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:02 smithi136 ceph-mon[35571]: osdmap e83: 12 total, 12 up, 12 in 2024-03-12T04:12:03.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:03 smithi042 ceph-mon[32260]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:03.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:03 smithi119 ceph-mon[36119]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:03.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:03 smithi136 ceph-mon[35571]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 -- 172.21.15.42:0/2853373470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c100b90 msgr2=0x7fce3c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 --2- 172.21.15.42:0/2853373470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c100b90 0x7fce3c100fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0099c0 tx=0x7fce2c009cd0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 -- 172.21.15.42:0/2853373470 shutdown_connections 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 --2- 172.21.15.42:0/2853373470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce3c102730 0x7fce3c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 --2- 172.21.15.42:0/2853373470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c100b90 0x7fce3c100fb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 --2- 172.21.15.42:0/2853373470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce3c101d90 0x7fce3c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 -- 172.21.15.42:0/2853373470 >> 172.21.15.42:0/2853373470 conn(0x7fce3c0fc130 msgr2=0x7fce3c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 -- 172.21.15.42:0/2853373470 shutdown_connections 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.026+0000 7fce41104700 1 -- 172.21.15.42:0/2853373470 wait complete. 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 Processor -- start 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 -- start start 2024-03-12T04:12:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce3c100b90 0x7fce3c1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce3c101d90 0x7fce3c1a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c102730 0x7fce3c1a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1a5cd0 con 0x7fce3c102730 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1a5e40 con 0x7fce3c101d90 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce41104700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1a6000 con 0x7fce3c100b90 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3b59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c102730 0x7fce3c1a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3b59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c102730 0x7fce3c1a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43652/0 (socket says 172.21.15.42:43652) 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3b59e700 1 -- 172.21.15.42:0/1258547299 learned_addr learned my addr 172.21.15.42:0/1258547299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3ad9d700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce3c100b90 0x7fce3c1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3a59c700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce3c101d90 0x7fce3c1a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3b59e700 1 -- 172.21.15.42:0/1258547299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce3c100b90 msgr2=0x7fce3c1a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3b59e700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce3c100b90 0x7fce3c1a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3b59e700 1 -- 172.21.15.42:0/1258547299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce3c101d90 msgr2=0x7fce3c1a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.027+0000 7fce3b59e700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce3c101d90 0x7fce3c1a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce3b59e700 1 -- 172.21.15.42:0/1258547299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0095f0 con 0x7fce3c102730 2024-03-12T04:12:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce3a59c700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce3c101d90 0x7fce3c1a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce3b59e700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c102730 0x7fce3c1a5500 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce3400ada0 tx=0x7fce3400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce33fff700 1 -- 172.21.15.42:0/1258547299 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400f070 con 0x7fce3c102730 2024-03-12T04:12:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c1a62e0 con 0x7fce3c102730 2024-03-12T04:12:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c1a67b0 con 0x7fce3c102730 2024-03-12T04:12:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce33fff700 1 -- 172.21.15.42:0/1258547299 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fce34007b30 con 0x7fce3c102730 2024-03-12T04:12:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.028+0000 7fce33fff700 1 -- 172.21.15.42:0/1258547299 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce340053e0 con 0x7fce3c102730 2024-03-12T04:12:04.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.029+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c005360 con 0x7fce3c102730 2024-03-12T04:12:04.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.030+0000 7fce33fff700 1 -- 172.21.15.42:0/1258547299 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fce34004ad0 con 0x7fce3c102730 2024-03-12T04:12:04.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.031+0000 7fce33fff700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fce280a1b40 0x7fce280a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:04.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.031+0000 7fce3ad9d700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fce280a1b40 0x7fce280a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:04.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.031+0000 7fce33fff700 1 -- 172.21.15.42:0/1258547299 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fce340cc480 con 0x7fce3c102730 2024-03-12T04:12:04.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.031+0000 7fce3ad9d700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fce280a1b40 0x7fce280a4000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:04.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.035+0000 7fce33fff700 1 -- 172.21.15.42:0/1258547299 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce34095bc0 con 0x7fce3c102730 2024-03-12T04:12:04.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.238+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fce1c0058e0 con 0x7fce3c102730 2024-03-12T04:12:04.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.238+0000 7fce33fff700 1 -- 172.21.15.42:0/1258547299 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) v1 ==== 74+0+133 (secure 0 0 0) 0x7fce34099280 con 0x7fce3c102730 2024-03-12T04:12:04.240 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fce280a1b40 msgr2=0x7fce280a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fce280a1b40 0x7fce280a4000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c005040 comp rx=0 tx=0).stop 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c102730 msgr2=0x7fce3c1a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c102730 0x7fce3c1a5500 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce3400ada0 tx=0x7fce3400c330 comp rx=0 tx=0).stop 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 shutdown_connections 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fce280a1b40 0x7fce280a4000 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce3c102730 0x7fce3c1a5500 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce3c100b90 0x7fce3c1a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 --2- 172.21.15.42:0/1258547299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce3c101d90 0x7fce3c1a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 >> 172.21.15.42:0/1258547299 conn(0x7fce3c0fc130 msgr2=0x7fce3c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:04.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 shutdown_connections 2024-03-12T04:12:04.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:04.240+0000 7fce41104700 1 -- 172.21.15.42:0/1258547299 wait complete. 2024-03-12T04:12:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1258547299' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T04:12:04.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1258547299' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T04:12:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1258547299' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T04:12:05.099 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1710216720,"num_in_osds":12,"osd_in_since":1710216699,"num_remapped_pgs":0} 2024-03-12T04:12:05.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T04:12:05.346 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:12:05.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:05 smithi042 ceph-mon[32260]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:05.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:05 smithi119 ceph-mon[36119]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:05.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:05 smithi136 ceph-mon[35571]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:07.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 -- 172.21.15.42:0/2100284001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c105530 msgr2=0x7f993c10d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 --2- 172.21.15.42:0/2100284001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c105530 0x7f993c10d100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9934009990 tx=0x7f9934009ca0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 -- 172.21.15.42:0/2100284001 shutdown_connections 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 --2- 172.21.15.42:0/2100284001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c105530 0x7f993c10d100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 --2- 172.21.15.42:0/2100284001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c100680 0x7f993c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 --2- 172.21.15.42:0/2100284001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f993c101030 0x7f993c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 -- 172.21.15.42:0/2100284001 >> 172.21.15.42:0/2100284001 conn(0x7f993c075db0 msgr2=0x7f993c0761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 -- 172.21.15.42:0/2100284001 shutdown_connections 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.165+0000 7f99421a0700 1 -- 172.21.15.42:0/2100284001 wait complete. 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 Processor -- start 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 -- start start 2024-03-12T04:12:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f993c100680 0x7f993c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c101030 0x7f993c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c105530 0x7f993c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c19ce00 con 0x7f993c101030 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c19cf70 con 0x7f993c105530 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f99421a0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c19d130 con 0x7f993c100680 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c101030 0x7f993c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c101030 0x7f993c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43678/0 (socket says 172.21.15.42:43678) 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 -- 172.21.15.42:0/4219848205 learned_addr learned my addr 172.21.15.42:0/4219848205 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994119e700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f993c100680 0x7f993c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994199f700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c105530 0x7f993c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 -- 172.21.15.42:0/4219848205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f993c100680 msgr2=0x7f993c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f993c100680 0x7f993c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 -- 172.21.15.42:0/4219848205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c105530 msgr2=0x7f993c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:07.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c105530 0x7f993c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 -- 172.21.15.42:0/4219848205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99340095f0 con 0x7f993c101030 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994199f700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c105530 0x7f993c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.166+0000 7f994099d700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c101030 0x7f993c198460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f993000ddf0 tx=0x7f993000a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.167+0000 7f993a7fc700 1 -- 172.21.15.42:0/4219848205 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000de20 con 0x7f993c101030 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.167+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c19d410 con 0x7f993c101030 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.167+0000 7f993a7fc700 1 -- 172.21.15.42:0/4219848205 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f993000e900 con 0x7f993c101030 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.167+0000 7f993a7fc700 1 -- 172.21.15.42:0/4219848205 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930016860 con 0x7f993c101030 2024-03-12T04:12:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.167+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c1a4c70 con 0x7f993c101030 2024-03-12T04:12:07.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.168+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c10a7d0 con 0x7f993c101030 2024-03-12T04:12:07.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.169+0000 7f993a7fc700 1 -- 172.21.15.42:0/4219848205 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f993000ea70 con 0x7f993c101030 2024-03-12T04:12:07.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.169+0000 7f993a7fc700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99240a1b40 0x7f99240a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:07.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.170+0000 7f994119e700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99240a1b40 0x7f99240a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:07.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.170+0000 7f993a7fc700 1 -- 172.21.15.42:0/4219848205 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f99300cceb0 con 0x7f993c101030 2024-03-12T04:12:07.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.170+0000 7f994119e700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99240a1b40 0x7f99240a4000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f992c00a400 tx=0x7f992c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:07.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.174+0000 7f993a7fc700 1 -- 172.21.15.42:0/4219848205 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9930096560 con 0x7f993c101030 2024-03-12T04:12:07.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:06 smithi042 ceph-mon[32260]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:07.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.378+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f993c068930 con 0x7f993c101030 2024-03-12T04:12:07.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.379+0000 7f993a7fc700 1 -- 172.21.15.42:0/4219848205 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18456 (secure 0 0 0) 0x7f9930099c20 con 0x7f993c101030 2024-03-12T04:12:07.382 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:07.382 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:02.144274+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99240a1b40 msgr2=0x7f99240a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99240a1b40 0x7f99240a4000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f992c00a400 tx=0x7f992c00d040 comp rx=0 tx=0).stop 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c101030 msgr2=0x7f993c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c101030 0x7f993c198460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f993000ddf0 tx=0x7f993000a530 comp rx=0 tx=0).stop 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 shutdown_connections 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f99240a1b40 0x7f99240a4000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c105530 0x7f993c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f993c100680 0x7f993c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 --2- 172.21.15.42:0/4219848205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f993c101030 0x7f993c198460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:07.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 >> 172.21.15.42:0/4219848205 conn(0x7f993c075db0 msgr2=0x7f993c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:07.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 shutdown_connections 2024-03-12T04:12:07.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:07.381+0000 7f99421a0700 1 -- 172.21.15.42:0/4219848205 wait complete. 2024-03-12T04:12:07.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:06 smithi119 ceph-mon[36119]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:06 smithi136 ceph-mon[35571]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:08.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:07 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/4219848205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:08.271 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:07 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/4219848205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:08.303 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:12:08.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-12T04:12:08.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:07 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/4219848205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:08.550 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:12:09.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:08 smithi119 ceph-mon[36119]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:09.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:08 smithi042 ceph-mon[32260]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:08 smithi136 ceph-mon[35571]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 -- 172.21.15.42:0/883196311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc069920 msgr2=0x7f2dfc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/883196311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc069920 0x7f2dfc069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2de40099c0 tx=0x7f2de4009cd0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 -- 172.21.15.42:0/883196311 shutdown_connections 2024-03-12T04:12:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/883196311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dfc06a2c0 0x7f2dfc10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/883196311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc107cc0 0x7f2dfc1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/883196311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc069920 0x7f2dfc069d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 -- 172.21.15.42:0/883196311 >> 172.21.15.42:0/883196311 conn(0x7f2dfc077aa0 msgr2=0x7f2dfc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 -- 172.21.15.42:0/883196311 shutdown_connections 2024-03-12T04:12:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.313+0000 7f2e00fa8700 1 -- 172.21.15.42:0/883196311 wait complete. 2024-03-12T04:12:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 Processor -- start 2024-03-12T04:12:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 -- start start 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc069920 0x7f2dfc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc06a2c0 0x7f2dfc198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dfc107cc0 0x7f2dfc19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc19cfe0 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc19d150 con 0x7f2dfc069920 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2e00fa8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc19d310 con 0x7f2dfc107cc0 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2df9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc06a2c0 0x7f2dfc198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2df9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc06a2c0 0x7f2dfc198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43698/0 (socket says 172.21.15.42:43698) 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2df9d9b700 1 -- 172.21.15.42:0/3389945849 learned_addr learned my addr 172.21.15.42:0/3389945849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2dfa59c700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc069920 0x7f2dfc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.314+0000 7f2dfad9d700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dfc107cc0 0x7f2dfc19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df9d9b700 1 -- 172.21.15.42:0/3389945849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dfc107cc0 msgr2=0x7f2dfc19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df9d9b700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dfc107cc0 0x7f2dfc19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df9d9b700 1 -- 172.21.15.42:0/3389945849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc069920 msgr2=0x7f2dfc198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df9d9b700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc069920 0x7f2dfc198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df9d9b700 1 -- 172.21.15.42:0/3389945849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40095f0 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df9d9b700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc06a2c0 0x7f2dfc198640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b580 tx=0x7f2de4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df37fe700 1 -- 172.21.15.42:0/3389945849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de401d070 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc19d590 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df37fe700 1 -- 172.21.15.42:0/3389945849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2de400fb20 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2df37fe700 1 -- 172.21.15.42:0/3389945849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4022d00 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.315+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc1a4e50 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.317+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc10a1e0 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.317+0000 7f2df37fe700 1 -- 172.21.15.42:0/3389945849 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2de400fc90 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.318+0000 7f2df37fe700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2de80a1ac0 0x7f2de80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.318+0000 7f2dfa59c700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2de80a1ac0 0x7f2de80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.318+0000 7f2df37fe700 1 -- 172.21.15.42:0/3389945849 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2de40cdbe0 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.318+0000 7f2dfa59c700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2de80a1ac0 0x7f2de80a3f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2dec007fd0 tx=0x7f2dec00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:10.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.322+0000 7f2df37fe700 1 -- 172.21.15.42:0/3389945849 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2de4097320 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.524+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2dfc068930 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.525+0000 7f2df37fe700 1 -- 172.21.15.42:0/3389945849 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2de4022e60 con 0x7f2dfc06a2c0 2024-03-12T04:12:10.527 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:12:10.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2de80a1ac0 msgr2=0x7f2de80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:10.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2de80a1ac0 0x7f2de80a3f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2dec007fd0 tx=0x7f2dec00c040 comp rx=0 tx=0).stop 2024-03-12T04:12:10.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc06a2c0 msgr2=0x7f2dfc198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:10.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc06a2c0 0x7f2dfc198640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b580 tx=0x7f2de4004730 comp rx=0 tx=0).stop 2024-03-12T04:12:10.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 shutdown_connections 2024-03-12T04:12:10.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2de80a1ac0 0x7f2de80a3f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dfc107cc0 0x7f2dfc19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc069920 0x7f2dfc198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 --2- 172.21.15.42:0/3389945849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc06a2c0 0x7f2dfc198640 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:10.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 >> 172.21.15.42:0/3389945849 conn(0x7f2dfc077aa0 msgr2=0x7f2dfc0ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:10.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 shutdown_connections 2024-03-12T04:12:10.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:10.527+0000 7f2e00fa8700 1 -- 172.21.15.42:0/3389945849 wait complete. 2024-03-12T04:12:10.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:10 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3389945849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:10.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:10 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3389945849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:10 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3389945849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:11.275 INFO:tasks.cephadm:Adding mds.a on smithi042 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.d on smithi042 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.g on smithi042 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.j on smithi042 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.b on smithi119 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.e on smithi119 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.h on smithi119 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.k on smithi119 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.c on smithi136 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.f on smithi136 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.i on smithi136 2024-03-12T04:12:11.276 INFO:tasks.cephadm:Adding mds.l on smithi136 2024-03-12T04:12:11.277 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph orch apply mds all '12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l' 2024-03-12T04:12:11.527 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:12:11.910 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:11 smithi136 ceph-mon[35571]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:11.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:11 smithi119 ceph-mon[36119]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:12.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:11 smithi042 ceph-mon[32260]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:13.237 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:12 smithi136 ceph-mon[35571]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:13.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 -- 172.21.15.136:0/799690149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8105090 msgr2=0x7f06a8107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:13.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/799690149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8105090 0x7f06a8107480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f069c009990 tx=0x7f069c009ca0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 -- 172.21.15.136:0/799690149 shutdown_connections 2024-03-12T04:12:13.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/799690149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a81079c0 0x7f06a8109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/799690149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a8102760 0x7f06a8104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/799690149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8105090 0x7f06a8107480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 -- 172.21.15.136:0/799690149 >> 172.21.15.136:0/799690149 conn(0x7f06a80fc170 msgr2=0x7f06a80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:13.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 -- 172.21.15.136:0/799690149 shutdown_connections 2024-03-12T04:12:13.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 -- 172.21.15.136:0/799690149 wait complete. 2024-03-12T04:12:13.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.238+0000 7f06ad3b0700 1 Processor -- start 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06ad3b0700 1 -- start start 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06ad3b0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a8102760 0x7f06a81020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06ad3b0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a8105090 0x7f06a8100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06ad3b0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a81079c0 0x7f06a8100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06ad3b0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a81013d0 con 0x7f06a8102760 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06ad3b0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8101540 con 0x7f06a8105090 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06ad3b0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a81a1c60 con 0x7f06a81079c0 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a81079c0 0x7f06a8100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a81079c0 0x7f06a8100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:38788/0 (socket says 172.21.15.136:38788) 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 -- 172.21.15.136:0/2486157273 learned_addr learned my addr 172.21.15.136:0/2486157273 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a6ffd700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a8102760 0x7f06a81020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a67fc700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a8105090 0x7f06a8100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 -- 172.21.15.136:0/2486157273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a8105090 msgr2=0x7f06a8100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a8105090 0x7f06a8100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 -- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a8102760 msgr2=0x7f06a81020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a8102760 0x7f06a81020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 -- 172.21.15.136:0/2486157273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0095f0 con 0x7f06a81079c0 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.239+0000 7f06a77fe700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a81079c0 0x7f06a8100c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06a0007ec0 tx=0x7f06a000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.240+0000 7f068ffff700 1 -- 172.21.15.136:0/2486157273 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0017070 con 0x7f06a81079c0 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.240+0000 7f068ffff700 1 -- 172.21.15.136:0/2486157273 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f06a000f040 con 0x7f06a81079c0 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.240+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a81a1f40 con 0x7f06a81079c0 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.240+0000 7f068ffff700 1 -- 172.21.15.136:0/2486157273 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0004280 con 0x7f06a81079c0 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.240+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a81a2350 con 0x7f06a81079c0 2024-03-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.241+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a80fdcf0 con 0x7f06a81079c0 2024-03-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.242+0000 7f068ffff700 1 -- 172.21.15.136:0/2486157273 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f06a000dcf0 con 0x7f06a81079c0 2024-03-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.242+0000 7f068ffff700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f06900a1ac0 0x7f06900a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.242+0000 7f068ffff700 1 -- 172.21.15.136:0/2486157273 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06a00cc5c0 con 0x7f06a81079c0 2024-03-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.242+0000 7f06a6ffd700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f06900a1ac0 0x7f06900a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.243+0000 7f06a6ffd700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f06900a1ac0 0x7f06900a3f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0698009a10 tx=0x7f0698009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:13.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.248+0000 7f068ffff700 1 -- 172.21.15.136:0/2486157273 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06a0095d00 con 0x7f06a81079c0 2024-03-12T04:12:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:12 smithi042 ceph-mon[32260]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:12 smithi119 ceph-mon[36119]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:13.483 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.481+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f06a802da90 con 0x7f06900a1ac0 2024-03-12T04:12:13.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.490+0000 7f068ffff700 1 -- 172.21.15.136:0/2486157273 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f06a802da90 con 0x7f06900a1ac0 2024-03-12T04:12:13.492 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mds.all update... 2024-03-12T04:12:13.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f06900a1ac0 msgr2=0x7f06900a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:13.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f06900a1ac0 0x7f06900a3f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0698009a10 tx=0x7f0698009190 comp rx=0 tx=0).stop 2024-03-12T04:12:13.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a81079c0 msgr2=0x7f06a8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:13.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a81079c0 0x7f06a8100c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06a0007ec0 tx=0x7f06a000d3b0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 shutdown_connections 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f06900a1ac0 0x7f06900a3f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a81079c0 0x7f06a8100c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a8102760 0x7f06a81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 --2- 172.21.15.136:0/2486157273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a8105090 0x7f06a8100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 >> 172.21.15.136:0/2486157273 conn(0x7f06a80fc170 msgr2=0x7f06a81030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 shutdown_connections 2024-03-12T04:12:13.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-12T04:12:13.492+0000 7f06ad3b0700 1 -- 172.21.15.136:0/2486157273 wait complete. 2024-03-12T04:12:14.201 DEBUG:teuthology.orchestra.run.smithi042:mds.a> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.a.service 2024-03-12T04:12:14.204 DEBUG:teuthology.orchestra.run.smithi042:mds.d> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.d.service 2024-03-12T04:12:14.247 DEBUG:teuthology.orchestra.run.smithi042:mds.g> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.g.service 2024-03-12T04:12:14.291 DEBUG:teuthology.orchestra.run.smithi042:mds.j> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.j.service 2024-03-12T04:12:14.334 DEBUG:teuthology.orchestra.run.smithi119:mds.b> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.b.service 2024-03-12T04:12:14.337 DEBUG:teuthology.orchestra.run.smithi119:mds.e> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.e.service 2024-03-12T04:12:14.380 DEBUG:teuthology.orchestra.run.smithi119:mds.h> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.h.service 2024-03-12T04:12:14.424 DEBUG:teuthology.orchestra.run.smithi119:mds.k> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.k.service 2024-03-12T04:12:14.467 DEBUG:teuthology.orchestra.run.smithi136:mds.c> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.c.service 2024-03-12T04:12:14.469 DEBUG:teuthology.orchestra.run.smithi136:mds.f> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.f.service 2024-03-12T04:12:14.513 DEBUG:teuthology.orchestra.run.smithi136:mds.i> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.i.service 2024-03-12T04:12:14.556 DEBUG:teuthology.orchestra.run.smithi136:mds.l> sudo journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.l.service 2024-03-12T04:12:14.600 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-12T04:12:14.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:12:14.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:14.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 -- 172.21.15.42:0/4132415396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601012b0 msgr2=0x7f6660101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:14.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 --2- 172.21.15.42:0/4132415396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601012b0 0x7f6660101730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6650009900 tx=0x7f665002bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:14.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 -- 172.21.15.42:0/4132415396 shutdown_connections 2024-03-12T04:12:14.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 --2- 172.21.15.42:0/4132415396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6660101c70 0x7f6660108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 --2- 172.21.15.42:0/4132415396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601012b0 0x7f6660101730 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 --2- 172.21.15.42:0/4132415396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66601000c0 0x7f66601004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 -- 172.21.15.42:0/4132415396 >> 172.21.15.42:0/4132415396 conn(0x7f66600fb630 msgr2=0x7f66600fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 -- 172.21.15.42:0/4132415396 shutdown_connections 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.717+0000 7f66663b5640 1 -- 172.21.15.42:0/4132415396 wait complete. 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 Processor -- start 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 -- start start 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601000c0 0x7f6660196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66601012b0 0x7f6660196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6660101c70 0x7f666019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666019db30 con 0x7f66601000c0 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666019dca0 con 0x7f6660101c70 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f66663b5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666019de60 con 0x7f66601012b0 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601000c0 0x7f6660196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601000c0 0x7f6660196040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39442/0 (socket says 172.21.15.42:39442) 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 -- 172.21.15.42:0/2371460591 learned_addr learned my addr 172.21.15.42:0/2371460591 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665f7fe640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66601012b0 0x7f6660196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:14.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f6664952640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6660101c70 0x7f666019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 -- 172.21.15.42:0/2371460591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66601012b0 msgr2=0x7f6660196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66601012b0 0x7f6660196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 -- 172.21.15.42:0/2371460591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6660101c70 msgr2=0x7f666019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6660101c70 0x7f666019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.718+0000 7f665ffff640 1 -- 172.21.15.42:0/2371460591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650009560 con 0x7f66601000c0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.719+0000 7f665ffff640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601000c0 0x7f6660196040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f664800d770 tx=0x7f664800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.719+0000 7f665d7fa640 1 -- 172.21.15.42:0/2371460591 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648009c30 con 0x7f66601000c0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.719+0000 7f665d7fa640 1 -- 172.21.15.42:0/2371460591 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66480044d0 con 0x7f66601000c0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.719+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666019e140 con 0x7f66601000c0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.719+0000 7f665d7fa640 1 -- 172.21.15.42:0/2371460591 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648003de0 con 0x7f66601000c0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.719+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666019e620 con 0x7f66601000c0 2024-03-12T04:12:14.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.720+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6660107400 con 0x7f66601000c0 2024-03-12T04:12:14.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.721+0000 7f665d7fa640 1 -- 172.21.15.42:0/2371460591 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6648003f40 con 0x7f66601000c0 2024-03-12T04:12:14.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.721+0000 7f665d7fa640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f664c0a1a30 0x7f664c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:14.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.722+0000 7f665f7fe640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f664c0a1a30 0x7f664c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:14.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.722+0000 7f665d7fa640 1 -- 172.21.15.42:0/2371460591 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66480cc820 con 0x7f66601000c0 2024-03-12T04:12:14.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.722+0000 7f665f7fe640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f664c0a1a30 0x7f664c0a3ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6650009230 tx=0x7f6650005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:14.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.726+0000 7f665d7fa640 1 -- 172.21.15.42:0/2371460591 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6648095f80 con 0x7f66601000c0 2024-03-12T04:12:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:12:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-03-12T04:12:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:14.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:14.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:14.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:14.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:14.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:14.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:14.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:14.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.908+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f666002ce40 con 0x7f66601000c0 2024-03-12T04:12:14.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.909+0000 7f665d7fa640 1 -- 172.21.15.42:0/2371460591 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18456 (secure 0 0 0) 0x7f6648099640 con 0x7f66601000c0 2024-03-12T04:12:14.912 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:14.912 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:02.144274+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:12:14.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f664c0a1a30 msgr2=0x7f664c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:14.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f664c0a1a30 0x7f664c0a3ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6650009230 tx=0x7f6650005fd0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601000c0 msgr2=0x7f6660196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601000c0 0x7f6660196040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f664800d770 tx=0x7f664800dc30 comp rx=0 tx=0).stop 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 shutdown_connections 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f664c0a1a30 0x7f664c0a3ef0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6660101c70 0x7f666019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66601012b0 0x7f6660196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 --2- 172.21.15.42:0/2371460591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66601000c0 0x7f6660196040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 >> 172.21.15.42:0/2371460591 conn(0x7f66600fb630 msgr2=0x7f66601045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 shutdown_connections 2024-03-12T04:12:14.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:14.911+0000 7f66663b5640 1 -- 172.21.15.42:0/2371460591 wait complete. 2024-03-12T04:12:14.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:12:14.925 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:12:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:12:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-03-12T04:12:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:14.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:14.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:14.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:14.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:14.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:14.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.028+0000 7fc057e15640 1 -- 172.21.15.42:0/3211936344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0500ffa70 msgr2=0x7fc0500ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.028+0000 7fc057e15640 1 --2- 172.21.15.42:0/3211936344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0500ffa70 0x7fc0500ffe90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc048009900 tx=0x7fc04802bcd0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.028+0000 7fc057e15640 1 -- 172.21.15.42:0/3211936344 shutdown_connections 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.028+0000 7fc057e15640 1 --2- 172.21.15.42:0/3211936344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc050100d70 0x7fc0501011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.028+0000 7fc057e15640 1 --2- 172.21.15.42:0/3211936344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0501003d0 0x7fc050100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.028+0000 7fc057e15640 1 --2- 172.21.15.42:0/3211936344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0500ffa70 0x7fc0500ffe90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.028+0000 7fc057e15640 1 -- 172.21.15.42:0/3211936344 >> 172.21.15.42:0/3211936344 conn(0x7fc0500fb610 msgr2=0x7fc0500fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 -- 172.21.15.42:0/3211936344 shutdown_connections 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 -- 172.21.15.42:0/3211936344 wait complete. 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 Processor -- start 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 -- start start 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0500ffa70 0x7fc0501993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0501003d0 0x7fc050199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050100d70 0x7fc0501a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501a0ec0 con 0x7fc050100d70 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501a1030 con 0x7fc0501003d0 2024-03-12T04:12:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc057e15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501a11f0 con 0x7fc0500ffa70 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc0563b2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050100d70 0x7fc0501a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc0553b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0501003d0 0x7fc050199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc055bb1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0500ffa70 0x7fc0501993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc0563b2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050100d70 0x7fc0501a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39452/0 (socket says 172.21.15.42:39452) 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.029+0000 7fc0563b2640 1 -- 172.21.15.42:0/1169144100 learned_addr learned my addr 172.21.15.42:0/1169144100 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc0563b2640 1 -- 172.21.15.42:0/1169144100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0500ffa70 msgr2=0x7fc0501993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc0563b2640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0500ffa70 0x7fc0501993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc0563b2640 1 -- 172.21.15.42:0/1169144100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0501003d0 msgr2=0x7fc050199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc0563b2640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0501003d0 0x7fc050199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc0563b2640 1 -- 172.21.15.42:0/1169144100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048009560 con 0x7fc050100d70 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc0563b2640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050100d70 0x7fc0501a0980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc04000cad0 tx=0x7fc04000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc04effd640 1 -- 172.21.15.42:0/1169144100 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04000ae20 con 0x7fc050100d70 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc04effd640 1 -- 172.21.15.42:0/1169144100 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc040005020 con 0x7fc050100d70 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0501a14d0 con 0x7fc050100d70 2024-03-12T04:12:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc04effd640 1 -- 172.21.15.42:0/1169144100 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04000f690 con 0x7fc050100d70 2024-03-12T04:12:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.030+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0501a19e0 con 0x7fc050100d70 2024-03-12T04:12:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.031+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05010aaa0 con 0x7fc050100d70 2024-03-12T04:12:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.032+0000 7fc04effd640 1 -- 172.21.15.42:0/1169144100 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc040005190 con 0x7fc050100d70 2024-03-12T04:12:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.033+0000 7fc04effd640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc0380a1ab0 0x7fc0380a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.033+0000 7fc055bb1640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc0380a1ab0 0x7fc0380a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.033+0000 7fc04effd640 1 -- 172.21.15.42:0/1169144100 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc0400cceb0 con 0x7fc050100d70 2024-03-12T04:12:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.033+0000 7fc055bb1640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc0380a1ab0 0x7fc0380a3f70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc048009230 tx=0x7fc048005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:15.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.037+0000 7fc04effd640 1 -- 172.21.15.42:0/1169144100 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc040096610 con 0x7fc050100d70 2024-03-12T04:12:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.218+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc05002ce40 con 0x7fc050100d70 2024-03-12T04:12:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.218+0000 7fc04effd640 1 -- 172.21.15.42:0/1169144100 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc040099cd0 con 0x7fc050100d70 2024-03-12T04:12:15.221 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc0380a1ab0 msgr2=0x7fc0380a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc0380a1ab0 0x7fc0380a3f70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc048009230 tx=0x7fc048005fd0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050100d70 msgr2=0x7fc0501a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050100d70 0x7fc0501a0980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc04000cad0 tx=0x7fc04000cf90 comp rx=0 tx=0).stop 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 shutdown_connections 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050100d70 0x7fc0501a0980 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc0380a1ab0 0x7fc0380a3f70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0501003d0 0x7fc050199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 --2- 172.21.15.42:0/1169144100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0500ffa70 0x7fc0501993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 >> 172.21.15.42:0/1169144100 conn(0x7fc0500fb610 msgr2=0x7fc0500fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 shutdown_connections 2024-03-12T04:12:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.220+0000 7fc057e15640 1 -- 172.21.15.42:0/1169144100 wait complete. 2024-03-12T04:12:15.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:15.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 -- 172.21.15.42:0/3308504320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102d50 msgr2=0x7f41801095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3308504320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102d50 0x7f41801095e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4178009ca0 tx=0x7f417802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 -- 172.21.15.42:0/3308504320 shutdown_connections 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3308504320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102d50 0x7f41801095e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3308504320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4180102390 0x7f4180102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3308504320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41801011a0 0x7f41801015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 -- 172.21.15.42:0/3308504320 >> 172.21.15.42:0/3308504320 conn(0x7f41800fc730 msgr2=0x7f41800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 -- 172.21.15.42:0/3308504320 shutdown_connections 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.333+0000 7f4187fd7640 1 -- 172.21.15.42:0/3308504320 wait complete. 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 Processor -- start 2024-03-12T04:12:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 -- start start 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41801011a0 0x7f41801970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102390 0x7f4180197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4180102d50 0x7f418019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019ebd0 con 0x7f4180102390 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019ed40 con 0x7f41801011a0 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019ef00 con 0x7f4180102d50 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102390 0x7f4180197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102390 0x7f4180197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39480/0 (socket says 172.21.15.42:39480) 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 -- 172.21.15.42:0/3548537931 learned_addr learned my addr 172.21.15.42:0/3548537931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185d73640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41801011a0 0x7f41801970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4186574640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4180102d50 0x7f418019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 -- 172.21.15.42:0/3548537931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4180102d50 msgr2=0x7f418019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4180102d50 0x7f418019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 -- 172.21.15.42:0/3548537931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41801011a0 msgr2=0x7f41801970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41801011a0 0x7f41801970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 -- 172.21.15.42:0/3548537931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178009900 con 0x7f4180102390 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185d73640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41801011a0 0x7f41801970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4185572640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102390 0x7f4180197620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f417000d950 tx=0x7f417000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4176ffd640 1 -- 172.21.15.42:0/3548537931 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170009a30 con 0x7f4180102390 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4176ffd640 1 -- 172.21.15.42:0/3548537931 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41700044d0 con 0x7f4180102390 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4176ffd640 1 -- 172.21.15.42:0/3548537931 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700047a0 con 0x7f4180102390 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.334+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418019f1e0 con 0x7f4180102390 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.335+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418019f740 con 0x7f4180102390 2024-03-12T04:12:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.335+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4180108540 con 0x7f4180102390 2024-03-12T04:12:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.336+0000 7f4176ffd640 1 -- 172.21.15.42:0/3548537931 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4170009b90 con 0x7f4180102390 2024-03-12T04:12:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.337+0000 7f4176ffd640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f416c0a1ab0 0x7f416c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.337+0000 7f4185d73640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f416c0a1ab0 0x7f416c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.337+0000 7f4176ffd640 1 -- 172.21.15.42:0/3548537931 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41700cd070 con 0x7f4180102390 2024-03-12T04:12:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.337+0000 7f4185d73640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f416c0a1ab0 0x7f416c0a3f70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f417c0058c0 tx=0x7f417c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:15.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.341+0000 7f4176ffd640 1 -- 172.21.15.42:0/3548537931 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4170096700 con 0x7f4180102390 2024-03-12T04:12:15.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.561+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f418010ca60 con 0x7f4180102390 2024-03-12T04:12:15.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.561+0000 7f4176ffd640 1 -- 172.21.15.42:0/3548537931 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f4170017090 con 0x7f4180102390 2024-03-12T04:12:15.564 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:15.564 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-12T04:12:15.564 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 1 2024-03-12T04:12:15.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f416c0a1ab0 msgr2=0x7f416c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f416c0a1ab0 0x7f416c0a3f70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f417c0058c0 tx=0x7f417c005850 comp rx=0 tx=0).stop 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102390 msgr2=0x7f4180197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102390 0x7f4180197620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f417000d950 tx=0x7f417000de10 comp rx=0 tx=0).stop 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 shutdown_connections 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4180102d50 0x7f418019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4180102390 0x7f4180197620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f416c0a1ab0 0x7f416c0a3f70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 --2- 172.21.15.42:0/3548537931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41801011a0 0x7f41801970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 >> 172.21.15.42:0/3548537931 conn(0x7f41800fc730 msgr2=0x7f4180105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 shutdown_connections 2024-03-12T04:12:15.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:15.563+0000 7f4187fd7640 1 -- 172.21.15.42:0/3548537931 wait complete. 2024-03-12T04:12:15.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:15 smithi136 ceph-mon[35571]: Deploying daemon mds.l on smithi136 2024-03-12T04:12:15.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:15 smithi136 ceph-mon[35571]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:15.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:15 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2371460591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:15.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:15 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1169144100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:15.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:15 smithi042 ceph-mon[32260]: Deploying daemon mds.l on smithi136 2024-03-12T04:12:15.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:15 smithi042 ceph-mon[32260]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:15.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:15 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2371460591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:15.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:15 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1169144100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:15.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:15 smithi119 ceph-mon[36119]: Deploying daemon mds.l on smithi136 2024-03-12T04:12:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:15 smithi119 ceph-mon[36119]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:15 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2371460591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:15.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:15 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1169144100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:16 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3548537931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:16.708 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:16 smithi136 systemd[1]: Starting Ceph mds.l for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:16.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:16 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3548537931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:16.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:16 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3548537931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:17.014 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:16 smithi136 podman[65176]: 2024-03-12 04:12:16.66252348 +0000 UTC m=+0.017620128 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:17.014 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:16 smithi136 podman[65176]: 2024-03-12 04:12:16.877439968 +0000 UTC m=+0.232536611 container create d76170f81d9f56f205d3578e75dfc9821eed980b1847fb27b8f470dea8cc3008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:12:17.457 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:17 smithi136 podman[65176]: 2024-03-12 04:12:17.138783196 +0000 UTC m=+0.493879855 container init d76170f81d9f56f205d3578e75dfc9821eed980b1847fb27b8f470dea8cc3008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:12:17.458 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:17 smithi136 podman[65176]: 2024-03-12 04:12:17.14136523 +0000 UTC m=+0.496461886 container start d76170f81d9f56f205d3578e75dfc9821eed980b1847fb27b8f470dea8cc3008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-12T04:12:17.458 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mds[65195]: -- 172.21.15.136:0/1873426630 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55db086741a0 con 0x55db0864a800 2024-03-12T04:12:17.458 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l[65191]: starting mds.l at 2024-03-12T04:12:17.458 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:17 smithi136 bash[65176]: d76170f81d9f56f205d3578e75dfc9821eed980b1847fb27b8f470dea8cc3008 2024-03-12T04:12:17.458 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:12:17 smithi136 systemd[1]: Started Ceph mds.l for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:17.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 -- 172.21.15.42:0/3828439893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f01045d0 msgr2=0x7f18f01069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 --2- 172.21.15.42:0/3828439893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f01045d0 0x7f18f01069c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009900 tx=0x7f18d802bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 -- 172.21.15.42:0/3828439893 shutdown_connections 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 --2- 172.21.15.42:0/3828439893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f0106f00 0x7f18f01092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 --2- 172.21.15.42:0/3828439893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f01045d0 0x7f18f01069c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 --2- 172.21.15.42:0/3828439893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18f0101ca0 0x7f18f0104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 -- 172.21.15.42:0/3828439893 >> 172.21.15.42:0/3828439893 conn(0x7f18f00fb610 msgr2=0x7f18f00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.689+0000 7f18f59b6640 1 -- 172.21.15.42:0/3828439893 shutdown_connections 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 -- 172.21.15.42:0/3828439893 wait complete. 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 Processor -- start 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 -- start start 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f0101ca0 0x7f18f01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f01045d0 0x7f18f01976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18f0106f00 0x7f18f019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f019eca0 con 0x7f18f0101ca0 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f019ee10 con 0x7f18f0106f00 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.690+0000 7f18f59b6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f019efd0 con 0x7f18f01045d0 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f0101ca0 0x7f18f01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f0101ca0 0x7f18f01971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39496/0 (socket says 172.21.15.42:39496) 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 -- 172.21.15.42:0/1141840941 learned_addr learned my addr 172.21.15.42:0/1141840941 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18ef7fe640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18f0106f00 0x7f18f019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18ee7fc640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f01045d0 0x7f18f01976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 -- 172.21.15.42:0/1141840941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f01045d0 msgr2=0x7f18f01976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f01045d0 0x7f18f01976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 -- 172.21.15.42:0/1141840941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18f0106f00 msgr2=0x7f18f019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18f0106f00 0x7f18f019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 -- 172.21.15.42:0/1141840941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8009490 con 0x7f18f0101ca0 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18eeffd640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f0101ca0 0x7f18f01971b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18e800ea60 tx=0x7f18e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18f49b4640 1 -- 172.21.15.42:0/1141840941 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800cde0 con 0x7f18f0101ca0 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d8009560 con 0x7f18f0101ca0 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18f49b4640 1 -- 172.21.15.42:0/1141840941 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f18e80044d0 con 0x7f18f0101ca0 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18f49b4640 1 -- 172.21.15.42:0/1141840941 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8010690 con 0x7f18f0101ca0 2024-03-12T04:12:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.691+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f019f630 con 0x7f18f0101ca0 2024-03-12T04:12:17.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.692+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f00fd100 con 0x7f18f0101ca0 2024-03-12T04:12:17.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.694+0000 7f18f49b4640 1 -- 172.21.15.42:0/1141840941 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f18e8003e40 con 0x7f18f0101ca0 2024-03-12T04:12:17.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.695+0000 7f18f49b4640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:17.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.695+0000 7f18ee7fc640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:17.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.695+0000 7f18f49b4640 1 -- 172.21.15.42:0/1141840941 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18e8014070 con 0x7f18f0101ca0 2024-03-12T04:12:17.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.695+0000 7f18ee7fc640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18d802c7d0 tx=0x7f18d8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:17.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.698+0000 7f18f49b4640 1 -- 172.21.15.42:0/1141840941 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18e8096cc0 con 0x7f18f0101ca0 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6832/3418533459,v1:172.21.15.136:6833/3418533459] up:boot 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: fsmap 1 up:standby 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:17.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:17.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:17 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.918+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18f010ca20 con 0x7f18f0101ca0 2024-03-12T04:12:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.918+0000 7f18f49b4640 1 -- 172.21.15.42:0/1141840941 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f18e809a380 con 0x7f18f0101ca0 2024-03-12T04:12:17.921 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:17.922 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-12T04:12:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-03-12T04:12:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f18dc0a1a30 msgr2=0x7f18dc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18d802c7d0 tx=0x7f18d8005fd0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f0101ca0 msgr2=0x7f18f01971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f0101ca0 0x7f18f01971b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18e800ea60 tx=0x7f18e800ef20 comp rx=0 tx=0).stop 2024-03-12T04:12:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 shutdown_connections 2024-03-12T04:12:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18f0106f00 0x7f18f019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f01045d0 0x7f18f01976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 --2- 172.21.15.42:0/1141840941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18f0101ca0 0x7f18f01971b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 >> 172.21.15.42:0/1141840941 conn(0x7f18f00fb610 msgr2=0x7f18f0103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:17.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 shutdown_connections 2024-03-12T04:12:17.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:17.920+0000 7f18f59b6640 1 -- 172.21.15.42:0/1141840941 wait complete. 2024-03-12T04:12:17.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6832/3418533459,v1:172.21.15.136:6833/3418533459] up:boot 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: fsmap 1 up:standby 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:17.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:17 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:17.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:17.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6832/3418533459,v1:172.21.15.136:6833/3418533459] up:boot 2024-03-12T04:12:17.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: fsmap 1 up:standby 2024-03-12T04:12:17.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:12:17.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:17.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:17.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:17.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:17 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:18.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:18 smithi042 ceph-mon[32260]: Deploying daemon mds.k on smithi119 2024-03-12T04:12:18.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:18 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1141840941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:18.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:18 smithi119 ceph-mon[36119]: Deploying daemon mds.k on smithi119 2024-03-12T04:12:18.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:18 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1141840941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:18.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:18 smithi136 ceph-mon[35571]: Deploying daemon mds.k on smithi119 2024-03-12T04:12:18.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:18 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1141840941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:19.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:19 smithi119 ceph-mon[36119]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:19.696 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:19 smithi119 systemd[1]: Starting Ceph mds.k for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:19.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:19 smithi042 ceph-mon[32260]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:19.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:19.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:19 smithi136 ceph-mon[35571]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:20.042 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:19 smithi119 podman[65821]: 2024-03-12 04:12:19.658865286 +0000 UTC m=+0.018188089 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:20.042 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:19 smithi119 podman[65821]: 2024-03-12 04:12:19.796045996 +0000 UTC m=+0.155368852 container create a47debd23377d0e4fffb38dc819ca8354c238dbbf0a0c319c10815b701c1714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-12T04:12:20.042 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:20 smithi119 podman[65821]: 2024-03-12 04:12:20.040354966 +0000 UTC m=+0.399677760 container init a47debd23377d0e4fffb38dc819ca8354c238dbbf0a0c319c10815b701c1714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 -- 172.21.15.42:0/3493384377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 msgr2=0x7f64b8101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/3493384377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8101720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009900 tx=0x7f64ac02bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 -- 172.21.15.42:0/3493384377 shutdown_connections 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/3493384377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b8101c60 0x7f64b81084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/3493384377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8101720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/3493384377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64b81000b0 0x7f64b81004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 -- 172.21.15.42:0/3493384377 >> 172.21.15.42:0/3493384377 conn(0x7f64b80fb660 msgr2=0x7f64b80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 -- 172.21.15.42:0/3493384377 shutdown_connections 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.047+0000 7f64bf8c0640 1 -- 172.21.15.42:0/3493384377 wait complete. 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 Processor -- start 2024-03-12T04:12:20.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 -- start start 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64b81000b0 0x7f64b8197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b8101c60 0x7f64b819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b819ed10 con 0x7f64b81012a0 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b819ee80 con 0x7f64b8101c60 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bf8c0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b819f040 con 0x7f64b81000b0 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39526/0 (socket says 172.21.15.42:39526) 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 -- 172.21.15.42:0/2578158257 learned_addr learned my addr 172.21.15.42:0/2578158257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bde5d640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b8101c60 0x7f64b819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bd65c640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64b81000b0 0x7f64b8197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 -- 172.21.15.42:0/2578158257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64b81000b0 msgr2=0x7f64b8197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64b81000b0 0x7f64b8197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 -- 172.21.15.42:0/2578158257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b8101c60 msgr2=0x7f64b819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b8101c60 0x7f64b819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bce5b640 1 -- 172.21.15.42:0/2578158257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac009560 con 0x7f64b81012a0 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bd65c640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64b81000b0 0x7f64b8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.048+0000 7f64bde5d640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b8101c60 0x7f64b819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.049+0000 7f64bce5b640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8197760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64ac036040 tx=0x7f64ac004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.049+0000 7f64b67fc640 1 -- 172.21.15.42:0/2578158257 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0035f0 con 0x7f64b81012a0 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.049+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b819f2c0 con 0x7f64b81012a0 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.049+0000 7f64b67fc640 1 -- 172.21.15.42:0/2578158257 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64ac02de90 con 0x7f64b81012a0 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.049+0000 7f64b67fc640 1 -- 172.21.15.42:0/2578158257 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac005490 con 0x7f64b81012a0 2024-03-12T04:12:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.049+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b819f8a0 con 0x7f64b81012a0 2024-03-12T04:12:20.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.050+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b810b9d0 con 0x7f64b81012a0 2024-03-12T04:12:20.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.051+0000 7f64b67fc640 1 -- 172.21.15.42:0/2578158257 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f64ac003750 con 0x7f64b81012a0 2024-03-12T04:12:20.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.051+0000 7f64b67fc640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f64a00a1a30 0x7f64a00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:20.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.052+0000 7f64bd65c640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f64a00a1a30 0x7f64a00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:20.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.052+0000 7f64b67fc640 1 -- 172.21.15.42:0/2578158257 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f64ac0efa00 con 0x7f64b81012a0 2024-03-12T04:12:20.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.052+0000 7f64bd65c640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f64a00a1a30 0x7f64a00a3ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64b8101100 tx=0x7f64a8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:20.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.055+0000 7f64b67fc640 1 -- 172.21.15.42:0/2578158257 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64ac0b9010 con 0x7f64b81012a0 2024-03-12T04:12:20.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.285+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64b819f450 con 0x7f64b81012a0 2024-03-12T04:12:20.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.285+0000 7f64b67fc640 1 -- 172.21.15.42:0/2578158257 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f64b819f450 con 0x7f64b81012a0 2024-03-12T04:12:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:20.288 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-12T04:12:20.288 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-03-12T04:12:20.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f64a00a1a30 msgr2=0x7f64a00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f64a00a1a30 0x7f64a00a3ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64b8101100 tx=0x7f64a8009570 comp rx=0 tx=0).stop 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 msgr2=0x7f64b8197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8197760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64ac036040 tx=0x7f64ac004140 comp rx=0 tx=0).stop 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 shutdown_connections 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f64a00a1a30 0x7f64a00a3ef0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b8101c60 0x7f64b819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b81012a0 0x7f64b8197760 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 --2- 172.21.15.42:0/2578158257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64b81000b0 0x7f64b8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 >> 172.21.15.42:0/2578158257 conn(0x7f64b80fb660 msgr2=0x7f64b80fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 shutdown_connections 2024-03-12T04:12:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:20.288+0000 7f64bf8c0640 1 -- 172.21.15.42:0/2578158257 wait complete. 2024-03-12T04:12:20.446 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:20 smithi119 podman[65821]: 2024-03-12 04:12:20.043075526 +0000 UTC m=+0.402398322 container start a47debd23377d0e4fffb38dc819ca8354c238dbbf0a0c319c10815b701c1714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-12T04:12:20.446 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:20 smithi119 ceph-mds[65840]: -- 172.21.15.119:0/3307206707 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5624be76a1a0 con 0x5624be740800 2024-03-12T04:12:20.446 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k[65836]: starting mds.k at 2024-03-12T04:12:20.446 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:20 smithi119 bash[65821]: a47debd23377d0e4fffb38dc819ca8354c238dbbf0a0c319c10815b701c1714c 2024-03-12T04:12:20.446 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:12:20 smithi119 systemd[1]: Started Ceph mds.k for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: Deploying daemon mds.j on smithi042 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2578158257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:21.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6832/2928023683,v1:172.21.15.119:6833/2928023683] up:boot 2024-03-12T04:12:21.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: fsmap 2 up:standby 2024-03-12T04:12:21.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:12:21.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:21 smithi119 ceph-mon[36119]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:21.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:21.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: Deploying daemon mds.j on smithi042 2024-03-12T04:12:21.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2578158257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:21.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6832/2928023683,v1:172.21.15.119:6833/2928023683] up:boot 2024-03-12T04:12:21.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: fsmap 2 up:standby 2024-03-12T04:12:21.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:12:21.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:21 smithi042 ceph-mon[32260]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:21.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:21.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: Deploying daemon mds.j on smithi042 2024-03-12T04:12:21.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2578158257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:21.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6832/2928023683,v1:172.21.15.119:6833/2928023683] up:boot 2024-03-12T04:12:21.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: fsmap 2 up:standby 2024-03-12T04:12:21.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:12:21.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:21 smithi136 ceph-mon[35571]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:22.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 -- 172.21.15.42:0/293192380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 msgr2=0x7f47a41084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 --2- 172.21.15.42:0/293192380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a41084e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f479c009ca0 tx=0x7f479c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 -- 172.21.15.42:0/293192380 shutdown_connections 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 --2- 172.21.15.42:0/293192380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a41084e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 --2- 172.21.15.42:0/293192380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a4101290 0x7f47a4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 --2- 172.21.15.42:0/293192380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47a41000a0 0x7f47a41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 -- 172.21.15.42:0/293192380 >> 172.21.15.42:0/293192380 conn(0x7f47a40fb610 msgr2=0x7f47a40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:22.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 -- 172.21.15.42:0/293192380 shutdown_connections 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.416+0000 7f47a9a81640 1 -- 172.21.15.42:0/293192380 wait complete. 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 Processor -- start 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 -- start start 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47a41000a0 0x7f47a4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a4101290 0x7f47a4197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a419ec40 con 0x7f47a4101c50 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a419edb0 con 0x7f47a41000a0 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a9a81640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a419ef70 con 0x7f47a4101290 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47a41000a0 0x7f47a4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:22.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45966/0 (socket says 172.21.15.42:45966) 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.417+0000 7f47a3fff640 1 -- 172.21.15.42:0/2671078758 learned_addr learned my addr 172.21.15.42:0/2671078758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a3fff640 1 -- 172.21.15.42:0/2671078758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a4101290 msgr2=0x7f47a4197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a2ffd640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a4101290 0x7f47a4197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a3fff640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a4101290 0x7f47a4197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a3fff640 1 -- 172.21.15.42:0/2671078758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47a41000a0 msgr2=0x7f47a4197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a3fff640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47a41000a0 0x7f47a4197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a3fff640 1 -- 172.21.15.42:0/2671078758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009900 con 0x7f47a4101c50 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a37fe640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47a41000a0 0x7f47a4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a3fff640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a419e700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f479c037040 tx=0x7f479c032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a0ff9640 1 -- 172.21.15.42:0/2671078758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c034e70 con 0x7f47a4101c50 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a419f1f0 con 0x7f47a4101c50 2024-03-12T04:12:22.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a0ff9640 1 -- 172.21.15.42:0/2671078758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f479c003e20 con 0x7f47a4101c50 2024-03-12T04:12:22.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a419f6f0 con 0x7f47a4101c50 2024-03-12T04:12:22.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.418+0000 7f47a0ff9640 1 -- 172.21.15.42:0/2671078758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c03d3f0 con 0x7f47a4101c50 2024-03-12T04:12:22.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.419+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a410b960 con 0x7f47a4101c50 2024-03-12T04:12:22.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.421+0000 7f47a0ff9640 1 -- 172.21.15.42:0/2671078758 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f479c03b040 con 0x7f47a4101c50 2024-03-12T04:12:22.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.422+0000 7f47a0ff9640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f47900a1a30 0x7f47900a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:22.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.422+0000 7f47a37fe640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f47900a1a30 0x7f47900a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:22.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.422+0000 7f47a0ff9640 1 -- 172.21.15.42:0/2671078758 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f479c0efae0 con 0x7f47a4101c50 2024-03-12T04:12:22.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.422+0000 7f47a37fe640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f47900a1a30 0x7f47900a3ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47a41010f0 tx=0x7f479400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:22.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.424+0000 7f47a0ff9640 1 -- 172.21.15.42:0/2671078758 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f479c0b9190 con 0x7f47a4101c50 2024-03-12T04:12:22.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.662+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47a410baf0 con 0x7f47a4101c50 2024-03-12T04:12:22.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.663+0000 7f47a0ff9640 1 -- 172.21.15.42:0/2671078758 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f479c0bc850 con 0x7f47a4101c50 2024-03-12T04:12:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:22.666 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-12T04:12:22.666 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2024-03-12T04:12:22.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.665+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f47900a1a30 msgr2=0x7f47900a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:22.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.665+0000 7f47a9a81640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f47900a1a30 0x7f47900a3ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47a41010f0 tx=0x7f479400b390 comp rx=0 tx=0).stop 2024-03-12T04:12:22.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.665+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 msgr2=0x7f47a419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:22.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.665+0000 7f47a9a81640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a419e700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f479c037040 tx=0x7f479c032980 comp rx=0 tx=0).stop 2024-03-12T04:12:22.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 shutdown_connections 2024-03-12T04:12:22.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a4101c50 0x7f47a419e700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a4101290 0x7f47a4197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f47900a1a30 0x7f47900a3ef0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 --2- 172.21.15.42:0/2671078758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47a41000a0 0x7f47a4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:22.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 >> 172.21.15.42:0/2671078758 conn(0x7f47a40fb610 msgr2=0x7f47a40682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:22.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 shutdown_connections 2024-03-12T04:12:22.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:22.666+0000 7f47a9a81640 1 -- 172.21.15.42:0/2671078758 wait complete. 2024-03-12T04:12:22.678 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:22 smithi042 systemd[1]: Starting Ceph mds.j for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:23.050 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2671078758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:23.051 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:22 smithi042 podman[69875]: 2024-03-12 04:12:22.659632541 +0000 UTC m=+0.019126331 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:23.051 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:22 smithi042 podman[69875]: 2024-03-12 04:12:22.77994213 +0000 UTC m=+0.139435907 container create 2e78f253e8940dca465192e29551affd1283e172de4b54b34e4fd019689668e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:12:23.051 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:23 smithi042 podman[69875]: 2024-03-12 04:12:23.049103815 +0000 UTC m=+0.408597605 container init 2e78f253e8940dca465192e29551affd1283e172de4b54b34e4fd019689668e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-12T04:12:23.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:22 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2671078758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:23.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:22 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2671078758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:23.317 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:23 smithi042 podman[69875]: 2024-03-12 04:12:23.052058079 +0000 UTC m=+0.411551859 container start 2e78f253e8940dca465192e29551affd1283e172de4b54b34e4fd019689668e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux ) 2024-03-12T04:12:23.317 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mds[69896]: -- 172.21.15.42:0/1609516806 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5596bf3b21a0 con 0x5596bf388c00 2024-03-12T04:12:23.317 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j[69892]: starting mds.j at 2024-03-12T04:12:23.317 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:23 smithi042 bash[69875]: 2e78f253e8940dca465192e29551affd1283e172de4b54b34e4fd019689668e1 2024-03-12T04:12:23.317 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:12:23 smithi042 systemd[1]: Started Ceph mds.j for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:23.980 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:23 smithi136 ceph-mon[35571]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:23.980 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:23.980 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:23.981 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:23.981 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:23.981 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:23.981 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:23 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:24.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mon[32260]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:24.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:24.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:24.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:24.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:24.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:24.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:23 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:23 smithi119 ceph-mon[36119]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:24.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:23 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:24.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:24.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.792+0000 7fe400e15640 1 -- 172.21.15.42:0/577931009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 msgr2=0x7fe3fc1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:24.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.792+0000 7fe400e15640 1 --2- 172.21.15.42:0/577931009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1084e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009ca0 tx=0x7fe3f402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:24.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 -- 172.21.15.42:0/577931009 shutdown_connections 2024-03-12T04:12:24.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 --2- 172.21.15.42:0/577931009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1084e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 --2- 172.21.15.42:0/577931009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3fc101290 0x7fe3fc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 --2- 172.21.15.42:0/577931009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3fc1000a0 0x7fe3fc1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 -- 172.21.15.42:0/577931009 >> 172.21.15.42:0/577931009 conn(0x7fe3fc0fb610 msgr2=0x7fe3fc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 -- 172.21.15.42:0/577931009 shutdown_connections 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 -- 172.21.15.42:0/577931009 wait complete. 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 Processor -- start 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.793+0000 7fe400e15640 1 -- start start 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3fc1000a0 0x7fe3fc10a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3fc101290 0x7fe3fc108e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc109b00 con 0x7fe3fc101c50 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc109c70 con 0x7fe3fc101290 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc1a11d0 con 0x7fe3fc1000a0 2024-03-12T04:12:24.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46030/0 (socket says 172.21.15.42:46030) 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 -- 172.21.15.42:0/3234841230 learned_addr learned my addr 172.21.15.42:0/3234841230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fa59c640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3fc1000a0 0x7fe3fc10a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3f9d9b640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3fc101290 0x7fe3fc108e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 -- 172.21.15.42:0/3234841230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3fc1000a0 msgr2=0x7fe3fc10a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3fc1000a0 0x7fe3fc10a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 -- 172.21.15.42:0/3234841230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3fc101290 msgr2=0x7fe3fc108e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3fc101290 0x7fe3fc108e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 -- 172.21.15.42:0/3234841230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f4009900 con 0x7fe3fc101c50 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3f9d9b640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3fc101290 0x7fe3fc108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3fad9d640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1093a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4033000 tx=0x7fe3f4032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3eb7fe640 1 -- 172.21.15.42:0/3234841230 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4034e70 con 0x7fe3fc101c50 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3eb7fe640 1 -- 172.21.15.42:0/3234841230 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe3f4003e20 con 0x7fe3fc101c50 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc1a1420 con 0x7fe3fc101c50 2024-03-12T04:12:24.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe3eb7fe640 1 -- 172.21.15.42:0/3234841230 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f403d3f0 con 0x7fe3fc101c50 2024-03-12T04:12:24.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.794+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc1a1950 con 0x7fe3fc101c50 2024-03-12T04:12:24.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.795+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc107440 con 0x7fe3fc101c50 2024-03-12T04:12:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.796+0000 7fe3eb7fe640 1 -- 172.21.15.42:0/3234841230 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe3f403b040 con 0x7fe3fc101c50 2024-03-12T04:12:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.797+0000 7fe3eb7fe640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe3e40a1a30 0x7fe3e40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.797+0000 7fe3fa59c640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe3e40a1a30 0x7fe3e40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.797+0000 7fe3eb7fe640 1 -- 172.21.15.42:0/3234841230 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3f40efa30 con 0x7fe3fc101c50 2024-03-12T04:12:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.797+0000 7fe3fa59c640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe3e40a1a30 0x7fe3e40a3ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec005fd0 tx=0x7fe3ec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:24.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:24.801+0000 7fe3eb7fe640 1 -- 172.21.15.42:0/3234841230 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3f40b90e0 con 0x7fe3fc101c50 2024-03-12T04:12:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:24 smithi136 ceph-mon[35571]: Deploying daemon mds.d on smithi042 2024-03-12T04:12:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:24 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.42:6834/43691567,v1:172.21.15.42:6835/43691567] up:boot 2024-03-12T04:12:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:24 smithi136 ceph-mon[35571]: fsmap 3 up:standby 2024-03-12T04:12:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:24 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T04:12:25.009 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:24 smithi042 ceph-mon[32260]: Deploying daemon mds.d on smithi042 2024-03-12T04:12:25.009 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:24 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.42:6834/43691567,v1:172.21.15.42:6835/43691567] up:boot 2024-03-12T04:12:25.009 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:24 smithi042 ceph-mon[32260]: fsmap 3 up:standby 2024-03-12T04:12:25.009 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:24 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T04:12:25.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.023+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3fc02ce40 con 0x7fe3fc101c50 2024-03-12T04:12:25.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.023+0000 7fe3eb7fe640 1 -- 172.21.15.42:0/3234841230 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7fe3f40bc7a0 con 0x7fe3fc101c50 2024-03-12T04:12:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:25.026 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-12T04:12:25.027 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 4 2024-03-12T04:12:25.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe3e40a1a30 msgr2=0x7fe3e40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:25.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe3e40a1a30 0x7fe3e40a3ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec005fd0 tx=0x7fe3ec00b390 comp rx=0 tx=0).stop 2024-03-12T04:12:25.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 msgr2=0x7fe3fc1093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:25.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1093a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4033000 tx=0x7fe3f4032980 comp rx=0 tx=0).stop 2024-03-12T04:12:25.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 shutdown_connections 2024-03-12T04:12:25.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3fc101c50 0x7fe3fc1093a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:25.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe3e40a1a30 0x7fe3e40a3ef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:25.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3fc101290 0x7fe3fc108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:25.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 --2- 172.21.15.42:0/3234841230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3fc1000a0 0x7fe3fc10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:25.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 >> 172.21.15.42:0/3234841230 conn(0x7fe3fc0fb610 msgr2=0x7fe3fc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:25.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.025+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 shutdown_connections 2024-03-12T04:12:25.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:25.026+0000 7fe400e15640 1 -- 172.21.15.42:0/3234841230 wait complete. 2024-03-12T04:12:25.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:24 smithi119 ceph-mon[36119]: Deploying daemon mds.d on smithi042 2024-03-12T04:12:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:24 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.42:6834/43691567,v1:172.21.15.42:6835/43691567] up:boot 2024-03-12T04:12:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:24 smithi119 ceph-mon[36119]: fsmap 3 up:standby 2024-03-12T04:12:25.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:24 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T04:12:25.566 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:25 smithi042 systemd[1]: Starting Ceph mds.d for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:25.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:25 smithi042 ceph-mon[32260]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:25.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:25 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3234841230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:25.956 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:25 smithi042 podman[70231]: 2024-03-12 04:12:25.593090722 +0000 UTC m=+0.018398912 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:25.956 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:25 smithi042 podman[70231]: 2024-03-12 04:12:25.718833044 +0000 UTC m=+0.144141269 container create ca93b1335576c2e0f00f382c69dd42b54cd0fbf10de93a2994f57122490b1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-12T04:12:25.957 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:25 smithi042 podman[70231]: 2024-03-12 04:12:25.95483509 +0000 UTC m=+0.380143280 container init ca93b1335576c2e0f00f382c69dd42b54cd0fbf10de93a2994f57122490b1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-12T04:12:26.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:25 smithi119 ceph-mon[36119]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:26.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:25 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3234841230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:25 smithi136 ceph-mon[35571]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:26.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:25 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3234841230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:26.316 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:25 smithi042 podman[70231]: 2024-03-12 04:12:25.957747179 +0000 UTC m=+0.383055374 container start ca93b1335576c2e0f00f382c69dd42b54cd0fbf10de93a2994f57122490b1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=) 2024-03-12T04:12:26.316 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:25 smithi042 ceph-mds[70250]: -- 172.21.15.42:0/1533053240 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562f4c9861a0 con 0x562f4c95d000 2024-03-12T04:12:26.316 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:25 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d[70246]: starting mds.d at 2024-03-12T04:12:26.316 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:26 smithi042 bash[70231]: ca93b1335576c2e0f00f382c69dd42b54cd0fbf10de93a2994f57122490b1cf9 2024-03-12T04:12:26.316 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:12:26 smithi042 systemd[1]: Started Ceph mds.d for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:27.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: Deploying daemon mds.e on smithi119 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.42:6836/3380811405,v1:172.21.15.42:6837/3380811405] up:boot 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: fsmap 4 up:standby 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T04:12:27.135 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:27 smithi136 ceph-mon[35571]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 -- 172.21.15.42:0/3197413814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc101c50 msgr2=0x7fb0fc1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 --2- 172.21.15.42:0/3197413814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc101c50 0x7fb0fc1084e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009ca0 tx=0x7fb0f402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 -- 172.21.15.42:0/3197413814 shutdown_connections 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 --2- 172.21.15.42:0/3197413814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc101c50 0x7fb0fc1084e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 --2- 172.21.15.42:0/3197413814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0fc101290 0x7fb0fc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 --2- 172.21.15.42:0/3197413814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc1000a0 0x7fb0fc1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 -- 172.21.15.42:0/3197413814 >> 172.21.15.42:0/3197413814 conn(0x7fb0fc0fb610 msgr2=0x7fb0fc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 -- 172.21.15.42:0/3197413814 shutdown_connections 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.157+0000 7fb101040640 1 -- 172.21.15.42:0/3197413814 wait complete. 2024-03-12T04:12:27.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 Processor -- start 2024-03-12T04:12:27.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 -- start start 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc1000a0 0x7fb0fc068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0fc101290 0x7fb0fc19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc101c50 0x7fb0fc19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19ec70 con 0x7fb0fc1000a0 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19ede0 con 0x7fb0fc101c50 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb101040640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19efa0 con 0x7fb0fc101290 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc1000a0 0x7fb0fc068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc1000a0 0x7fb0fc068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46056/0 (socket says 172.21.15.42:46056) 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 -- 172.21.15.42:0/1828591082 learned_addr learned my addr 172.21.15.42:0/1828591082 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fa59c640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0fc101290 0x7fb0fc19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fb59e640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc101c50 0x7fb0fc19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 -- 172.21.15.42:0/1828591082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0fc101290 msgr2=0x7fb0fc19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0fc101290 0x7fb0fc19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 -- 172.21.15.42:0/1828591082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc101c50 msgr2=0x7fb0fc19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc101c50 0x7fb0fc19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.158+0000 7fb0fad9d640 1 -- 172.21.15.42:0/1828591082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4009900 con 0x7fb0fc1000a0 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.159+0000 7fb0fad9d640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc1000a0 0x7fb0fc068e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00d770 tx=0x7fb0ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.159+0000 7fb0e3fff640 1 -- 172.21.15.42:0/1828591082 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec009c30 con 0x7fb0fc1000a0 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.159+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc19f220 con 0x7fb0fc1000a0 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.159+0000 7fb0e3fff640 1 -- 172.21.15.42:0/1828591082 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0ec0044d0 con 0x7fb0fc1000a0 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.159+0000 7fb0e3fff640 1 -- 172.21.15.42:0/1828591082 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003de0 con 0x7fb0fc1000a0 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.159+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc19f730 con 0x7fb0fc1000a0 2024-03-12T04:12:27.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.160+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc1073c0 con 0x7fb0fc1000a0 2024-03-12T04:12:27.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.161+0000 7fb0e3fff640 1 -- 172.21.15.42:0/1828591082 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb0ec003f40 con 0x7fb0fc1000a0 2024-03-12T04:12:27.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.162+0000 7fb0e3fff640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb0e40a1a30 0x7fb0e40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:27.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.162+0000 7fb0fa59c640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb0e40a1a30 0x7fb0e40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:27.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.162+0000 7fb0e3fff640 1 -- 172.21.15.42:0/1828591082 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb0ec0ccc50 con 0x7fb0fc1000a0 2024-03-12T04:12:27.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.162+0000 7fb0fa59c640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb0e40a1a30 0x7fb0e40a3ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc068bc0 tx=0x7fb0f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:27.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.165+0000 7fb0e3fff640 1 -- 172.21.15.42:0/1828591082 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0ec0963b0 con 0x7fb0fc1000a0 2024-03-12T04:12:27.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.385+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0fc02cdd0 con 0x7fb0fc1000a0 2024-03-12T04:12:27.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.385+0000 7fb0e3fff640 1 -- 172.21.15.42:0/1828591082 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7fb0ec017e90 con 0x7fb0fc1000a0 2024-03-12T04:12:27.388 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:27.388 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-12T04:12:27.388 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb0e40a1a30 msgr2=0x7fb0e40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb0e40a1a30 0x7fb0e40a3ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc068bc0 tx=0x7fb0f0005f90 comp rx=0 tx=0).stop 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc1000a0 msgr2=0x7fb0fc068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc1000a0 0x7fb0fc068e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00d770 tx=0x7fb0ec00dc30 comp rx=0 tx=0).stop 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 shutdown_connections 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb0e40a1a30 0x7fb0e40a3ef0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc101c50 0x7fb0fc19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0fc101290 0x7fb0fc19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 --2- 172.21.15.42:0/1828591082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc1000a0 0x7fb0fc068e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 >> 172.21.15.42:0/1828591082 conn(0x7fb0fc0fb610 msgr2=0x7fb0fc104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 shutdown_connections 2024-03-12T04:12:27.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:27.387+0000 7fb101040640 1 -- 172.21.15.42:0/1828591082 wait complete. 2024-03-12T04:12:27.399 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: Deploying daemon mds.e on smithi119 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.42:6836/3380811405,v1:172.21.15.42:6837/3380811405] up:boot 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: fsmap 4 up:standby 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T04:12:27.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:27 smithi042 ceph-mon[32260]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: Deploying daemon mds.e on smithi119 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.42:6836/3380811405,v1:172.21.15.42:6837/3380811405] up:boot 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: fsmap 4 up:standby 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T04:12:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:27 smithi119 ceph-mon[36119]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:28.365 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:28 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1828591082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:28.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:28 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1828591082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:28.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:28 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1828591082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:28.621 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 podman[66143]: 2024-03-12 04:12:28.483552827 +0000 UTC m=+0.018224571 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:28.621 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 podman[66143]: 2024-03-12 04:12:28.619302632 +0000 UTC m=+0.153974402 container create 41cc2f8e28edffbe93d7865fc79586548033f2375e463210b4a5b8fa67a9904d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-12T04:12:28.890 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 podman[66143]: 2024-03-12 04:12:28.863617609 +0000 UTC m=+0.398289343 container init 41cc2f8e28edffbe93d7865fc79586548033f2375e463210b4a5b8fa67a9904d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-12T04:12:28.890 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 podman[66143]: 2024-03-12 04:12:28.866160096 +0000 UTC m=+0.400831832 container start 41cc2f8e28edffbe93d7865fc79586548033f2375e463210b4a5b8fa67a9904d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 ceph-mds[66163]: -- 172.21.15.119:0/2800677378 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bac41b81a0 con 0x55bac418e800 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e[66159]: starting mds.e at 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 bash[66143]: 41cc2f8e28edffbe93d7865fc79586548033f2375e463210b4a5b8fa67a9904d 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:12:28 smithi119 systemd[1]: Started Ceph mds.e for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:29 smithi119 ceph-mon[36119]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:29 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:29 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:29 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:29 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:29.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:29 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:29.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:29 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:29.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:29.430 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:29 smithi042 ceph-mon[32260]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:29.430 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:29 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.430 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:29 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.430 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:29 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.430 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:29 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:29.430 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:29 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:29.430 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:29 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:29 smithi136 ceph-mon[35571]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:29 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:29 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:29 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:29 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:29 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:29 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/668695498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48102d80 msgr2=0x7f8a48109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/668695498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48102d80 0x7f8a48109610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009ca0 tx=0x7f8a3802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/668695498 shutdown_connections 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/668695498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48102d80 0x7f8a48109610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/668695498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481023c0 0x7f8a48102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/668695498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a481011d0 0x7f8a481015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/668695498 >> 172.21.15.42:0/668695498 conn(0x7f8a480fc720 msgr2=0x7f8a480feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/668695498 shutdown_connections 2024-03-12T04:12:29.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.515+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/668695498 wait complete. 2024-03-12T04:12:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 Processor -- start 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 -- start start 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a481011d0 0x7f8a48074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481023c0 0x7f8a48073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a48102d80 0x7f8a48073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48074210 con 0x7f8a481011d0 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48074380 con 0x7f8a48102d80 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4cd6c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481a11a0 con 0x7f8a481023c0 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a481011d0 0x7f8a48074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a481011d0 0x7f8a48074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46068/0 (socket says 172.21.15.42:46068) 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 -- 172.21.15.42:0/2993139105 learned_addr learned my addr 172.21.15.42:0/2993139105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a45d9b640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481023c0 0x7f8a48073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a46d9d640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a48102d80 0x7f8a48073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 -- 172.21.15.42:0/2993139105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481023c0 msgr2=0x7f8a48073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481023c0 0x7f8a48073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 -- 172.21.15.42:0/2993139105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a48102d80 msgr2=0x7f8a48073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a48102d80 0x7f8a48073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.516+0000 7f8a4659c640 1 -- 172.21.15.42:0/2993139105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38009900 con 0x7f8a481011d0 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.517+0000 7f8a4659c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a481011d0 0x7f8a48074f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000e9f0 tx=0x7f8a4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.517+0000 7f8a3f7fe640 1 -- 172.21.15.42:0/2993139105 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40009700 con 0x7f8a481011d0 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.517+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a481a1450 con 0x7f8a481011d0 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.517+0000 7f8a3f7fe640 1 -- 172.21.15.42:0/2993139105 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a400044d0 con 0x7f8a481011d0 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.517+0000 7f8a3f7fe640 1 -- 172.21.15.42:0/2993139105 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40010430 con 0x7f8a481011d0 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.517+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a481a1960 con 0x7f8a481011d0 2024-03-12T04:12:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.518+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a481084f0 con 0x7f8a481011d0 2024-03-12T04:12:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.519+0000 7f8a3f7fe640 1 -- 172.21.15.42:0/2993139105 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8a4000cd80 con 0x7f8a481011d0 2024-03-12T04:12:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.520+0000 7f8a3f7fe640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8a340a1a30 0x7f8a340a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.520+0000 7f8a45d9b640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8a340a1a30 0x7f8a340a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.520+0000 7f8a3f7fe640 1 -- 172.21.15.42:0/2993139105 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a40014070 con 0x7f8a481011d0 2024-03-12T04:12:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.520+0000 7f8a45d9b640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8a340a1a30 0x7f8a340a3ef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a48074ca0 tx=0x7f8a30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.524+0000 7f8a3f7fe640 1 -- 172.21.15.42:0/2993139105 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a40096a00 con 0x7f8a481011d0 2024-03-12T04:12:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.742+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a4810ca90 con 0x7f8a481011d0 2024-03-12T04:12:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.743+0000 7f8a3f7fe640 1 -- 172.21.15.42:0/2993139105 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7f8a4009a0c0 con 0x7f8a481011d0 2024-03-12T04:12:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:29.746 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-12T04:12:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 6 2024-03-12T04:12:29.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8a340a1a30 msgr2=0x7f8a340a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:29.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8a340a1a30 0x7f8a340a3ef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a48074ca0 tx=0x7f8a30005f90 comp rx=0 tx=0).stop 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a481011d0 msgr2=0x7f8a48074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a481011d0 0x7f8a48074f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000e9f0 tx=0x7f8a4000eeb0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 shutdown_connections 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8a340a1a30 0x7f8a340a3ef0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a48102d80 0x7f8a48073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481023c0 0x7f8a48073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 --2- 172.21.15.42:0/2993139105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a481011d0 0x7f8a48074f30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 >> 172.21.15.42:0/2993139105 conn(0x7f8a480fc720 msgr2=0x7f8a481056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 shutdown_connections 2024-03-12T04:12:29.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:29.745+0000 7f8a4cd6c640 1 -- 172.21.15.42:0/2993139105 wait complete. 2024-03-12T04:12:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:30 smithi136 ceph-mon[35571]: Deploying daemon mds.f on smithi136 2024-03-12T04:12:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:30 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6834/2794337720,v1:172.21.15.119:6835/2794337720] up:boot 2024-03-12T04:12:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:30 smithi136 ceph-mon[35571]: fsmap 5 up:standby 2024-03-12T04:12:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:30 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:12:30.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:30 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2993139105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:30.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:30 smithi119 ceph-mon[36119]: Deploying daemon mds.f on smithi136 2024-03-12T04:12:30.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:30 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6834/2794337720,v1:172.21.15.119:6835/2794337720] up:boot 2024-03-12T04:12:30.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:30 smithi119 ceph-mon[36119]: fsmap 5 up:standby 2024-03-12T04:12:30.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:30 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:12:30.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:30 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2993139105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:30.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:30 smithi042 ceph-mon[32260]: Deploying daemon mds.f on smithi136 2024-03-12T04:12:30.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:30 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6834/2794337720,v1:172.21.15.119:6835/2794337720] up:boot 2024-03-12T04:12:30.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:30 smithi042 ceph-mon[32260]: fsmap 5 up:standby 2024-03-12T04:12:30.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:30 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:12:30.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:30 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2993139105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:31.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:31 smithi119 ceph-mon[36119]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:31 smithi136 ceph-mon[35571]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:31.458 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 systemd[1]: Starting Ceph mds.f for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:31.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:31 smithi042 ceph-mon[32260]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:31.728 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 podman[65496]: 2024-03-12 04:12:31.373105047 +0000 UTC m=+0.018018431 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:31.728 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 podman[65496]: 2024-03-12 04:12:31.482491559 +0000 UTC m=+0.127404967 container create 055b1c6b7828cf91a3ae1cd9cc938b43451a8f63ef2b5b3c8b4d3f34ac5a207c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-12T04:12:31.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.872+0000 7fa738b08640 1 -- 172.21.15.42:0/906536646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7341045c0 msgr2=0x7fa7341069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.872+0000 7fa738b08640 1 --2- 172.21.15.42:0/906536646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7341045c0 0x7fa7341069b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa728009900 tx=0x7fa72802bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 -- 172.21.15.42:0/906536646 shutdown_connections 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 --2- 172.21.15.42:0/906536646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa734106ef0 0x7fa7341092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 --2- 172.21.15.42:0/906536646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7341045c0 0x7fa7341069b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 --2- 172.21.15.42:0/906536646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa734101c90 0x7fa734104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 -- 172.21.15.42:0/906536646 >> 172.21.15.42:0/906536646 conn(0x7fa7340fb660 msgr2=0x7fa7340fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 -- 172.21.15.42:0/906536646 shutdown_connections 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 -- 172.21.15.42:0/906536646 wait complete. 2024-03-12T04:12:31.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 Processor -- start 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.873+0000 7fa738b08640 1 -- start start 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa734101c90 0x7fa7341971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7341045c0 0x7fa734197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa734106ef0 0x7fa73419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73419ecb0 con 0x7fa734101c90 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73419ee20 con 0x7fa7341045c0 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73419efe0 con 0x7fa734106ef0 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa734101c90 0x7fa7341971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa734101c90 0x7fa7341971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42720/0 (socket says 172.21.15.42:42720) 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 -- 172.21.15.42:0/3660188682 learned_addr learned my addr 172.21.15.42:0/3660188682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa732d9d640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa734106ef0 0x7fa73419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa731d9b640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7341045c0 0x7fa734197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:31.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 -- 172.21.15.42:0/3660188682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa734106ef0 msgr2=0x7fa73419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa734106ef0 0x7fa73419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 -- 172.21.15.42:0/3660188682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7341045c0 msgr2=0x7fa734197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7341045c0 0x7fa734197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 -- 172.21.15.42:0/3660188682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa728009560 con 0x7fa734101c90 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa731d9b640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7341045c0 0x7fa734197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa732d9d640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa734106ef0 0x7fa73419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa73259c640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa734101c90 0x7fa7341971c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa72400d950 tx=0x7fa72400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa7237fe640 1 -- 172.21.15.42:0/3660188682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724009c30 con 0x7fa734101c90 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73419f2c0 con 0x7fa734101c90 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa7237fe640 1 -- 172.21.15.42:0/3660188682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa724009d90 con 0x7fa734101c90 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa7237fe640 1 -- 172.21.15.42:0/3660188682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400f6b0 con 0x7fa734101c90 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.874+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73419f7a0 con 0x7fa734101c90 2024-03-12T04:12:31.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.875+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7340fd150 con 0x7fa734101c90 2024-03-12T04:12:31.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.877+0000 7fa7237fe640 1 -- 172.21.15.42:0/3660188682 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa724010aa0 con 0x7fa734101c90 2024-03-12T04:12:31.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.877+0000 7fa7237fe640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa71c0a1a30 0x7fa71c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:31.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.877+0000 7fa731d9b640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa71c0a1a30 0x7fa71c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:31.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.877+0000 7fa7237fe640 1 -- 172.21.15.42:0/3660188682 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7240cccf0 con 0x7fa734101c90 2024-03-12T04:12:31.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.878+0000 7fa731d9b640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa71c0a1a30 0x7fa71c0a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa72802c240 tx=0x7fa728005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:31.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:31.881+0000 7fa7237fe640 1 -- 172.21.15.42:0/3660188682 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7240963a0 con 0x7fa734101c90 2024-03-12T04:12:32.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.105+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa73410cbb0 con 0x7fa734101c90 2024-03-12T04:12:32.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.105+0000 7fa7237fe640 1 -- 172.21.15.42:0/3660188682 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fa724020090 con 0x7fa734101c90 2024-03-12T04:12:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:32.108 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-12T04:12:32.108 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 6 2024-03-12T04:12:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.107+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa71c0a1a30 msgr2=0x7fa71c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.107+0000 7fa738b08640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa71c0a1a30 0x7fa71c0a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa72802c240 tx=0x7fa728005fd0 comp rx=0 tx=0).stop 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.107+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa734101c90 msgr2=0x7fa7341971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.107+0000 7fa738b08640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa734101c90 0x7fa7341971c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa72400d950 tx=0x7fa72400de10 comp rx=0 tx=0).stop 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 shutdown_connections 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa734106ef0 0x7fa73419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa71c0a1a30 0x7fa71c0a3ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7341045c0 0x7fa734197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 --2- 172.21.15.42:0/3660188682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa734101c90 0x7fa7341971c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 >> 172.21.15.42:0/3660188682 conn(0x7fa7340fb660 msgr2=0x7fa7341026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 shutdown_connections 2024-03-12T04:12:32.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:32.108+0000 7fa738b08640 1 -- 172.21.15.42:0/3660188682 wait complete. 2024-03-12T04:12:32.207 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 podman[65496]: 2024-03-12 04:12:31.727059251 +0000 UTC m=+0.371972630 container init 055b1c6b7828cf91a3ae1cd9cc938b43451a8f63ef2b5b3c8b4d3f34ac5a207c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-12T04:12:32.207 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 podman[65496]: 2024-03-12 04:12:31.729620085 +0000 UTC m=+0.374533453 container start 055b1c6b7828cf91a3ae1cd9cc938b43451a8f63ef2b5b3c8b4d3f34ac5a207c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-12T04:12:32.208 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 ceph-mds[65515]: -- 172.21.15.136:0/1408479442 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a4581f61a0 con 0x55a4581cd000 2024-03-12T04:12:32.208 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f[65511]: starting mds.f at 2024-03-12T04:12:32.208 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 bash[65496]: 055b1c6b7828cf91a3ae1cd9cc938b43451a8f63ef2b5b3c8b4d3f34ac5a207c 2024-03-12T04:12:32.208 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:12:31 smithi136 systemd[1]: Started Ceph mds.f for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3660188682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6834/2718353674,v1:172.21.15.136:6835/2718353674] up:boot 2024-03-12T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: fsmap 6 up:standby 2024-03-12T04:12:33.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:32 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3660188682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6834/2718353674,v1:172.21.15.136:6835/2718353674] up:boot 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: fsmap 6 up:standby 2024-03-12T04:12:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:32 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:12:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:33.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:33.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3660188682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:33.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6834/2718353674,v1:172.21.15.136:6835/2718353674] up:boot 2024-03-12T04:12:33.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: fsmap 6 up:standby 2024-03-12T04:12:33.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:32 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:12:34.123 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:34.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:33 smithi042 ceph-mon[32260]: Deploying daemon mds.c on smithi136 2024-03-12T04:12:34.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:33 smithi042 ceph-mon[32260]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:34.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:33 smithi119 ceph-mon[36119]: Deploying daemon mds.c on smithi136 2024-03-12T04:12:34.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:33 smithi119 ceph-mon[36119]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:34.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:33 smithi136 ceph-mon[35571]: Deploying daemon mds.c on smithi136 2024-03-12T04:12:34.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:33 smithi136 ceph-mon[35571]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:34.207 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 systemd[1]: Starting Ceph mds.c for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 -- 172.21.15.42:0/712895884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 msgr2=0x7fecb0102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 --2- 172.21.15.42:0/712895884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0102810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feca4009900 tx=0x7feca402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 -- 172.21.15.42:0/712895884 shutdown_connections 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 --2- 172.21.15.42:0/712895884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fecb0102d50 0x7fecb01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 --2- 172.21.15.42:0/712895884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0102810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 --2- 172.21.15.42:0/712895884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecb01011a0 0x7fecb01015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 -- 172.21.15.42:0/712895884 >> 172.21.15.42:0/712895884 conn(0x7fecb00fc730 msgr2=0x7fecb00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 -- 172.21.15.42:0/712895884 shutdown_connections 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 -- 172.21.15.42:0/712895884 wait complete. 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 Processor -- start 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.236+0000 7fecb67bc640 1 -- start start 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecb67bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fecb01011a0 0x7fecb0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecb67bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecb67bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecb0102d50 0x7fecb01a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecb67bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01a0e40 con 0x7fecb0102390 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecb67bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01a0fb0 con 0x7fecb0102d50 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecb67bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01a1170 con 0x7fecb01011a0 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42738/0 (socket says 172.21.15.42:42738) 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 -- 172.21.15.42:0/737244929 learned_addr learned my addr 172.21.15.42:0/737244929 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaffff640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fecb01011a0 0x7fecb0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecb4d59640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecb0102d50 0x7fecb01a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 -- 172.21.15.42:0/737244929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fecb01011a0 msgr2=0x7fecb0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fecb01011a0 0x7fecb0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 -- 172.21.15.42:0/737244929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecb0102d50 msgr2=0x7fecb01a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecb0102d50 0x7fecb01a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 -- 172.21.15.42:0/737244929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca4009560 con 0x7fecb0102390 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.237+0000 7fecaf7fe640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0199890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feca4005ab0 tx=0x7feca40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.238+0000 7fecad7fa640 1 -- 172.21.15.42:0/737244929 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40044e0 con 0x7fecb0102390 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.238+0000 7fecad7fa640 1 -- 172.21.15.42:0/737244929 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feca4003cb0 con 0x7fecb0102390 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.238+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb01a13f0 con 0x7fecb0102390 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.238+0000 7fecad7fa640 1 -- 172.21.15.42:0/737244929 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4005680 con 0x7fecb0102390 2024-03-12T04:12:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.238+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb01a1900 con 0x7fecb0102390 2024-03-12T04:12:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.239+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb010cab0 con 0x7fecb0102390 2024-03-12T04:12:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.240+0000 7fecad7fa640 1 -- 172.21.15.42:0/737244929 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7feca4004e50 con 0x7fecb0102390 2024-03-12T04:12:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.241+0000 7fecad7fa640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fec980a1a30 0x7fec980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.241+0000 7fecaffff640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fec980a1a30 0x7fec980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.241+0000 7fecad7fa640 1 -- 172.21.15.42:0/737244929 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feca402de40 con 0x7fecb0102390 2024-03-12T04:12:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.241+0000 7fecaffff640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fec980a1a30 0x7fec980a3ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fecb01021f0 tx=0x7feca0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:34.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.245+0000 7fecad7fa640 1 -- 172.21.15.42:0/737244929 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feca40b9000 con 0x7fecb0102390 2024-03-12T04:12:34.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.464+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecb01a1a90 con 0x7fecb0102390 2024-03-12T04:12:34.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.465+0000 7fecad7fa640 1 -- 172.21.15.42:0/737244929 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7fecb01a1a90 con 0x7fecb0102390 2024-03-12T04:12:34.526 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:34.526 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-12T04:12:34.527 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 7 2024-03-12T04:12:34.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fec980a1a30 msgr2=0x7fec980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:34.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fec980a1a30 0x7fec980a3ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fecb01021f0 tx=0x7feca0005f90 comp rx=0 tx=0).stop 2024-03-12T04:12:34.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 msgr2=0x7fecb0199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0199890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feca4005ab0 tx=0x7feca40042f0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 shutdown_connections 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fec980a1a30 0x7fec980a3ef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecb0102d50 0x7fecb01a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fecb0102390 0x7fecb0199890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 --2- 172.21.15.42:0/737244929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fecb01011a0 0x7fecb0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 >> 172.21.15.42:0/737244929 conn(0x7fecb00fc730 msgr2=0x7fecb00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 shutdown_connections 2024-03-12T04:12:34.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:34.467+0000 7fecb67bc640 1 -- 172.21.15.42:0/737244929 wait complete. 2024-03-12T04:12:34.529 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 podman[65802]: 2024-03-12 04:12:34.155208731 +0000 UTC m=+0.018830774 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:34.529 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 podman[65802]: 2024-03-12 04:12:34.256371549 +0000 UTC m=+0.119993594 container create 8fde661ac2918e0dac40b85cd278099e8075528917ba02e3d6422aa101035dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:12:34.529 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 podman[65802]: 2024-03-12 04:12:34.517938825 +0000 UTC m=+0.381560882 container init 8fde661ac2918e0dac40b85cd278099e8075528917ba02e3d6422aa101035dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-12T04:12:34.898 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 podman[65802]: 2024-03-12 04:12:34.520470777 +0000 UTC m=+0.384092822 container start 8fde661ac2918e0dac40b85cd278099e8075528917ba02e3d6422aa101035dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:12:34.898 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mds[65821]: -- 172.21.15.136:0/2133307036 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5636d5fe81a0 con 0x5636d5fbec00 2024-03-12T04:12:34.898 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c[65817]: starting mds.c at 2024-03-12T04:12:34.898 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 bash[65802]: 8fde661ac2918e0dac40b85cd278099e8075528917ba02e3d6422aa101035dc9 2024-03-12T04:12:34.898 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:12:34 smithi136 systemd[1]: Started Ceph mds.c for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:35.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:34 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/737244929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:34 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/737244929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:34 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:34 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/737244929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:35.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:34 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:36.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:35 smithi119 ceph-mon[36119]: Deploying daemon mds.h on smithi119 2024-03-12T04:12:36.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:35 smithi119 ceph-mon[36119]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:36.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:35 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] up:boot 2024-03-12T04:12:36.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:35 smithi119 ceph-mon[36119]: fsmap 7 up:standby 2024-03-12T04:12:36.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:35 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:12:36.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:35 smithi136 ceph-mon[35571]: Deploying daemon mds.h on smithi119 2024-03-12T04:12:36.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:35 smithi136 ceph-mon[35571]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:36.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:35 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] up:boot 2024-03-12T04:12:36.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:35 smithi136 ceph-mon[35571]: fsmap 7 up:standby 2024-03-12T04:12:36.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:35 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:12:36.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:35 smithi042 ceph-mon[32260]: Deploying daemon mds.h on smithi119 2024-03-12T04:12:36.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:35 smithi042 ceph-mon[32260]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:36.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:35 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] up:boot 2024-03-12T04:12:36.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:35 smithi042 ceph-mon[32260]: fsmap 7 up:standby 2024-03-12T04:12:36.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:35 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:12:36.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 -- 172.21.15.42:0/1251535417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc106ea0 msgr2=0x7fe2fc1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 --2- 172.21.15.42:0/1251535417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc106ea0 0x7fe2fc1092f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009ca0 tx=0x7fe2f402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 -- 172.21.15.42:0/1251535417 shutdown_connections 2024-03-12T04:12:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 --2- 172.21.15.42:0/1251535417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc106ea0 0x7fe2fc1092f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 --2- 172.21.15.42:0/1251535417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2fc104570 0x7fe2fc106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 --2- 172.21.15.42:0/1251535417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2fc101c40 0x7fe2fc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 -- 172.21.15.42:0/1251535417 >> 172.21.15.42:0/1251535417 conn(0x7fe2fc0fb630 msgr2=0x7fe2fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 -- 172.21.15.42:0/1251535417 shutdown_connections 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.642+0000 7fe300977640 1 -- 172.21.15.42:0/1251535417 wait complete. 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 Processor -- start 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 -- start start 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc101c40 0x7fe2fc197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2fc104570 0x7fe2fc1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2fc106ea0 0x7fe2fc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc19ec70 con 0x7fe2fc101c40 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc19ede0 con 0x7fe2fc104570 2024-03-12T04:12:36.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.643+0000 7fe300977640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc19efa0 con 0x7fe2fc106ea0 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc101c40 0x7fe2fc197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc101c40 0x7fe2fc197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42744/0 (socket says 172.21.15.42:42744) 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 -- 172.21.15.42:0/906894712 learned_addr learned my addr 172.21.15.42:0/906894712 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2f3fff640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2fc104570 0x7fe2fc1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fad9d640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2fc106ea0 0x7fe2fc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 -- 172.21.15.42:0/906894712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2fc106ea0 msgr2=0x7fe2fc19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2fc106ea0 0x7fe2fc19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 -- 172.21.15.42:0/906894712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2fc104570 msgr2=0x7fe2fc1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2fc104570 0x7fe2fc1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 -- 172.21.15.42:0/906894712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f4009900 con 0x7fe2fc101c40 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2fa59c640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc101c40 0x7fe2fc197180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00e9f0 tx=0x7fe2ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2f37fe640 1 -- 172.21.15.42:0/906894712 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec009700 con 0x7fe2fc101c40 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2f37fe640 1 -- 172.21.15.42:0/906894712 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2ec00cd80 con 0x7fe2fc101c40 2024-03-12T04:12:36.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc19f280 con 0x7fe2fc101c40 2024-03-12T04:12:36.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe2f37fe640 1 -- 172.21.15.42:0/906894712 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec018840 con 0x7fe2fc101c40 2024-03-12T04:12:36.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.644+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc19f790 con 0x7fe2fc101c40 2024-03-12T04:12:36.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.645+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc0fd120 con 0x7fe2fc101c40 2024-03-12T04:12:36.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.646+0000 7fe2f37fe640 1 -- 172.21.15.42:0/906894712 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe2ec0189a0 con 0x7fe2fc101c40 2024-03-12T04:12:36.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.647+0000 7fe2f37fe640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2e80a1a30 0x7fe2e80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:36.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.647+0000 7fe2f3fff640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2e80a1a30 0x7fe2e80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:36.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.647+0000 7fe2f37fe640 1 -- 172.21.15.42:0/906894712 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2ec014070 con 0x7fe2fc101c40 2024-03-12T04:12:36.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.648+0000 7fe2f3fff640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2e80a1a30 0x7fe2e80a3ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc198690 tx=0x7fe2e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:36.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.652+0000 7fe2f37fe640 1 -- 172.21.15.42:0/906894712 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2ec052d20 con 0x7fe2fc101c40 2024-03-12T04:12:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.872+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2fc0fcb90 con 0x7fe2fc101c40 2024-03-12T04:12:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.873+0000 7fe2f37fe640 1 -- 172.21.15.42:0/906894712 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6074 (secure 0 0 0) 0x7fe2ec0969f0 con 0x7fe2fc101c40 2024-03-12T04:12:36.876 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:36.876 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-12T04:12:36.876 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-03-12T04:12:36.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2e80a1a30 msgr2=0x7fe2e80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2e80a1a30 0x7fe2e80a3ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc198690 tx=0x7fe2e400b390 comp rx=0 tx=0).stop 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc101c40 msgr2=0x7fe2fc197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc101c40 0x7fe2fc197180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00e9f0 tx=0x7fe2ec00eeb0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 shutdown_connections 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2fc106ea0 0x7fe2fc19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2e80a1a30 0x7fe2e80a3ef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2fc104570 0x7fe2fc1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 --2- 172.21.15.42:0/906894712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2fc101c40 0x7fe2fc197180 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 >> 172.21.15.42:0/906894712 conn(0x7fe2fc0fb630 msgr2=0x7fe2fc102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 shutdown_connections 2024-03-12T04:12:36.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:36.875+0000 7fe300977640 1 -- 172.21.15.42:0/906894712 wait complete. 2024-03-12T04:12:37.085 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:36 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/906894712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:37.085 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:36 smithi119 systemd[1]: Starting Ceph mds.h for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:37.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:36 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/906894712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:37.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:36 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/906894712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:37.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 podman[66451]: 2024-03-12 04:12:37.039916995 +0000 UTC m=+0.018487827 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:37.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 podman[66451]: 2024-03-12 04:12:37.176100167 +0000 UTC m=+0.154670988 container create 06d5e2c6cabb93e5e21c8884b35f985d7420ce8e97317f80b9745ff3c48f9491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-12T04:12:37.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 podman[66451]: 2024-03-12 04:12:37.420529107 +0000 UTC m=+0.399099928 container init 06d5e2c6cabb93e5e21c8884b35f985d7420ce8e97317f80b9745ff3c48f9491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-12T04:12:37.695 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 podman[66451]: 2024-03-12 04:12:37.423133402 +0000 UTC m=+0.401704224 container start 06d5e2c6cabb93e5e21c8884b35f985d7420ce8e97317f80b9745ff3c48f9491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397) 2024-03-12T04:12:37.696 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mds[66470]: -- 172.21.15.119:0/479472681 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cefc88c1a0 con 0x55cefc863000 2024-03-12T04:12:37.696 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h[66466]: starting mds.h at 2024-03-12T04:12:37.696 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 bash[66451]: 06d5e2c6cabb93e5e21c8884b35f985d7420ce8e97317f80b9745ff3c48f9491 2024-03-12T04:12:37.696 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:12:37 smithi119 systemd[1]: Started Ceph mds.h for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:38.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mon[36119]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:38.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:38.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:38.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:37 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:37 smithi136 ceph-mon[35571]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:38.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:37 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:38.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:37 smithi042 ceph-mon[32260]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:38.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:38.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:38.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:38.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:37 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:38.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 -- 172.21.15.42:0/368608164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c102d60 msgr2=0x7f4a9c1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/368608164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c102d60 0x7f4a9c1095f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009ca0 tx=0x7f4a8c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 -- 172.21.15.42:0/368608164 shutdown_connections 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/368608164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c102d60 0x7f4a9c1095f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/368608164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a9c1023a0 0x7f4a9c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/368608164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a9c1011b0 0x7f4a9c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 -- 172.21.15.42:0/368608164 >> 172.21.15.42:0/368608164 conn(0x7f4a9c0fc760 msgr2=0x7f4a9c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 -- 172.21.15.42:0/368608164 shutdown_connections 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.002+0000 7f4aa3af4640 1 -- 172.21.15.42:0/368608164 wait complete. 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 Processor -- start 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 -- start start 2024-03-12T04:12:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a9c1011b0 0x7f4a9c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c1023a0 0x7f4a9c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a9c102d60 0x7f4a9c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19eb60 con 0x7f4a9c1023a0 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19ecd0 con 0x7f4a9c102d60 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa3af4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19ee90 con 0x7f4a9c1011b0 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c1023a0 0x7f4a9c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c1023a0 0x7f4a9c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42748/0 (socket says 172.21.15.42:42748) 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 -- 172.21.15.42:0/2697505067 learned_addr learned my addr 172.21.15.42:0/2697505067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa1890640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a9c1011b0 0x7f4a9c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa2091640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a9c102d60 0x7f4a9c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 -- 172.21.15.42:0/2697505067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a9c1011b0 msgr2=0x7f4a9c197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a9c1011b0 0x7f4a9c197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 -- 172.21.15.42:0/2697505067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a9c102d60 msgr2=0x7f4a9c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a9c102d60 0x7f4a9c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 -- 172.21.15.42:0/2697505067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c009900 con 0x7f4a9c1023a0 2024-03-12T04:12:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa2091640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a9c102d60 0x7f4a9c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.003+0000 7f4aa108f640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c1023a0 0x7f4a9c1975b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c920 tx=0x7f4a9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.004+0000 7f4a9affd640 1 -- 172.21.15.42:0/2697505067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90007c70 con 0x7f4a9c1023a0 2024-03-12T04:12:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.004+0000 7f4a9affd640 1 -- 172.21.15.42:0/2697505067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a90007dd0 con 0x7f4a9c1023a0 2024-03-12T04:12:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.004+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c19f170 con 0x7f4a9c1023a0 2024-03-12T04:12:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.004+0000 7f4a9affd640 1 -- 172.21.15.42:0/2697505067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a900187c0 con 0x7f4a9c1023a0 2024-03-12T04:12:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.004+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c19f680 con 0x7f4a9c1023a0 2024-03-12T04:12:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.005+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9c1084d0 con 0x7f4a9c1023a0 2024-03-12T04:12:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.006+0000 7f4a9affd640 1 -- 172.21.15.42:0/2697505067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4a9000f450 con 0x7f4a9c1023a0 2024-03-12T04:12:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.006+0000 7f4a9affd640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a840a1a30 0x7f4a840a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.006+0000 7f4aa1890640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a840a1a30 0x7f4a840a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.006+0000 7f4a9affd640 1 -- 172.21.15.42:0/2697505067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a900cd150 con 0x7f4a9c1023a0 2024-03-12T04:12:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.007+0000 7f4aa1890640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a840a1a30 0x7f4a840a3ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a94005db0 tx=0x7f4a94005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:39.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.011+0000 7f4a9affd640 1 -- 172.21.15.42:0/2697505067 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a90096800 con 0x7f4a9c1023a0 2024-03-12T04:12:39.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:38 smithi119 ceph-mon[36119]: Deploying daemon mds.i on smithi136 2024-03-12T04:12:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:38 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] up:boot 2024-03-12T04:12:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:38 smithi119 ceph-mon[36119]: fsmap 8 up:standby 2024-03-12T04:12:39.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:38 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:12:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:38 smithi136 ceph-mon[35571]: Deploying daemon mds.i on smithi136 2024-03-12T04:12:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:38 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] up:boot 2024-03-12T04:12:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:38 smithi136 ceph-mon[35571]: fsmap 8 up:standby 2024-03-12T04:12:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:38 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:12:39.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.228+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a9c10ca70 con 0x7f4a9c1023a0 2024-03-12T04:12:39.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.229+0000 7f4a9affd640 1 -- 172.21.15.42:0/2697505067 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7f4a90099ec0 con 0x7f4a9c1023a0 2024-03-12T04:12:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:39.232 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-12T04:12:39.232 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 9 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a840a1a30 msgr2=0x7f4a840a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a840a1a30 0x7f4a840a3ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a94005db0 tx=0x7f4a94005d40 comp rx=0 tx=0).stop 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c1023a0 msgr2=0x7f4a9c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c1023a0 0x7f4a9c1975b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c920 tx=0x7f4a9000cde0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 shutdown_connections 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f4a840a1a30 0x7f4a840a3ef0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a9c102d60 0x7f4a9c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a9c1023a0 0x7f4a9c1975b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 --2- 172.21.15.42:0/2697505067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a9c1011b0 0x7f4a9c197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 >> 172.21.15.42:0/2697505067 conn(0x7f4a9c0fc760 msgr2=0x7f4a9c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:39.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 shutdown_connections 2024-03-12T04:12:39.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:39.231+0000 7f4aa3af4640 1 -- 172.21.15.42:0/2697505067 wait complete. 2024-03-12T04:12:39.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:38 smithi042 ceph-mon[32260]: Deploying daemon mds.i on smithi136 2024-03-12T04:12:39.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:38 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] up:boot 2024-03-12T04:12:39.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:38 smithi042 ceph-mon[32260]: fsmap 8 up:standby 2024-03-12T04:12:39.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:38 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:12:39.911 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:39 smithi136 systemd[1]: Starting Ceph mds.i for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:40.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:39 smithi119 ceph-mon[36119]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:40.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:39 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2697505067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:39 smithi136 ceph-mon[35571]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:39 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2697505067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:40.207 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:39 smithi136 podman[66106]: 2024-03-12 04:12:39.870288791 +0000 UTC m=+0.017787713 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:40.207 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:39 smithi136 podman[66106]: 2024-03-12 04:12:39.987777747 +0000 UTC m=+0.135276659 container create e92a959ee9ecf2b9d0c795d5f3dd1a7ebd1242e7b63f684c17aec5935946aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-12T04:12:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:39 smithi042 ceph-mon[32260]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:40.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:39 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2697505067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:40.707 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:40 smithi136 podman[66106]: 2024-03-12 04:12:40.235139551 +0000 UTC m=+0.382638463 container init e92a959ee9ecf2b9d0c795d5f3dd1a7ebd1242e7b63f684c17aec5935946aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:12:40.707 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:40 smithi136 podman[66106]: 2024-03-12 04:12:40.238506816 +0000 UTC m=+0.386005731 container start e92a959ee9ecf2b9d0c795d5f3dd1a7ebd1242e7b63f684c17aec5935946aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:12:40.708 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:40 smithi136 ceph-mds[66125]: -- 172.21.15.136:0/651536995 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d43d861a0 con 0x556d43d5d000 2024-03-12T04:12:40.708 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i[66121]: starting mds.i at 2024-03-12T04:12:40.708 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:40 smithi136 bash[66106]: e92a959ee9ecf2b9d0c795d5f3dd1a7ebd1242e7b63f684c17aec5935946aca8 2024-03-12T04:12:40.708 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:12:40 smithi136 systemd[1]: Started Ceph mds.i for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:41.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 -- 172.21.15.42:0/4129107619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658102bb0 msgr2=0x7fd658109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 --2- 172.21.15.42:0/4129107619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658102bb0 0x7fd658109440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd650009ca0 tx=0x7fd65002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 -- 172.21.15.42:0/4129107619 shutdown_connections 2024-03-12T04:12:41.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 --2- 172.21.15.42:0/4129107619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658102bb0 0x7fd658109440 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 --2- 172.21.15.42:0/4129107619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581021f0 0x7fd658102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 --2- 172.21.15.42:0/4129107619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd658101000 0x7fd658101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 -- 172.21.15.42:0/4129107619 >> 172.21.15.42:0/4129107619 conn(0x7fd6580fc590 msgr2=0x7fd6580fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:41.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 -- 172.21.15.42:0/4129107619 shutdown_connections 2024-03-12T04:12:41.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.352+0000 7fd65d802640 1 -- 172.21.15.42:0/4129107619 wait complete. 2024-03-12T04:12:41.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 Processor -- start 2024-03-12T04:12:41.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 -- start start 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658101000 0x7fd658074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6581021f0 0x7fd658073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd658102bb0 0x7fd658073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd658074210 con 0x7fd658101000 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd658074380 con 0x7fd658102bb0 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.353+0000 7fd65d802640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a1040 con 0x7fd6581021f0 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658101000 0x7fd658074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658101000 0x7fd658074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42760/0 (socket says 172.21.15.42:42760) 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 -- 172.21.15.42:0/1560522304 learned_addr learned my addr 172.21.15.42:0/1560522304 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd6577fe640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd658102bb0 0x7fd658073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd6567fc640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6581021f0 0x7fd658073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 -- 172.21.15.42:0/1560522304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6581021f0 msgr2=0x7fd658073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:41.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6581021f0 0x7fd658073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 -- 172.21.15.42:0/1560522304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd658102bb0 msgr2=0x7fd658073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd658102bb0 0x7fd658073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 -- 172.21.15.42:0/1560522304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650009900 con 0x7fd658101000 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd6567fc640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6581021f0 0x7fd658073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd656ffd640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658101000 0x7fd658074f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd64800b480 tx=0x7fd64800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd65c800640 1 -- 172.21.15.42:0/1560522304 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648010870 con 0x7fd658101000 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6581a12f0 con 0x7fd658101000 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd65c800640 1 -- 172.21.15.42:0/1560522304 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd648010e80 con 0x7fd658101000 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd65c800640 1 -- 172.21.15.42:0/1560522304 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648017360 con 0x7fd658101000 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.354+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6581a18d0 con 0x7fd658101000 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.355+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd658108320 con 0x7fd658101000 2024-03-12T04:12:41.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.356+0000 7fd65c800640 1 -- 172.21.15.42:0/1560522304 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd6480109d0 con 0x7fd658101000 2024-03-12T04:12:41.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.357+0000 7fd65c800640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6400a1a30 0x7fd6400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:41.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.357+0000 7fd6567fc640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6400a1a30 0x7fd6400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:41.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.357+0000 7fd65c800640 1 -- 172.21.15.42:0/1560522304 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6480cc9e0 con 0x7fd658101000 2024-03-12T04:12:41.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.357+0000 7fd6567fc640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6400a1a30 0x7fd6400a3ef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd658074ca0 tx=0x7fd64c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:41.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.361+0000 7fd65c800640 1 -- 172.21.15.42:0/1560522304 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd648096120 con 0x7fd658101000 2024-03-12T04:12:41.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.582+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd65810c930 con 0x7fd658101000 2024-03-12T04:12:41.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.582+0000 7fd65c800640 1 -- 172.21.15.42:0/1560522304 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7fd6480997e0 con 0x7fd658101000 2024-03-12T04:12:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:41.587 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 10 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.584+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6400a1a30 msgr2=0x7fd6400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.584+0000 7fd65d802640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6400a1a30 0x7fd6400a3ef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd658074ca0 tx=0x7fd64c005d30 comp rx=0 tx=0).stop 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.584+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658101000 msgr2=0x7fd658074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.584+0000 7fd65d802640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658101000 0x7fd658074f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd64800b480 tx=0x7fd64800b940 comp rx=0 tx=0).stop 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 shutdown_connections 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd6400a1a30 0x7fd6400a3ef0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd658102bb0 0x7fd658073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6581021f0 0x7fd658073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 --2- 172.21.15.42:0/1560522304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658101000 0x7fd658074f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:41.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 >> 172.21.15.42:0/1560522304 conn(0x7fd6580fc590 msgr2=0x7fd6581054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:41.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 shutdown_connections 2024-03-12T04:12:41.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:41.585+0000 7fd65d802640 1 -- 172.21.15.42:0/1560522304 wait complete. 2024-03-12T04:12:41.597 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.597 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.597 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.597 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:41.598 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:41.598 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:41.598 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: Deploying daemon mds.g on smithi042 2024-03-12T04:12:41.598 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:41.598 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] up:boot 2024-03-12T04:12:41.598 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: fsmap 9 up:standby 2024-03-12T04:12:41.598 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:41 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: Deploying daemon mds.g on smithi042 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] up:boot 2024-03-12T04:12:41.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: fsmap 9 up:standby 2024-03-12T04:12:41.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:41 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:12:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: Deploying daemon mds.g on smithi042 2024-03-12T04:12:41.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:41.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] up:boot 2024-03-12T04:12:41.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: fsmap 9 up:standby 2024-03-12T04:12:41.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:41 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:12:42.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:42 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1560522304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:42 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1560522304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:42.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:42 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1560522304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:42.768 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:42 smithi042 systemd[1]: Starting Ceph mds.g for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:43.066 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:42 smithi042 podman[70840]: 2024-03-12 04:12:42.668102647 +0000 UTC m=+0.018634972 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:43.066 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:42 smithi042 podman[70840]: 2024-03-12 04:12:42.772688851 +0000 UTC m=+0.123221168 container create 01229492569ce6efb85fb855ff686ee218f738c3155ee0b1c6796c96e57586c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:12:43.416 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:43 smithi042 podman[70840]: 2024-03-12 04:12:43.067294955 +0000 UTC m=+0.417827283 container init 01229492569ce6efb85fb855ff686ee218f738c3155ee0b1c6796c96e57586c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-12T04:12:43.417 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:43 smithi042 podman[70840]: 2024-03-12 04:12:43.06977853 +0000 UTC m=+0.420310856 container start 01229492569ce6efb85fb855ff686ee218f738c3155ee0b1c6796c96e57586c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-12T04:12:43.417 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mds[70859]: -- 172.21.15.42:0/2025226029 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564917b8c1a0 con 0x564917b62800 2024-03-12T04:12:43.417 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g[70855]: starting mds.g at 2024-03-12T04:12:43.417 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:43 smithi042 bash[70840]: 01229492569ce6efb85fb855ff686ee218f738c3155ee0b1c6796c96e57586c5 2024-03-12T04:12:43.417 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:12:43 smithi042 systemd[1]: Started Ceph mds.g for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:43.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:43.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:43 smithi119 ceph-mon[36119]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:43.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:43.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:43.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:43 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:43.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:43 smithi136 ceph-mon[35571]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:43.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:43.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:43.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:43 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:43.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 -- 172.21.15.42:0/3582505127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 msgr2=0x7fc1f8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:43.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/3582505127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f8100830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009900 tx=0x7fc1e002bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:43.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 -- 172.21.15.42:0/3582505127 shutdown_connections 2024-03-12T04:12:43.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/3582505127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1f8100d70 0x7fc1f81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/3582505127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f8100830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/3582505127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1f80ffa70 0x7fc1f80ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 -- 172.21.15.42:0/3582505127 >> 172.21.15.42:0/3582505127 conn(0x7fc1f80fb610 msgr2=0x7fc1f80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:43.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 -- 172.21.15.42:0/3582505127 shutdown_connections 2024-03-12T04:12:43.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.711+0000 7fc1fc8db640 1 -- 172.21.15.42:0/3582505127 wait complete. 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 Processor -- start 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 -- start start 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1f80ffa70 0x7fc1f8199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f81998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1f8100d70 0x7fc1f81a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81a0e50 con 0x7fc1f81003d0 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81a0fc0 con 0x7fc1f8100d70 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1fc8db640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81a1180 con 0x7fc1f80ffa70 2024-03-12T04:12:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f81998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f81998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44334/0 (socket says 172.21.15.42:44334) 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.712+0000 7fc1effff640 1 -- 172.21.15.42:0/1422725701 learned_addr learned my addr 172.21.15.42:0/1422725701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1f659c640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1f80ffa70 0x7fc1f8199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1f6d9d640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1f8100d70 0x7fc1f81a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1effff640 1 -- 172.21.15.42:0/1422725701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1f80ffa70 msgr2=0x7fc1f8199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1effff640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1f80ffa70 0x7fc1f8199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1effff640 1 -- 172.21.15.42:0/1422725701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1f8100d70 msgr2=0x7fc1f81a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1effff640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1f8100d70 0x7fc1f81a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1effff640 1 -- 172.21.15.42:0/1422725701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0009560 con 0x7fc1f81003d0 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1f659c640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1f80ffa70 0x7fc1f8199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1f6d9d640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1f8100d70 0x7fc1f81a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1effff640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f81998a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0036040 tx=0x7fc1e0004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1ef7fe640 1 -- 172.21.15.42:0/1422725701 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e00035f0 con 0x7fc1f81003d0 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f81a1400 con 0x7fc1f81003d0 2024-03-12T04:12:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1ef7fe640 1 -- 172.21.15.42:0/1422725701 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1e002de90 con 0x7fc1f81003d0 2024-03-12T04:12:43.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1ef7fe640 1 -- 172.21.15.42:0/1422725701 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0005490 con 0x7fc1f81003d0 2024-03-12T04:12:43.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.713+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f81a1960 con 0x7fc1f81003d0 2024-03-12T04:12:43.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.714+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f810aa10 con 0x7fc1f81003d0 2024-03-12T04:12:43.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.715+0000 7fc1ef7fe640 1 -- 172.21.15.42:0/1422725701 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc1e0003750 con 0x7fc1f81003d0 2024-03-12T04:12:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.716+0000 7fc1ef7fe640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc1e40a1a30 0x7fc1e40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.716+0000 7fc1ef7fe640 1 -- 172.21.15.42:0/1422725701 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1e00efa20 con 0x7fc1f81003d0 2024-03-12T04:12:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.716+0000 7fc1f659c640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc1e40a1a30 0x7fc1e40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.717+0000 7fc1f659c640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc1e40a1a30 0x7fc1e40a3ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009c20 tx=0x7fc1e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:43.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.721+0000 7fc1ef7fe640 1 -- 172.21.15.42:0/1422725701 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1e00b90b0 con 0x7fc1f81003d0 2024-03-12T04:12:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mon[32260]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:43.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:43 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:43.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.951+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1f81a1590 con 0x7fc1f81003d0 2024-03-12T04:12:43.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.952+0000 7fc1ef7fe640 1 -- 172.21.15.42:0/1422725701 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7fc1e00d0e60 con 0x7fc1f81003d0 2024-03-12T04:12:43.955 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:43.955 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T04:12:43.955 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 11 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc1e40a1a30 msgr2=0x7fc1e40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc1e40a1a30 0x7fc1e40a3ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009c20 tx=0x7fc1e8005f90 comp rx=0 tx=0).stop 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 msgr2=0x7fc1f81998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f81998a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0036040 tx=0x7fc1e0004140 comp rx=0 tx=0).stop 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 shutdown_connections 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc1e40a1a30 0x7fc1e40a3ef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1f8100d70 0x7fc1f81a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f81003d0 0x7fc1f81998a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 --2- 172.21.15.42:0/1422725701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1f80ffa70 0x7fc1f8199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 >> 172.21.15.42:0/1422725701 conn(0x7fc1f80fb610 msgr2=0x7fc1f80fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 shutdown_connections 2024-03-12T04:12:43.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:43.954+0000 7fc1fc8db640 1 -- 172.21.15.42:0/1422725701 wait complete. 2024-03-12T04:12:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:44 smithi042 ceph-mon[32260]: Deploying daemon mds.a on smithi042 2024-03-12T04:12:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:44 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.42:6838/1870620293,v1:172.21.15.42:6839/1870620293] up:boot 2024-03-12T04:12:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:44 smithi042 ceph-mon[32260]: fsmap 10 up:standby 2024-03-12T04:12:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:12:44.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:44 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1422725701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:44.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:44 smithi119 ceph-mon[36119]: Deploying daemon mds.a on smithi042 2024-03-12T04:12:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:44 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.42:6838/1870620293,v1:172.21.15.42:6839/1870620293] up:boot 2024-03-12T04:12:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:44 smithi119 ceph-mon[36119]: fsmap 10 up:standby 2024-03-12T04:12:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:12:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:44 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1422725701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:44 smithi136 ceph-mon[35571]: Deploying daemon mds.a on smithi042 2024-03-12T04:12:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:44 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.42:6838/1870620293,v1:172.21.15.42:6839/1870620293] up:boot 2024-03-12T04:12:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:44 smithi136 ceph-mon[35571]: fsmap 10 up:standby 2024-03-12T04:12:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:12:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:44 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1422725701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:45.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:45 smithi042 ceph-mon[32260]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:45.566 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:45 smithi042 systemd[1]: Starting Ceph mds.a for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:12:45.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:45 smithi119 ceph-mon[36119]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:45.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:45 smithi136 ceph-mon[35571]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:45.917 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:45 smithi042 podman[71189]: 2024-03-12 04:12:45.566323658 +0000 UTC m=+0.018187843 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:12:45.918 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:45 smithi042 podman[71189]: 2024-03-12 04:12:45.677468663 +0000 UTC m=+0.129332836 container create b8e97fa10157d3ab195fbc4ca70dc3c02e840a3cec80b10606e9ee1b70385f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-03-12T04:12:45.918 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:45 smithi042 podman[71189]: 2024-03-12 04:12:45.913441001 +0000 UTC m=+0.365305183 container init b8e97fa10157d3ab195fbc4ca70dc3c02e840a3cec80b10606e9ee1b70385f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-03-12T04:12:45.918 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:45 smithi042 podman[71189]: 2024-03-12 04:12:45.916049705 +0000 UTC m=+0.367913880 container start b8e97fa10157d3ab195fbc4ca70dc3c02e840a3cec80b10606e9ee1b70385f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-12T04:12:45.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 -- 172.21.15.42:0/944389888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 msgr2=0x7fd1641015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/944389888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641015f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd154009900 tx=0x7fd15402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 -- 172.21.15.42:0/944389888 shutdown_connections 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/944389888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd164102d80 0x7fd164109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/944389888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1641023c0 0x7fd164102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/944389888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641015f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 -- 172.21.15.42:0/944389888 >> 172.21.15.42:0/944389888 conn(0x7fd1640fc720 msgr2=0x7fd1640feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 -- 172.21.15.42:0/944389888 shutdown_connections 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.079+0000 7fd16a9d0640 1 -- 172.21.15.42:0/944389888 wait complete. 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 Processor -- start 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 -- start start 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1641023c0 0x7fd1641976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd164102d80 0x7fd16419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419eca0 con 0x7fd1641011d0 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419ee10 con 0x7fd164102d80 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd16a9d0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419efd0 con 0x7fd1641023c0 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44364/0 (socket says 172.21.15.42:44364) 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 -- 172.21.15.42:0/1294027384 learned_addr learned my addr 172.21.15.42:0/1294027384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd1637fe640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1641023c0 0x7fd1641976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd168f6d640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd164102d80 0x7fd16419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 -- 172.21.15.42:0/1294027384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1641023c0 msgr2=0x7fd1641976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1641023c0 0x7fd1641976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 -- 172.21.15.42:0/1294027384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd164102d80 msgr2=0x7fd16419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd164102d80 0x7fd16419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.080+0000 7fd163fff640 1 -- 172.21.15.42:0/1294027384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154009560 con 0x7fd1641011d0 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.081+0000 7fd163fff640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641971b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd154000c00 tx=0x7fd1540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.081+0000 7fd1617fa640 1 -- 172.21.15.42:0/1294027384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1540044e0 con 0x7fd1641011d0 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.081+0000 7fd1617fa640 1 -- 172.21.15.42:0/1294027384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd15402de90 con 0x7fd1641011d0 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.081+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16419f250 con 0x7fd1641011d0 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.081+0000 7fd1617fa640 1 -- 172.21.15.42:0/1294027384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154005410 con 0x7fd1641011d0 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.081+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16419f760 con 0x7fd1641011d0 2024-03-12T04:12:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.082+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1641085c0 con 0x7fd1641011d0 2024-03-12T04:12:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.083+0000 7fd1617fa640 1 -- 172.21.15.42:0/1294027384 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd154005570 con 0x7fd1641011d0 2024-03-12T04:12:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.084+0000 7fd1617fa640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd14c0a1a30 0x7fd14c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.084+0000 7fd1637fe640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd14c0a1a30 0x7fd14c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.084+0000 7fd1617fa640 1 -- 172.21.15.42:0/1294027384 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1540ef1a0 con 0x7fd1641011d0 2024-03-12T04:12:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.084+0000 7fd1637fe640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd14c0a1a30 0x7fd14c0a3ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1641986c0 tx=0x7fd15800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:46.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.088+0000 7fd1617fa640 1 -- 172.21.15.42:0/1294027384 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1540b8850 con 0x7fd1641011d0 2024-03-12T04:12:46.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.311+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd16402cdd0 con 0x7fd1641011d0 2024-03-12T04:12:46.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.312+0000 7fd1617fa640 1 -- 172.21.15.42:0/1294027384 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7fd1540bbf10 con 0x7fd1641011d0 2024-03-12T04:12:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:46.315 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T04:12:46.315 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 11 2024-03-12T04:12:46.316 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:45 smithi042 ceph-mds[71208]: -- 172.21.15.42:0/2818784209 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564267eb01a0 con 0x564267e87000 2024-03-12T04:12:46.316 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:45 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a[71204]: starting mds.a at 2024-03-12T04:12:46.316 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:46 smithi042 bash[71189]: b8e97fa10157d3ab195fbc4ca70dc3c02e840a3cec80b10606e9ee1b70385f32 2024-03-12T04:12:46.316 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:12:46 smithi042 systemd[1]: Started Ceph mds.a for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:46.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd14c0a1a30 msgr2=0x7fd14c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:46.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd14c0a1a30 0x7fd14c0a3ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1641986c0 tx=0x7fd15800b390 comp rx=0 tx=0).stop 2024-03-12T04:12:46.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 msgr2=0x7fd1641971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:46.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641971b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd154000c00 tx=0x7fd1540042f0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 shutdown_connections 2024-03-12T04:12:46.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd14c0a1a30 0x7fd14c0a3ef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd164102d80 0x7fd16419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1641023c0 0x7fd1641976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 --2- 172.21.15.42:0/1294027384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1641011d0 0x7fd1641971b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:46.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 >> 172.21.15.42:0/1294027384 conn(0x7fd1640fc720 msgr2=0x7fd164105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:46.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 shutdown_connections 2024-03-12T04:12:46.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:46.314+0000 7fd16a9d0640 1 -- 172.21.15.42:0/1294027384 wait complete. 2024-03-12T04:12:47.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: Deploying daemon mds.b on smithi119 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1294027384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] up:boot 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: fsmap 11 up:standby 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:12:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:47 smithi119 ceph-mon[36119]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: Deploying daemon mds.b on smithi119 2024-03-12T04:12:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1294027384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:47.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] up:boot 2024-03-12T04:12:47.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: fsmap 11 up:standby 2024-03-12T04:12:47.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:12:47.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:47 smithi136 ceph-mon[35571]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:47.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: Deploying daemon mds.b on smithi119 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1294027384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] up:boot 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: fsmap 11 up:standby 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:12:47.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:47 smithi042 ceph-mon[32260]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:48.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 -- 172.21.15.42:0/4042635727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774101ca0 msgr2=0x7f8774104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 --2- 172.21.15.42:0/4042635727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774101ca0 0x7f8774104090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f876402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 -- 172.21.15.42:0/4042635727 shutdown_connections 2024-03-12T04:12:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 --2- 172.21.15.42:0/4042635727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8774106f00 0x7f8774109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 --2- 172.21.15.42:0/4042635727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87741045d0 0x7f87741069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 --2- 172.21.15.42:0/4042635727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774101ca0 0x7f8774104090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 -- 172.21.15.42:0/4042635727 >> 172.21.15.42:0/4042635727 conn(0x7f87740fb6b0 msgr2=0x7f87740fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.442+0000 7f877ad81640 1 -- 172.21.15.42:0/4042635727 shutdown_connections 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 -- 172.21.15.42:0/4042635727 wait complete. 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 Processor -- start 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 -- start start 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8774101ca0 0x7f8774197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87741045d0 0x7f87741976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774106f00 0x7f877419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877419ec60 con 0x7f8774106f00 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877419edd0 con 0x7f87741045d0 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877ad81640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877419ef90 con 0x7f8774101ca0 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877931e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774106f00 0x7f877419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877931e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774106f00 0x7f877419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44388/0 (socket says 172.21.15.42:44388) 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f8773fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87741045d0 0x7f87741976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f877931e640 1 -- 172.21.15.42:0/4235148738 learned_addr learned my addr 172.21.15.42:0/4235148738 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.443+0000 7f8778b1d640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8774101ca0 0x7f8774197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877931e640 1 -- 172.21.15.42:0/4235148738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8774101ca0 msgr2=0x7f8774197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877931e640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8774101ca0 0x7f8774197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877931e640 1 -- 172.21.15.42:0/4235148738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87741045d0 msgr2=0x7f87741976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877931e640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87741045d0 0x7f87741976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877931e640 1 -- 172.21.15.42:0/4235148738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764009560 con 0x7f8774106f00 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877931e640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774106f00 0x7f877419e720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f876c00cad0 tx=0x7f876c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f8771ffb640 1 -- 172.21.15.42:0/4235148738 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00ae20 con 0x7f8774106f00 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f8771ffb640 1 -- 172.21.15.42:0/4235148738 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f876c005020 con 0x7f8774106f00 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877419f270 con 0x7f8774106f00 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f8771ffb640 1 -- 172.21.15.42:0/4235148738 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00f690 con 0x7f8774106f00 2024-03-12T04:12:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.444+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877419f7d0 con 0x7f8774106f00 2024-03-12T04:12:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.445+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87740fd1a0 con 0x7f8774106f00 2024-03-12T04:12:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.446+0000 7f8771ffb640 1 -- 172.21.15.42:0/4235148738 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f876c005190 con 0x7f8774106f00 2024-03-12T04:12:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.447+0000 7f8771ffb640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f875c0a1ab0 0x7f875c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.447+0000 7f8778b1d640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f875c0a1ab0 0x7f875c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.447+0000 7f8771ffb640 1 -- 172.21.15.42:0/4235148738 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f876c0cce40 con 0x7f8774106f00 2024-03-12T04:12:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.447+0000 7f8778b1d640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f875c0a1ab0 0x7f875c0a3f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c00 tx=0x7f876402c770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.451+0000 7f8771ffb640 1 -- 172.21.15.42:0/4235148738 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f876c0965a0 con 0x7f8774106f00 2024-03-12T04:12:48.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.683+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f877402cdd0 con 0x7f8774106f00 2024-03-12T04:12:48.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.684+0000 7f8771ffb640 1 -- 172.21.15.42:0/4235148738 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9245 (secure 0 0 0) 0x7f876c099c60 con 0x7f8774106f00 2024-03-12T04:12:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:48.687 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T04:12:48.687 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f875c0a1ab0 msgr2=0x7f875c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f875c0a1ab0 0x7f875c0a3f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c00 tx=0x7f876402c770 comp rx=0 tx=0).stop 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774106f00 msgr2=0x7f877419e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774106f00 0x7f877419e720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f876c00cad0 tx=0x7f876c00cf90 comp rx=0 tx=0).stop 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 shutdown_connections 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8774106f00 0x7f877419e720 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f875c0a1ab0 0x7f875c0a3f70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87741045d0 0x7f87741976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 --2- 172.21.15.42:0/4235148738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8774101ca0 0x7f8774197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:48.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 >> 172.21.15.42:0/4235148738 conn(0x7f87740fb6b0 msgr2=0x7f87741025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:48.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 shutdown_connections 2024-03-12T04:12:48.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:48.686+0000 7f877ad81640 1 -- 172.21.15.42:0/4235148738 wait complete. 2024-03-12T04:12:48.696 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:48 smithi119 podman[66761]: 2024-03-12 04:12:48.648902848 +0000 UTC m=+0.134449475 container create c6dd84ddb419b6e64c2b90ba82f31b3ac6b1884418f446af203c931d18c60447 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-94a5397, GIT_CLEAN=True) 2024-03-12T04:12:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:48 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/4235148738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:49.054 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:48 smithi119 podman[66761]: 2024-03-12 04:12:48.927207134 +0000 UTC m=+0.412753726 container init c6dd84ddb419b6e64c2b90ba82f31b3ac6b1884418f446af203c931d18c60447 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397) 2024-03-12T04:12:49.054 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:48 smithi119 podman[66761]: 2024-03-12 04:12:48.929704032 +0000 UTC m=+0.415250625 container start c6dd84ddb419b6e64c2b90ba82f31b3ac6b1884418f446af203c931d18c60447 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-12T04:12:49.055 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:48 smithi119 ceph-mds[66780]: -- 172.21.15.119:0/1339436508 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56055ea141a0 con 0x56055e9eac00 2024-03-12T04:12:49.055 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:48 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: starting mds.b at 2024-03-12T04:12:49.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:48 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/4235148738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:49.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:48 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/4235148738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:49.445 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:49 smithi119 bash[66761]: c6dd84ddb419b6e64c2b90ba82f31b3ac6b1884418f446af203c931d18c60447 2024-03-12T04:12:49.446 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:49 smithi119 systemd[1]: Started Ceph mds.b for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:12:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:49 smithi042 ceph-mon[32260]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:49 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:50.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:49 smithi119 ceph-mon[36119]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:50.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:49 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:50.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:49 smithi136 ceph-mon[35571]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:50.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:50.150 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:49 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:50.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:12:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/2840190374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4106ea0 msgr2=0x7fe7f41092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/2840190374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4106ea0 0x7fe7f41092f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009ca0 tx=0x7fe7ec02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/2840190374 shutdown_connections 2024-03-12T04:12:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/2840190374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4106ea0 0x7fe7f41092f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/2840190374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4104570 0x7fe7f4106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/2840190374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7f4101c40 0x7fe7f4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/2840190374 >> 172.21.15.42:0/2840190374 conn(0x7fe7f40fb630 msgr2=0x7fe7f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/2840190374 shutdown_connections 2024-03-12T04:12:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.811+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/2840190374 wait complete. 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 Processor -- start 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 -- start start 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7f4101c40 0x7fe7f41971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4104570 0x7fe7f41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4106ea0 0x7fe7f419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f419eca0 con 0x7fe7f4104570 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f419ee10 con 0x7fe7f4101c40 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f9d6c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f419efd0 con 0x7fe7f4106ea0 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4104570 0x7fe7f41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7f4101c40 0x7fe7f41971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4104570 0x7fe7f41976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44410/0 (socket says 172.21.15.42:44410) 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7f4101c40 0x7fe7f41971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:46308/0 (socket says 172.21.15.42:46308) 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f2ffd640 1 -- 172.21.15.42:0/756482625 learned_addr learned my addr 172.21.15.42:0/756482625 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.812+0000 7fe7f3fff640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4106ea0 0x7fe7f419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f2ffd640 1 -- 172.21.15.42:0/756482625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4106ea0 msgr2=0x7fe7f419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f2ffd640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4106ea0 0x7fe7f419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f2ffd640 1 -- 172.21.15.42:0/756482625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7f4101c40 msgr2=0x7fe7f41971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f2ffd640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7f4101c40 0x7fe7f41971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f2ffd640 1 -- 172.21.15.42:0/756482625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec009900 con 0x7fe7f4104570 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f2ffd640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4104570 0x7fe7f41976f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00d740 tx=0x7fe7dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f0ff9640 1 -- 172.21.15.42:0/756482625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc009c30 con 0x7fe7f4104570 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f0ff9640 1 -- 172.21.15.42:0/756482625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc00de50 con 0x7fe7f4104570 2024-03-12T04:12:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f419f2b0 con 0x7fe7f4104570 2024-03-12T04:12:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f0ff9640 1 -- 172.21.15.42:0/756482625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc004e40 con 0x7fe7f4104570 2024-03-12T04:12:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.813+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f419f770 con 0x7fe7f4104570 2024-03-12T04:12:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.814+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f40fd120 con 0x7fe7f4104570 2024-03-12T04:12:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.815+0000 7fe7f0ff9640 1 -- 172.21.15.42:0/756482625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe7dc004fa0 con 0x7fe7f4104570 2024-03-12T04:12:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.816+0000 7fe7f0ff9640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7e00a1a30 0x7fe7e00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.816+0000 7fe7f37fe640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7e00a1a30 0x7fe7e00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.816+0000 7fe7f0ff9640 1 -- 172.21.15.42:0/756482625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe7dc0ccc20 con 0x7fe7f4104570 2024-03-12T04:12:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.816+0000 7fe7f37fe640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7e00a1a30 0x7fe7e00a3ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005d00 tx=0x7fe7e400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:50.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:50.820+0000 7fe7f0ff9640 1 -- 172.21.15.42:0/756482625 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe7dc096380 con 0x7fe7f4104570 2024-03-12T04:12:51.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:50 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] up:boot 2024-03-12T04:12:51.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:50 smithi042 ceph-mon[32260]: fsmap 12 up:standby 2024-03-12T04:12:51.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:50 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:12:51.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:50 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:51.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.075+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7f410ca80 con 0x7fe7f4104570 2024-03-12T04:12:51.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.076+0000 7fe7f0ff9640 1 -- 172.21.15.42:0/756482625 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10040 (secure 0 0 0) 0x7fe7dc020090 con 0x7fe7f4104570 2024-03-12T04:12:51.078 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:51.078 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-12T04:12:51.079 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-03-12T04:12:51.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7e00a1a30 msgr2=0x7fe7e00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7e00a1a30 0x7fe7e00a3ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005d00 tx=0x7fe7e400b410 comp rx=0 tx=0).stop 2024-03-12T04:12:51.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4104570 msgr2=0x7fe7f41976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4104570 0x7fe7f41976f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00d740 tx=0x7fe7dc00dc00 comp rx=0 tx=0).stop 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 shutdown_connections 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4106ea0 0x7fe7f419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4104570 0x7fe7f41976f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7e00a1a30 0x7fe7e00a3ef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 --2- 172.21.15.42:0/756482625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7f4101c40 0x7fe7f41971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 >> 172.21.15.42:0/756482625 conn(0x7fe7f40fb630 msgr2=0x7fe7f4102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 shutdown_connections 2024-03-12T04:12:51.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.078+0000 7fe7f9d6c640 1 -- 172.21.15.42:0/756482625 wait complete. 2024-03-12T04:12:51.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:12:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:50 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] up:boot 2024-03-12T04:12:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:50 smithi119 ceph-mon[36119]: fsmap 12 up:standby 2024-03-12T04:12:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:50 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:12:51.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:50 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:51.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.199+0000 7f5ba549a640 1 -- 172.21.15.42:0/1705157116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba01023b0 msgr2=0x7f5ba0102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.199+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1705157116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba01023b0 0x7f5ba0102830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009900 tx=0x7f5b8802bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:51.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- 172.21.15.42:0/1705157116 shutdown_connections 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1705157116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ba0102d70 0x7f5ba0109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1705157116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba01023b0 0x7f5ba0102830 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1705157116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba01011c0 0x7f5ba01015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- 172.21.15.42:0/1705157116 >> 172.21.15.42:0/1705157116 conn(0x7f5ba00fc730 msgr2=0x7f5ba00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- 172.21.15.42:0/1705157116 shutdown_connections 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- 172.21.15.42:0/1705157116 wait complete. 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 Processor -- start 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- start start 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ba01011c0 0x7f5ba01971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba01023b0 0x7f5ba0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba0102d70 0x7f5ba019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba019ecd0 con 0x7f5ba0102d70 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba019ee40 con 0x7f5ba01023b0 2024-03-12T04:12:51.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5ba549a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba019f000 con 0x7f5ba01011c0 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5b9f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba0102d70 0x7f5ba019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5b9f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba0102d70 0x7f5ba019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44430/0 (socket says 172.21.15.42:44430) 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5b9f7fe640 1 -- 172.21.15.42:0/1490898728 learned_addr learned my addr 172.21.15.42:0/1490898728 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5b97fff640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba01023b0 0x7f5ba0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.200+0000 7f5b9effd640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ba01011c0 0x7f5ba01971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9f7fe640 1 -- 172.21.15.42:0/1490898728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ba01011c0 msgr2=0x7f5ba01971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9f7fe640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ba01011c0 0x7f5ba01971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9f7fe640 1 -- 172.21.15.42:0/1490898728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba01023b0 msgr2=0x7f5ba0197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9f7fe640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba01023b0 0x7f5ba0197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9f7fe640 1 -- 172.21.15.42:0/1490898728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b88009560 con 0x7f5ba0102d70 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9effd640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ba01011c0 0x7f5ba01971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9f7fe640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba0102d70 0x7f5ba019e790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fd0 tx=0x7f5b9800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:51.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9cff9640 1 -- 172.21.15.42:0/1490898728 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800bea0 con 0x7f5ba0102d70 2024-03-12T04:12:51.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9cff9640 1 -- 172.21.15.42:0/1490898728 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b98009940 con 0x7f5ba0102d70 2024-03-12T04:12:51.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5b9cff9640 1 -- 172.21.15.42:0/1490898728 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b980106c0 con 0x7f5ba0102d70 2024-03-12T04:12:51.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba019f2e0 con 0x7f5ba0102d70 2024-03-12T04:12:51.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.201+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba019f770 con 0x7f5ba0102d70 2024-03-12T04:12:51.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.202+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba0108600 con 0x7f5ba0102d70 2024-03-12T04:12:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:50 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] up:boot 2024-03-12T04:12:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:50 smithi136 ceph-mon[35571]: fsmap 12 up:standby 2024-03-12T04:12:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:50 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:12:51.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:50 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.203+0000 7f5b9cff9640 1 -- 172.21.15.42:0/1490898728 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5b98003970 con 0x7f5ba0102d70 2024-03-12T04:12:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.203+0000 7f5b9cff9640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5b8c0a1a30 0x7f5b8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.204+0000 7f5b9effd640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5b8c0a1a30 0x7f5b8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.204+0000 7f5b9cff9640 1 -- 172.21.15.42:0/1490898728 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b980ccd10 con 0x7f5ba0102d70 2024-03-12T04:12:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.204+0000 7f5b9effd640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5b8c0a1a30 0x7f5b8c0a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0102210 tx=0x7f5b90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.208+0000 7f5b9cff9640 1 -- 172.21.15.42:0/1490898728 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b980d2020 con 0x7f5ba0102d70 2024-03-12T04:12:51.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.437+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ba002ce40 con 0x7f5ba0102d70 2024-03-12T04:12:51.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.438+0000 7f5b9cff9640 1 -- 172.21.15.42:0/1490898728 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18456 (secure 0 0 0) 0x7f5b98096320 con 0x7f5ba0102d70 2024-03-12T04:12:51.440 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:51.441 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:02.144274+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:12:51.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5b8c0a1a30 msgr2=0x7f5b8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5b8c0a1a30 0x7f5b8c0a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0102210 tx=0x7f5b90005f90 comp rx=0 tx=0).stop 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba0102d70 msgr2=0x7f5ba019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba0102d70 0x7f5ba019e790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fd0 tx=0x7f5b9800b260 comp rx=0 tx=0).stop 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 shutdown_connections 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ba0102d70 0x7f5ba019e790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5b8c0a1a30 0x7f5b8c0a3ef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba01023b0 0x7f5ba0197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 --2- 172.21.15.42:0/1490898728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ba01011c0 0x7f5ba01971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 >> 172.21.15.42:0/1490898728 conn(0x7f5ba00fc730 msgr2=0x7f5ba01057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 shutdown_connections 2024-03-12T04:12:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.440+0000 7f5ba549a640 1 -- 172.21.15.42:0/1490898728 wait complete. 2024-03-12T04:12:51.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:12:51.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:12:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 -- 172.21.15.42:0/3583174770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341023a0 msgr2=0x7f8934102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 --2- 172.21.15.42:0/3583174770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341023a0 0x7f8934102820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8930009900 tx=0x7f893002bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 -- 172.21.15.42:0/3583174770 shutdown_connections 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 --2- 172.21.15.42:0/3583174770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934102d60 0x7f89341095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 --2- 172.21.15.42:0/3583174770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341023a0 0x7f8934102820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 --2- 172.21.15.42:0/3583174770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89341011b0 0x7f89341015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 -- 172.21.15.42:0/3583174770 >> 172.21.15.42:0/3583174770 conn(0x7f89340fc720 msgr2=0x7f89340feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 -- 172.21.15.42:0/3583174770 shutdown_connections 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 -- 172.21.15.42:0/3583174770 wait complete. 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.559+0000 7f893bb77640 1 Processor -- start 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893bb77640 1 -- start start 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893bb77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341011b0 0x7f8934074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893bb77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89341023a0 0x7f8934073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893bb77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934102d60 0x7f8934073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893bb77640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934074210 con 0x7f89341011b0 2024-03-12T04:12:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893bb77640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934074380 con 0x7f8934102d60 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893bb77640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89341a11f0 con 0x7f89341023a0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341011b0 0x7f8934074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341011b0 0x7f8934074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54364/0 (socket says 172.21.15.42:54364) 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 -- 172.21.15.42:0/3220581690 learned_addr learned my addr 172.21.15.42:0/3220581690 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f893a114640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934102d60 0x7f8934073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939112640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89341023a0 0x7f8934073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 -- 172.21.15.42:0/3220581690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89341023a0 msgr2=0x7f8934073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89341023a0 0x7f8934073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 -- 172.21.15.42:0/3220581690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934102d60 msgr2=0x7f8934073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934102d60 0x7f8934073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 -- 172.21.15.42:0/3220581690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8930009560 con 0x7f89341011b0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.560+0000 7f8939913640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341011b0 0x7f8934074f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f892400d980 tx=0x7f892400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.561+0000 7f892affd640 1 -- 172.21.15.42:0/3220581690 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924009c30 con 0x7f89341011b0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.561+0000 7f892affd640 1 -- 172.21.15.42:0/3220581690 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8924009d90 con 0x7f89341011b0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.561+0000 7f892affd640 1 -- 172.21.15.42:0/3220581690 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892400f720 con 0x7f89341011b0 2024-03-12T04:12:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.561+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89341a14a0 con 0x7f89341011b0 2024-03-12T04:12:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.561+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89341a19d0 con 0x7f89341011b0 2024-03-12T04:12:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.562+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89341085a0 con 0x7f89341011b0 2024-03-12T04:12:51.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.563+0000 7f892affd640 1 -- 172.21.15.42:0/3220581690 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8924010460 con 0x7f89341011b0 2024-03-12T04:12:51.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.563+0000 7f892affd640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89200a1ab0 0x7f89200a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.564+0000 7f8939112640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89200a1ab0 0x7f89200a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.564+0000 7f892affd640 1 -- 172.21.15.42:0/3220581690 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89240cce20 con 0x7f89341011b0 2024-03-12T04:12:51.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.564+0000 7f8939112640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89200a1ab0 0x7f89200a3f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8930009230 tx=0x7f8930005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:51.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.567+0000 7f892affd640 1 -- 172.21.15.42:0/3220581690 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8924096550 con 0x7f89341011b0 2024-03-12T04:12:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.750+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f893410cc20 con 0x7f89341011b0 2024-03-12T04:12:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.750+0000 7f892affd640 1 -- 172.21.15.42:0/3220581690 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8924099c10 con 0x7f89341011b0 2024-03-12T04:12:51.753 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89200a1ab0 msgr2=0x7f89200a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89200a1ab0 0x7f89200a3f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8930009230 tx=0x7f8930005fd0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341011b0 msgr2=0x7f8934074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341011b0 0x7f8934074f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f892400d980 tx=0x7f892400de40 comp rx=0 tx=0).stop 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 shutdown_connections 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89200a1ab0 0x7f89200a3f70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934102d60 0x7f8934073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89341023a0 0x7f8934073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 --2- 172.21.15.42:0/3220581690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89341011b0 0x7f8934074f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 >> 172.21.15.42:0/3220581690 conn(0x7f89340fc720 msgr2=0x7f8934105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:51.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 shutdown_connections 2024-03-12T04:12:51.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.752+0000 7f893bb77640 1 -- 172.21.15.42:0/3220581690 wait complete. 2024-03-12T04:12:51.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 -- 172.21.15.42:0/43097469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 msgr2=0x7f8b441028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 --2- 172.21.15.42:0/43097469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b441028a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009900 tx=0x7f8b3802bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 -- 172.21.15.42:0/43097469 shutdown_connections 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 --2- 172.21.15.42:0/43097469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b44102de0 0x7f8b44109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 --2- 172.21.15.42:0/43097469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b441028a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 --2- 172.21.15.42:0/43097469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b440ff450 0x7f8b440ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 -- 172.21.15.42:0/43097469 >> 172.21.15.42:0/43097469 conn(0x7f8b440fad80 msgr2=0x7f8b440fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 -- 172.21.15.42:0/43097469 shutdown_connections 2024-03-12T04:12:51.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 -- 172.21.15.42:0/43097469 wait complete. 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.874+0000 7f8b49e84640 1 Processor -- start 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b49e84640 1 -- start start 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b49e84640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b440ff450 0x7f8b44197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b49e84640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b44197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b49e84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b44102de0 0x7f8b4419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b49e84640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4419ec10 con 0x7f8b44102420 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b49e84640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4419ed80 con 0x7f8b440ff450 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b49e84640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4419ef40 con 0x7f8b44102de0 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b44197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b44197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54388/0 (socket says 172.21.15.42:54388) 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 -- 172.21.15.42:0/796847314 learned_addr learned my addr 172.21.15.42:0/796847314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b437fe640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b440ff450 0x7f8b44197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b43fff640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b44102de0 0x7f8b4419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 -- 172.21.15.42:0/796847314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b44102de0 msgr2=0x7f8b4419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b44102de0 0x7f8b4419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 -- 172.21.15.42:0/796847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b440ff450 msgr2=0x7f8b44197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b440ff450 0x7f8b44197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.875+0000 7f8b42ffd640 1 -- 172.21.15.42:0/796847314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009560 con 0x7f8b44102420 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.876+0000 7f8b42ffd640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b44197660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8b38036040 tx=0x7f8b38004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.876+0000 7f8b40ff9640 1 -- 172.21.15.42:0/796847314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38004aa0 con 0x7f8b44102420 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.876+0000 7f8b40ff9640 1 -- 172.21.15.42:0/796847314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b38003cb0 con 0x7f8b44102420 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.876+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4419f1c0 con 0x7f8b44102420 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.876+0000 7f8b40ff9640 1 -- 172.21.15.42:0/796847314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3803e450 con 0x7f8b44102420 2024-03-12T04:12:51.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.876+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4419f6d0 con 0x7f8b44102420 2024-03-12T04:12:51.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.877+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b44108640 con 0x7f8b44102420 2024-03-12T04:12:51.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.878+0000 7f8b40ff9640 1 -- 172.21.15.42:0/796847314 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8b38004e50 con 0x7f8b44102420 2024-03-12T04:12:51.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.879+0000 7f8b40ff9640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b2c0a1a30 0x7f8b2c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:51.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.879+0000 7f8b437fe640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b2c0a1a30 0x7f8b2c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:51.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.879+0000 7f8b40ff9640 1 -- 172.21.15.42:0/796847314 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b380efa30 con 0x7f8b44102420 2024-03-12T04:12:51.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.880+0000 7f8b437fe640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b2c0a1a30 0x7f8b2c0a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005fd0 tx=0x7f8b3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:51.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:51.884+0000 7f8b40ff9640 1 -- 172.21.15.42:0/796847314 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b380b9040 con 0x7f8b44102420 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/756482625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:52.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:52.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:51 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1490898728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.078+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8b4410ccb0 con 0x7f8b44102420 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.078+0000 7f8b40ff9640 1 -- 172.21.15.42:0/796847314 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8b380bc700 con 0x7f8b44102420 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:12:52.081 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-03-12T04:12:52.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b2c0a1a30 msgr2=0x7f8b2c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b2c0a1a30 0x7f8b2c0a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005fd0 tx=0x7f8b3400b390 comp rx=0 tx=0).stop 2024-03-12T04:12:52.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 msgr2=0x7f8b44197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b44197660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8b38036040 tx=0x7f8b38004370 comp rx=0 tx=0).stop 2024-03-12T04:12:52.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 shutdown_connections 2024-03-12T04:12:52.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b44102de0 0x7f8b4419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b44102420 0x7f8b44197660 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b2c0a1a30 0x7f8b2c0a3ef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 --2- 172.21.15.42:0/796847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b440ff450 0x7f8b44197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 >> 172.21.15.42:0/796847314 conn(0x7f8b440fad80 msgr2=0x7f8b44105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:52.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 shutdown_connections 2024-03-12T04:12:52.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.080+0000 7f8b49e84640 1 -- 172.21.15.42:0/796847314 wait complete. 2024-03-12T04:12:52.093 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-12T04:12:52.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-12T04:12:52.145 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/756482625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:51 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1490898728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.159 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/756482625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:51 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1490898728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:52.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 -- 172.21.15.42:0/3138661217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c100080 msgr2=0x7f346c1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 --2- 172.21.15.42:0/3138661217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c100080 0x7f346c1004a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 -- 172.21.15.42:0/3138661217 shutdown_connections 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 --2- 172.21.15.42:0/3138661217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c101c30 0x7f346c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 --2- 172.21.15.42:0/3138661217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f346c101270 0x7f346c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 --2- 172.21.15.42:0/3138661217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c100080 0x7f346c1004a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 -- 172.21.15.42:0/3138661217 >> 172.21.15.42:0/3138661217 conn(0x7f346c0fb610 msgr2=0x7f346c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 -- 172.21.15.42:0/3138661217 shutdown_connections 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.199+0000 7f34713d0640 1 -- 172.21.15.42:0/3138661217 wait complete. 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 Processor -- start 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 -- start start 2024-03-12T04:12:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f346c100080 0x7f346c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c101270 0x7f346c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c101c30 0x7f346c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c19ed00 con 0x7f346c101c30 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c19ee70 con 0x7f346c100080 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f34713d0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c19f030 con 0x7f346c101270 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c101c30 0x7f346c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c101c30 0x7f346c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54410/0 (socket says 172.21.15.42:54410) 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 -- 172.21.15.42:0/236048852 learned_addr learned my addr 172.21.15.42:0/236048852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346affd640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f346c100080 0x7f346c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346a7fc640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c101270 0x7f346c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 -- 172.21.15.42:0/236048852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c101270 msgr2=0x7f346c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c101270 0x7f346c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 -- 172.21.15.42:0/236048852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f346c100080 msgr2=0x7f346c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f346c100080 0x7f346c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346b7fe640 1 -- 172.21.15.42:0/236048852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c009560 con 0x7f346c101c30 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.200+0000 7f346affd640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f346c100080 0x7f346c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.201+0000 7f346b7fe640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c101c30 0x7f346c19e7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3464009fd0 tx=0x7f346400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.201+0000 7f3463fff640 1 -- 172.21.15.42:0/236048852 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400bea0 con 0x7f346c101c30 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.201+0000 7f3463fff640 1 -- 172.21.15.42:0/236048852 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3464009940 con 0x7f346c101c30 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.201+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c19f310 con 0x7f346c101c30 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.201+0000 7f3463fff640 1 -- 172.21.15.42:0/236048852 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640106c0 con 0x7f346c101c30 2024-03-12T04:12:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.201+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c19f7d0 con 0x7f346c101c30 2024-03-12T04:12:52.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.202+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c10b900 con 0x7f346c101c30 2024-03-12T04:12:52.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.204+0000 7f3463fff640 1 -- 172.21.15.42:0/236048852 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3464003970 con 0x7f346c101c30 2024-03-12T04:12:52.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.204+0000 7f3463fff640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34580a1a30 0x7f34580a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.205+0000 7f346affd640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34580a1a30 0x7f34580a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.205+0000 7f3463fff640 1 -- 172.21.15.42:0/236048852 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34640ccbc0 con 0x7f346c101c30 2024-03-12T04:12:52.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.205+0000 7f346affd640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34580a1a30 0x7f34580a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f345c02c7d0 tx=0x7f345c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.209+0000 7f3463fff640 1 -- 172.21.15.42:0/236048852 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3464096320 con 0x7f346c101c30 2024-03-12T04:12:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.395+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f346c19fc70 con 0x7f346c101c30 2024-03-12T04:12:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.751+0000 7f3463fff640 1 -- 172.21.15.42:0/236048852 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f34640999e0 con 0x7f346c101c30 2024-03-12T04:12:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34580a1a30 msgr2=0x7f34580a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34580a1a30 0x7f34580a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f345c02c7d0 tx=0x7f345c005fd0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c101c30 msgr2=0x7f346c19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c101c30 0x7f346c19e7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3464009fd0 tx=0x7f346400b260 comp rx=0 tx=0).stop 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 shutdown_connections 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f346c101c30 0x7f346c19e7c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c101270 0x7f346c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34580a1a30 0x7f34580a3ef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 --2- 172.21.15.42:0/236048852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f346c100080 0x7f346c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 >> 172.21.15.42:0/236048852 conn(0x7f346c0fb610 msgr2=0x7f346c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 shutdown_connections 2024-03-12T04:12:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.754+0000 7f34713d0640 1 -- 172.21.15.42:0/236048852 wait complete. 2024-03-12T04:12:52.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-12T04:12:52.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 -- 172.21.15.42:0/2886554533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 msgr2=0x7fa1e0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/2886554533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e0106940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009900 tx=0x7fa1d002bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 -- 172.21.15.42:0/2886554533 shutdown_connections 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/2886554533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1e0106e80 0x7fa1e01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/2886554533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e0106940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/2886554533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e0101c20 0x7fa1e0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 -- 172.21.15.42:0/2886554533 >> 172.21.15.42:0/2886554533 conn(0x7fa1e00fb650 msgr2=0x7fa1e00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 -- 172.21.15.42:0/2886554533 shutdown_connections 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.892+0000 7fa1e72f4640 1 -- 172.21.15.42:0/2886554533 wait complete. 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 Processor -- start 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 -- start start 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1e0101c20 0x7fa1e0197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e01976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e0106e80 0x7fa1e019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e019ec60 con 0x7fa1e0104550 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e019edd0 con 0x7fa1e0106e80 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e72f4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e019ef90 con 0x7fa1e0101c20 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e488f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e01976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e488f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e01976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54428/0 (socket says 172.21.15.42:54428) 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e488f640 1 -- 172.21.15.42:0/1351497647 learned_addr learned my addr 172.21.15.42:0/1351497647 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e5891640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e0106e80 0x7fa1e019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.893+0000 7fa1e5090640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1e0101c20 0x7fa1e0197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e488f640 1 -- 172.21.15.42:0/1351497647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1e0101c20 msgr2=0x7fa1e0197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e488f640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1e0101c20 0x7fa1e0197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e488f640 1 -- 172.21.15.42:0/1351497647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e0106e80 msgr2=0x7fa1e019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e488f640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e0106e80 0x7fa1e019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:52.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e488f640 1 -- 172.21.15.42:0/1351497647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d0009560 con 0x7fa1e0104550 2024-03-12T04:12:52.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e5891640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e0106e80 0x7fa1e019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:12:52.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e488f640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e01976b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1d002c7d0 tx=0x7fa1d0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:52.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1d67fc640 1 -- 172.21.15.42:0/1351497647 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0004500 con 0x7fa1e0104550 2024-03-12T04:12:52.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e019f210 con 0x7fa1e0104550 2024-03-12T04:12:52.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1d67fc640 1 -- 172.21.15.42:0/1351497647 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1d002de90 con 0x7fa1e0104550 2024-03-12T04:12:52.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1d67fc640 1 -- 172.21.15.42:0/1351497647 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0005350 con 0x7fa1e0104550 2024-03-12T04:12:52.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.894+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e019f720 con 0x7fa1e0104550 2024-03-12T04:12:52.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.895+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e00fd140 con 0x7fa1e0104550 2024-03-12T04:12:52.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.897+0000 7fa1d67fc640 1 -- 172.21.15.42:0/1351497647 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa1d00054b0 con 0x7fa1e0104550 2024-03-12T04:12:52.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.897+0000 7fa1d67fc640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa1cc0a1a30 0x7fa1cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:52.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.897+0000 7fa1d67fc640 1 -- 172.21.15.42:0/1351497647 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fa1d00ef140 con 0x7fa1e0104550 2024-03-12T04:12:52.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.897+0000 7fa1e5090640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa1cc0a1a30 0x7fa1cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:52.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.898+0000 7fa1e5090640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa1cc0a1a30 0x7fa1cc0a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009bd0 tx=0x7fa1dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:52.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:52.902+0000 7fa1d67fc640 1 -- 172.21.15.42:0/1351497647 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1d00b8680 con 0x7fa1e0104550 2024-03-12T04:12:53.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:52 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3220581690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:53.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:52 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/796847314' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-12T04:12:53.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:52 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/236048852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-12T04:12:53.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.132+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa1e010ca60 con 0x7fa1e0104550 2024-03-12T04:12:53.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:52 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3220581690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:53.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:52 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/796847314' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-12T04:12:53.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:52 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/236048852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-12T04:12:53.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:52 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3220581690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:12:53.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:52 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/796847314' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-12T04:12:53.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:52 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/236048852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-12T04:12:53.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.756+0000 7fa1d67fc640 1 -- 172.21.15.42:0/1351497647 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa1d00bbd40 con 0x7fa1e0104550 2024-03-12T04:12:53.759 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa1cc0a1a30 msgr2=0x7fa1cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa1cc0a1a30 0x7fa1cc0a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009bd0 tx=0x7fa1dc005f90 comp rx=0 tx=0).stop 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 msgr2=0x7fa1e01976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e01976b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1d002c7d0 tx=0x7fa1d0004040 comp rx=0 tx=0).stop 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 shutdown_connections 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa1cc0a1a30 0x7fa1cc0a3ef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e0106e80 0x7fa1e019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e0104550 0x7fa1e01976b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 --2- 172.21.15.42:0/1351497647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1e0101c20 0x7fa1e0197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.759+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 >> 172.21.15.42:0/1351497647 conn(0x7fa1e00fb650 msgr2=0x7fa1e0102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.760+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 shutdown_connections 2024-03-12T04:12:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.760+0000 7fa1e72f4640 1 -- 172.21.15.42:0/1351497647 wait complete. 2024-03-12T04:12:53.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.885+0000 7f269b40b640 1 -- 172.21.15.42:0/779611361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694101270 msgr2=0x7f26941016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.885+0000 7f269b40b640 1 --2- 172.21.15.42:0/779611361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694101270 0x7f26941016f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2684009900 tx=0x7f268402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.885+0000 7f269b40b640 1 -- 172.21.15.42:0/779611361 shutdown_connections 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.885+0000 7f269b40b640 1 --2- 172.21.15.42:0/779611361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694101c30 0x7f26941084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.885+0000 7f269b40b640 1 --2- 172.21.15.42:0/779611361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694101270 0x7f26941016f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.885+0000 7f269b40b640 1 --2- 172.21.15.42:0/779611361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2694100080 0x7f26941004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.885+0000 7f269b40b640 1 -- 172.21.15.42:0/779611361 >> 172.21.15.42:0/779611361 conn(0x7f26940fb610 msgr2=0x7f26940fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 -- 172.21.15.42:0/779611361 shutdown_connections 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 -- 172.21.15.42:0/779611361 wait complete. 2024-03-12T04:12:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 Processor -- start 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 -- start start 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694100080 0x7f26941960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2694101270 0x7f2694196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694101c30 0x7f269419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419dbd0 con 0x7f2694100080 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419dd40 con 0x7f2694101270 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f269b40b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419df00 con 0x7f2694101c30 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f26991a7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694100080 0x7f26941960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f26991a7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694100080 0x7f26941960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54448/0 (socket says 172.21.15.42:54448) 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.886+0000 7f26991a7640 1 -- 172.21.15.42:0/2226594480 learned_addr learned my addr 172.21.15.42:0/2226594480 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26989a6640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2694101270 0x7f2694196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26991a7640 1 -- 172.21.15.42:0/2226594480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694101c30 msgr2=0x7f269419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26999a8640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694101c30 0x7f269419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26991a7640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694101c30 0x7f269419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26991a7640 1 -- 172.21.15.42:0/2226594480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2694101270 msgr2=0x7f2694196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26991a7640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2694101270 0x7f2694196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26991a7640 1 -- 172.21.15.42:0/2226594480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2684009560 con 0x7f2694100080 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26989a6640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2694101270 0x7f2694196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f26991a7640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694100080 0x7f26941960e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f269000c970 tx=0x7f269000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f268a7fc640 1 -- 172.21.15.42:0/2226594480 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2690004220 con 0x7f2694100080 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f268a7fc640 1 -- 172.21.15.42:0/2226594480 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2690007e60 con 0x7f2694100080 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f268a7fc640 1 -- 172.21.15.42:0/2226594480 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2690003e40 con 0x7f2694100080 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269419e1e0 con 0x7f2694100080 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.887+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269419e770 con 0x7f2694100080 2024-03-12T04:12:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.888+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694107470 con 0x7f2694100080 2024-03-12T04:12:53.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.889+0000 7f268a7fc640 1 -- 172.21.15.42:0/2226594480 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2690003610 con 0x7f2694100080 2024-03-12T04:12:53.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.890+0000 7f268a7fc640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f26800a1a30 0x7f26800a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:53.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.890+0000 7f26989a6640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f26800a1a30 0x7f26800a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:53.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.890+0000 7f268a7fc640 1 -- 172.21.15.42:0/2226594480 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f26900cd080 con 0x7f2694100080 2024-03-12T04:12:53.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.890+0000 7f26989a6640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f26800a1a30 0x7f26800a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2684036040 tx=0x7f2684000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:53.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:53.894+0000 7f268a7fc640 1 -- 172.21.15.42:0/2226594480 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26900963a0 con 0x7f2694100080 2024-03-12T04:12:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/236048852' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-12T04:12:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: osdmap e84: 12 total, 12 up, 12 in 2024-03-12T04:12:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: pgmap v288: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:54.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:54.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:53 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1351497647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-12T04:12:54.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.114+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f269410b980 con 0x7f2694100080 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/236048852' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: osdmap e84: 12 total, 12 up, 12 in 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: pgmap v288: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:54.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:53 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1351497647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/236048852' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: osdmap e84: 12 total, 12 up, 12 in 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: pgmap v288: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:12:54.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:12:54.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:12:54.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:53 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1351497647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-12T04:12:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.758+0000 7f268a7fc640 1 -- 172.21.15.42:0/2226594480 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2690099a60 con 0x7f2694100080 2024-03-12T04:12:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2024-03-12T04:12:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f26800a1a30 msgr2=0x7f26800a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f26800a1a30 0x7f26800a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2684036040 tx=0x7f2684000bc0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694100080 msgr2=0x7f26941960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694100080 0x7f26941960e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f269000c970 tx=0x7f269000ce30 comp rx=0 tx=0).stop 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 shutdown_connections 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694101c30 0x7f269419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f26800a1a30 0x7f26800a3ef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2694101270 0x7f2694196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 --2- 172.21.15.42:0/2226594480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2694100080 0x7f26941960e0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f269000c970 tx=0x7f269000ce30 comp rx=0 tx=0).stop 2024-03-12T04:12:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.762+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 >> 172.21.15.42:0/2226594480 conn(0x7f26940fb610 msgr2=0x7f2694104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.764+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 shutdown_connections 2024-03-12T04:12:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.764+0000 7f269b40b640 1 -- 172.21.15.42:0/2226594480 wait complete. 2024-03-12T04:12:54.789 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 -- 172.21.15.42:0/1610974504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 msgr2=0x7f12d4101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/1610974504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4101710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009900 tx=0x7f12c402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 -- 172.21.15.42:0/1610974504 shutdown_connections 2024-03-12T04:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/1610974504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12d4101c50 0x7f12d41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/1610974504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4101710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/1610974504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12d41000a0 0x7f12d41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 -- 172.21.15.42:0/1610974504 >> 172.21.15.42:0/1610974504 conn(0x7f12d40fb610 msgr2=0x7f12d40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 -- 172.21.15.42:0/1610974504 shutdown_connections 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.897+0000 7f12dc7bc640 1 -- 172.21.15.42:0/1610974504 wait complete. 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 Processor -- start 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 -- start start 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12d41000a0 0x7f12d4104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12d4101c50 0x7f12d4102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4103310 con 0x7f12d4101290 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4103480 con 0x7f12d4101c50 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dc7bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d41a1260 con 0x7f12d41000a0 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54458/0 (socket says 172.21.15.42:54458) 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 -- 172.21.15.42:0/661744716 learned_addr learned my addr 172.21.15.42:0/661744716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12da558640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12d41000a0 0x7f12d4104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:54.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dad59640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12d4101c50 0x7f12d4102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 -- 172.21.15.42:0/661744716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12d41000a0 msgr2=0x7f12d4104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12d41000a0 0x7f12d4104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 -- 172.21.15.42:0/661744716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12d4101c50 msgr2=0x7f12d4102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12d4101c50 0x7f12d4102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 -- 172.21.15.42:0/661744716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009560 con 0x7f12d4101290 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12dad59640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12d4101c50 0x7f12d4102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.898+0000 7f12d9d57640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4102670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006010 tx=0x7f12c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.899+0000 7f12cb7fe640 1 -- 172.21.15.42:0/661744716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40044e0 con 0x7f12d4101290 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.899+0000 7f12cb7fe640 1 -- 172.21.15.42:0/661744716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f12c402de90 con 0x7f12d4101290 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.899+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d41a14b0 con 0x7f12d4101290 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.899+0000 7f12cb7fe640 1 -- 172.21.15.42:0/661744716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4003cd0 con 0x7f12d4101290 2024-03-12T04:12:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.899+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d41a1a10 con 0x7f12d4101290 2024-03-12T04:12:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.900+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d410b960 con 0x7f12d4101290 2024-03-12T04:12:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.901+0000 7f12cb7fe640 1 -- 172.21.15.42:0/661744716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f12c4004fd0 con 0x7f12d4101290 2024-03-12T04:12:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.901+0000 7f12cb7fe640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12c00a1ab0 0x7f12c00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.902+0000 7f12da558640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12c00a1ab0 0x7f12c00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.902+0000 7f12cb7fe640 1 -- 172.21.15.42:0/661744716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f12c40ef4b0 con 0x7f12d4101290 2024-03-12T04:12:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.902+0000 7f12da558640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12c00a1ab0 0x7f12c00a3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f12d00058c0 tx=0x7f12d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:54.906+0000 7f12cb7fe640 1 -- 172.21.15.42:0/661744716 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12c40b8770 con 0x7f12d4101290 2024-03-12T04:12:55.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:54 smithi042 ceph-mon[32260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-12T04:12:55.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:54 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1351497647' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-12T04:12:55.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:54 smithi042 ceph-mon[32260]: osdmap e85: 12 total, 12 up, 12 in 2024-03-12T04:12:55.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:54 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2226594480' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-12T04:12:55.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:54 smithi042 ceph-mon[32260]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-12T04:12:55.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.088+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12d410baf0 con 0x7f12d4101290 2024-03-12T04:12:55.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.089+0000 7f12cb7fe640 1 -- 172.21.15.42:0/661744716 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22232 (secure 0 0 0) 0x7f12c40bbe30 con 0x7f12d4101290 2024-03-12T04:12:55.092 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:55.092 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":86,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:54.115795+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:12:55.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12c00a1ab0 msgr2=0x7f12c00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12c00a1ab0 0x7f12c00a3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f12d00058c0 tx=0x7f12d000a2d0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 msgr2=0x7f12d4102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4102670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006010 tx=0x7f12c40042f0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 shutdown_connections 2024-03-12T04:12:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12c00a1ab0 0x7f12c00a3f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12d4101c50 0x7f12d4102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12d4101290 0x7f12d4102670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 --2- 172.21.15.42:0/661744716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12d41000a0 0x7f12d4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 >> 172.21.15.42:0/661744716 conn(0x7f12d40fb610 msgr2=0x7f12d40fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 shutdown_connections 2024-03-12T04:12:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.091+0000 7f12dc7bc640 1 -- 172.21.15.42:0/661744716 wait complete. 2024-03-12T04:12:55.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:12:55.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:54 smithi136 ceph-mon[35571]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-12T04:12:55.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:54 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1351497647' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-12T04:12:55.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:54 smithi136 ceph-mon[35571]: osdmap e85: 12 total, 12 up, 12 in 2024-03-12T04:12:55.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:54 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2226594480' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-12T04:12:55.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:54 smithi136 ceph-mon[35571]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-12T04:12:55.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:54 smithi119 ceph-mon[36119]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-12T04:12:55.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:54 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1351497647' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-12T04:12:55.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:54 smithi119 ceph-mon[36119]: osdmap e85: 12 total, 12 up, 12 in 2024-03-12T04:12:55.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:54 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2226594480' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-12T04:12:55.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:54 smithi119 ceph-mon[36119]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-12T04:12:55.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 -- 172.21.15.42:0/440789495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c106e80 msgr2=0x7f395c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 --2- 172.21.15.42:0/440789495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c106e80 0x7f395c1092d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3954009900 tx=0x7f395402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:55.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 -- 172.21.15.42:0/440789495 shutdown_connections 2024-03-12T04:12:55.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 --2- 172.21.15.42:0/440789495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c106e80 0x7f395c1092d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 --2- 172.21.15.42:0/440789495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395c104550 0x7f395c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 --2- 172.21.15.42:0/440789495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f395c101c20 0x7f395c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 -- 172.21.15.42:0/440789495 >> 172.21.15.42:0/440789495 conn(0x7f395c0fb610 msgr2=0x7f395c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 -- 172.21.15.42:0/440789495 shutdown_connections 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.205+0000 7f3961335640 1 -- 172.21.15.42:0/440789495 wait complete. 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 Processor -- start 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 -- start start 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c101c20 0x7f395c197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f395c104550 0x7f395c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395c106e80 0x7f395c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19ec20 con 0x7f395c101c20 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19ed90 con 0x7f395c104550 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f3961335640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19ef50 con 0x7f395c106e80 2024-03-12T04:12:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c101c20 0x7f395c197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c101c20 0x7f395c197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54470/0 (socket says 172.21.15.42:54470) 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 -- 172.21.15.42:0/4033941409 learned_addr learned my addr 172.21.15.42:0/4033941409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395a7fc640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f395c104550 0x7f395c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395b7fe640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395c106e80 0x7f395c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 -- 172.21.15.42:0/4033941409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395c106e80 msgr2=0x7f395c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395c106e80 0x7f395c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 -- 172.21.15.42:0/4033941409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f395c104550 msgr2=0x7f395c197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f395c104550 0x7f395c197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.206+0000 7f395affd640 1 -- 172.21.15.42:0/4033941409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954009560 con 0x7f395c101c20 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.207+0000 7f395a7fc640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f395c104550 0x7f395c197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.207+0000 7f395affd640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c101c20 0x7f395c197130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f394c00ebc0 tx=0x7f394c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.207+0000 7f3943fff640 1 -- 172.21.15.42:0/4033941409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0099b0 con 0x7f395c101c20 2024-03-12T04:12:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.207+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c19f230 con 0x7f395c101c20 2024-03-12T04:12:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.207+0000 7f3943fff640 1 -- 172.21.15.42:0/4033941409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f394c00eec0 con 0x7f395c101c20 2024-03-12T04:12:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.207+0000 7f3943fff640 1 -- 172.21.15.42:0/4033941409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c018a80 con 0x7f395c101c20 2024-03-12T04:12:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.207+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c19f790 con 0x7f395c101c20 2024-03-12T04:12:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.208+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c0fd100 con 0x7f395c101c20 2024-03-12T04:12:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.209+0000 7f3943fff640 1 -- 172.21.15.42:0/4033941409 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f394c018be0 con 0x7f395c101c20 2024-03-12T04:12:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.209+0000 7f3943fff640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39440a1a30 0x7f39440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.210+0000 7f395a7fc640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39440a1a30 0x7f39440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.210+0000 7f3943fff640 1 -- 172.21.15.42:0/4033941409 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f394c014070 con 0x7f395c101c20 2024-03-12T04:12:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.210+0000 7f395a7fc640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39440a1a30 0x7f39440a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3950005fd0 tx=0x7f3950005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:55.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.214+0000 7f3943fff640 1 -- 172.21.15.42:0/4033941409 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f394c052d20 con 0x7f395c101c20 2024-03-12T04:12:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.394+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f395c10ca60 con 0x7f395c101c20 2024-03-12T04:12:55.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.396+0000 7f3943fff640 1 -- 172.21.15.42:0/4033941409 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22232 (secure 0 0 0) 0x7f394c096d30 con 0x7f395c101c20 2024-03-12T04:12:55.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:12:55.399 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":86,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:54.115795+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39440a1a30 msgr2=0x7f39440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39440a1a30 0x7f39440a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3950005fd0 tx=0x7f3950005ea0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c101c20 msgr2=0x7f395c197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c101c20 0x7f395c197130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f394c00ebc0 tx=0x7f394c00c6a0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 shutdown_connections 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395c106e80 0x7f395c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39440a1a30 0x7f39440a3ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f395c104550 0x7f395c197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 --2- 172.21.15.42:0/4033941409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f395c101c20 0x7f395c197130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 >> 172.21.15.42:0/4033941409 conn(0x7f395c0fb610 msgr2=0x7f395c102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 shutdown_connections 2024-03-12T04:12:55.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.398+0000 7f3961335640 1 -- 172.21.15.42:0/4033941409 wait complete. 2024-03-12T04:12:55.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-12T04:12:55.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 -- 172.21.15.42:0/4042593460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101290 msgr2=0x7f8b20101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 --2- 172.21.15.42:0/4042593460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101290 0x7f8b20101710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009900 tx=0x7f8b1402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 -- 172.21.15.42:0/4042593460 shutdown_connections 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 --2- 172.21.15.42:0/4042593460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b20101c50 0x7f8b201084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 --2- 172.21.15.42:0/4042593460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101290 0x7f8b20101710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 --2- 172.21.15.42:0/4042593460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b201000a0 0x7f8b201004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 -- 172.21.15.42:0/4042593460 >> 172.21.15.42:0/4042593460 conn(0x7f8b200fb630 msgr2=0x7f8b200fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 -- 172.21.15.42:0/4042593460 shutdown_connections 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.515+0000 7f8b27b99640 1 -- 172.21.15.42:0/4042593460 wait complete. 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 Processor -- start 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 -- start start 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b201000a0 0x7f8b201970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b20101290 0x7f8b20197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101c50 0x7f8b2019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2019ebe0 con 0x7f8b20101c50 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2019ed50 con 0x7f8b201000a0 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b27b99640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2019ef10 con 0x7f8b20101290 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101c50 0x7f8b2019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101c50 0x7f8b2019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54480/0 (socket says 172.21.15.42:54480) 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 -- 172.21.15.42:0/3393063262 learned_addr learned my addr 172.21.15.42:0/3393063262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b25935640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b201000a0 0x7f8b201970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b25134640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b20101290 0x7f8b20197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 -- 172.21.15.42:0/3393063262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b20101290 msgr2=0x7f8b20197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b20101290 0x7f8b20197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 -- 172.21.15.42:0/3393063262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b201000a0 msgr2=0x7f8b201970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b201000a0 0x7f8b201970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.516+0000 7f8b26136640 1 -- 172.21.15.42:0/3393063262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b14009560 con 0x7f8b20101c50 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.517+0000 7f8b26136640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101c50 0x7f8b2019e6a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800d750 tx=0x7f8b1800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.517+0000 7f8b1effd640 1 -- 172.21.15.42:0/3393063262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18009bd0 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.517+0000 7f8b1effd640 1 -- 172.21.15.42:0/3393063262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b1800de60 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.517+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2019f1f0 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.517+0000 7f8b1effd640 1 -- 172.21.15.42:0/3393063262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18004e00 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.517+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2019f6d0 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.518+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b201073c0 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.519+0000 7f8b1effd640 1 -- 172.21.15.42:0/3393063262 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8b18004f60 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.519+0000 7f8b1effd640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b080a1a30 0x7f8b080a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.520+0000 7f8b25935640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b080a1a30 0x7f8b080a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.520+0000 7f8b1effd640 1 -- 172.21.15.42:0/3393063262 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8b180ccb80 con 0x7f8b20101c50 2024-03-12T04:12:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.520+0000 7f8b25935640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b080a1a30 0x7f8b080a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b201010f0 tx=0x7f8b10007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.524+0000 7f8b1effd640 1 -- 172.21.15.42:0/3393063262 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b18095f90 con 0x7f8b20101c50 2024-03-12T04:12:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:55.745+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8b2010ca60 con 0x7f8b20101c50 2024-03-12T04:12:56.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.023+0000 7f8b1effd640 1 -- 172.21.15.42:0/3393063262 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8b180170b0 con 0x7f8b20101c50 2024-03-12T04:12:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.025+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b080a1a30 msgr2=0x7f8b080a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.025+0000 7f8b27b99640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b080a1a30 0x7f8b080a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b201010f0 tx=0x7f8b10007600 comp rx=0 tx=0).stop 2024-03-12T04:12:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101c50 msgr2=0x7f8b2019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101c50 0x7f8b2019e6a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800d750 tx=0x7f8b1800dc10 comp rx=0 tx=0).stop 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 shutdown_connections 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b20101c50 0x7f8b2019e6a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b20101290 0x7f8b20197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8b080a1a30 0x7f8b080a3ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 --2- 172.21.15.42:0/3393063262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b201000a0 0x7f8b201970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 >> 172.21.15.42:0/3393063262 conn(0x7f8b200fb630 msgr2=0x7f8b20104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 shutdown_connections 2024-03-12T04:12:56.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.026+0000 7f8b27b99640 1 -- 172.21.15.42:0/3393063262 wait complete. 2024-03-12T04:12:56.039 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2226594480' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-12T04:12:56.039 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: osdmap e86: 12 total, 12 up, 12 in 2024-03-12T04:12:56.039 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-12T04:12:56.039 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-12T04:12:56.039 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: pgmap v291: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-12T04:12:56.039 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] up:active 2024-03-12T04:12:56.039 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:56.040 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/661744716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:56.040 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/4033941409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:56.040 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:55 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3393063262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-12T04:12:56.041 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-12T04:12:56.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-12T04:12:56.125 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2226594480' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: osdmap e86: 12 total, 12 up, 12 in 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: pgmap v291: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] up:active 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/661744716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/4033941409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:55 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3393063262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-12T04:12:56.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 -- 172.21.15.42:0/2635420367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ff230 msgr2=0x7fe7e00ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:56.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/2635420367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ff230 0x7fe7e00ff6b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009900 tx=0x7fe7d402bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:56.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 -- 172.21.15.42:0/2635420367 shutdown_connections 2024-03-12T04:12:56.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/2635420367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e00ffbf0 0x7fe7e01097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/2635420367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ff230 0x7fe7e00ff6b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/2635420367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e01027c0 0x7fe7e0102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 -- 172.21.15.42:0/2635420367 >> 172.21.15.42:0/2635420367 conn(0x7fe7e00fad70 msgr2=0x7fe7e00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 -- 172.21.15.42:0/2635420367 shutdown_connections 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 -- 172.21.15.42:0/2635420367 wait complete. 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 Processor -- start 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.147+0000 7fe7e5a88640 1 -- start start 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e00ff230 0x7fe7e01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ffbf0 0x7fe7e01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01027c0 0x7fe7e019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e019ec90 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e019ee00 con 0x7fe7e00ff230 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e019efc0 con 0x7fe7e01027c0 2024-03-12T04:12:56.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ffbf0 0x7fe7e01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ffbf0 0x7fe7e01976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54498/0 (socket says 172.21.15.42:54498) 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 -- 172.21.15.42:0/461127381 learned_addr learned my addr 172.21.15.42:0/461127381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7dffff640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01027c0 0x7fe7e019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7df7fe640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e00ff230 0x7fe7e01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 -- 172.21.15.42:0/461127381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01027c0 msgr2=0x7fe7e019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01027c0 0x7fe7e019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 -- 172.21.15.42:0/461127381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e00ff230 msgr2=0x7fe7e01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e00ff230 0x7fe7e01971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 -- 172.21.15.42:0/461127381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4009560 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7deffd640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ffbf0 0x7fe7e01976e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4036040 tx=0x7fe7d4004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7dcff9640 1 -- 172.21.15.42:0/461127381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4003800 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7dcff9640 1 -- 172.21.15.42:0/461127381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe7d402de90 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e019f240 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7dcff9640 1 -- 172.21.15.42:0/461127381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4005600 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.148+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e019f7d0 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.149+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e01086d0 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.150+0000 7fe7dcff9640 1 -- 172.21.15.42:0/461127381 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe7d4004dc0 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.151+0000 7fe7dcff9640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7c80a1a30 0x7fe7c80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:56.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.151+0000 7fe7df7fe640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7c80a1a30 0x7fe7c80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:56.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.151+0000 7fe7dcff9640 1 -- 172.21.15.42:0/461127381 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7d40ef680 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.151+0000 7fe7df7fe640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7c80a1a30 0x7fe7c80a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0069430 tx=0x7fe7d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:56.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.155+0000 7fe7dcff9640 1 -- 172.21.15.42:0/461127381 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe7d40b89e0 con 0x7fe7e00ffbf0 2024-03-12T04:12:56.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2226594480' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: osdmap e86: 12 total, 12 up, 12 in 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: pgmap v291: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] up:active 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/661744716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/4033941409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:55 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3393063262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-12T04:12:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:56.377+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fe7e010cd10 con 0x7fe7e00ffbf0 2024-03-12T04:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.025+0000 7fe7dcff9640 1 -- 172.21.15.42:0/461127381 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe7d40bc0a0 con 0x7fe7e00ffbf0 2024-03-12T04:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7c80a1a30 msgr2=0x7fe7c80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7c80a1a30 0x7fe7c80a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0069430 tx=0x7fe7d0008040 comp rx=0 tx=0).stop 2024-03-12T04:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ffbf0 msgr2=0x7fe7e01976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ffbf0 0x7fe7e01976e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4036040 tx=0x7fe7d4004370 comp rx=0 tx=0).stop 2024-03-12T04:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 shutdown_connections 2024-03-12T04:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01027c0 0x7fe7e019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7e00ffbf0 0x7fe7e01976e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7c80a1a30 0x7fe7c80a3ef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 --2- 172.21.15.42:0/461127381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e00ff230 0x7fe7e01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.027+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 >> 172.21.15.42:0/461127381 conn(0x7fe7e00fad70 msgr2=0x7fe7e01058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.028+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 shutdown_connections 2024-03-12T04:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.028+0000 7fe7e5a88640 1 -- 172.21.15.42:0/461127381 wait complete. 2024-03-12T04:12:57.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:56 smithi042 ceph-mon[32260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-12T04:12:57.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:56 smithi042 ceph-mon[32260]: Cluster is now healthy 2024-03-12T04:12:57.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:56 smithi042 ceph-mon[32260]: osdmap e87: 12 total, 12 up, 12 in 2024-03-12T04:12:57.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:56 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3393063262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-12T04:12:57.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:56 smithi042 ceph-mon[32260]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:57.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:56 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/461127381' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-12T04:12:57.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-12T04:12:57.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:56 smithi119 ceph-mon[36119]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-12T04:12:57.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:56 smithi119 ceph-mon[36119]: Cluster is now healthy 2024-03-12T04:12:57.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:56 smithi119 ceph-mon[36119]: osdmap e87: 12 total, 12 up, 12 in 2024-03-12T04:12:57.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:56 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3393063262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-12T04:12:57.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:56 smithi119 ceph-mon[36119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:57.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:56 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/461127381' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 -- 172.21.15.42:0/2557023878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 msgr2=0x7fead0101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 --2- 172.21.15.42:0/2557023878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead0101a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feac0009930 tx=0x7feac002bc30 comp rx=0 tx=0).stop 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 -- 172.21.15.42:0/2557023878 shutdown_connections 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 --2- 172.21.15.42:0/2557023878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fead0100d20 0x7fead01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 --2- 172.21.15.42:0/2557023878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fead0100360 0x7fead01007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 --2- 172.21.15.42:0/2557023878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead0101a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 -- 172.21.15.42:0/2557023878 >> 172.21.15.42:0/2557023878 conn(0x7fead00fad70 msgr2=0x7fead00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 -- 172.21.15.42:0/2557023878 shutdown_connections 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.156+0000 7fead4b7a640 1 -- 172.21.15.42:0/2557023878 wait complete. 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 Processor -- start 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 -- start start 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fead0100360 0x7fead0197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fead0100d20 0x7fead0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead019ed00 con 0x7fead0101630 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead019ee70 con 0x7fead0100360 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7fead4b7a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead019f030 con 0x7fead0100d20 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7feaced9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7feaced9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54516/0 (socket says 172.21.15.42:54516) 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7feaced9d640 1 -- 172.21.15.42:0/1900361528 learned_addr learned my addr 172.21.15.42:0/1900361528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7feac7fff640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fead0100d20 0x7fead0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.157+0000 7feace59c640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fead0100360 0x7fead0197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feaced9d640 1 -- 172.21.15.42:0/1900361528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fead0100d20 msgr2=0x7fead0197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feaced9d640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fead0100d20 0x7fead0197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feaced9d640 1 -- 172.21.15.42:0/1900361528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fead0100360 msgr2=0x7fead0197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feaced9d640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fead0100360 0x7fead0197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feaced9d640 1 -- 172.21.15.42:0/1900361528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0009560 con 0x7fead0101630 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feac7fff640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fead0100d20 0x7fead0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feaced9d640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead019e7c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feac800eaa0 tx=0x7feac800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feac77fe640 1 -- 172.21.15.42:0/1900361528 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac800cdb0 con 0x7fead0101630 2024-03-12T04:12:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feac77fe640 1 -- 172.21.15.42:0/1900361528 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feac8005020 con 0x7fead0101630 2024-03-12T04:12:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead019f310 con 0x7fead0101630 2024-03-12T04:12:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7feac77fe640 1 -- 172.21.15.42:0/1900361528 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac80116b0 con 0x7fead0101630 2024-03-12T04:12:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.158+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead019f7d0 con 0x7fead0101630 2024-03-12T04:12:57.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.159+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead010b980 con 0x7fead0101630 2024-03-12T04:12:57.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.160+0000 7feac77fe640 1 -- 172.21.15.42:0/1900361528 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7feac8005190 con 0x7fead0101630 2024-03-12T04:12:57.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.161+0000 7feac77fe640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feabc0a1a30 0x7feabc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:57.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.161+0000 7feace59c640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feabc0a1a30 0x7feabc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:57.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.161+0000 7feac77fe640 1 -- 172.21.15.42:0/1900361528 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feac80cd4d0 con 0x7fead0101630 2024-03-12T04:12:57.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.161+0000 7feace59c640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feabc0a1a30 0x7feabc0a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feac0000c00 tx=0x7feac0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.165+0000 7feac77fe640 1 -- 172.21.15.42:0/1900361528 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feac8096840 con 0x7fead0101630 2024-03-12T04:12:57.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:56 smithi136 ceph-mon[35571]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-12T04:12:57.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:56 smithi136 ceph-mon[35571]: Cluster is now healthy 2024-03-12T04:12:57.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:56 smithi136 ceph-mon[35571]: osdmap e87: 12 total, 12 up, 12 in 2024-03-12T04:12:57.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:56 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3393063262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-12T04:12:57.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:56 smithi136 ceph-mon[35571]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:57.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:56 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/461127381' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-12T04:12:57.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:57.386+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fead019fcd0 con 0x7fead0101630 2024-03-12T04:12:58.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.031+0000 7feac77fe640 1 -- 172.21.15.42:0/1900361528 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7feac8099f00 con 0x7fead0101630 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feabc0a1a30 msgr2=0x7feabc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feabc0a1a30 0x7feabc0a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feac0000c00 tx=0x7feac0005fd0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 msgr2=0x7fead019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead019e7c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feac800eaa0 tx=0x7feac800ef60 comp rx=0 tx=0).stop 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 shutdown_connections 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead0101630 0x7fead019e7c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fead0100d20 0x7fead0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feabc0a1a30 0x7feabc0a3ef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 --2- 172.21.15.42:0/1900361528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fead0100360 0x7fead0197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 >> 172.21.15.42:0/1900361528 conn(0x7fead00fad70 msgr2=0x7fead00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:58.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 shutdown_connections 2024-03-12T04:12:58.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.035+0000 7fead4b7a640 1 -- 172.21.15.42:0/1900361528 wait complete. 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: pgmap v293: 97 pgs: 11 creating+peering, 82 active+clean, 4 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 741 B/s wr, 6 op/s 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/461127381' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-12T04:12:58.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:57 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1900361528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-12T04:12:58.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 -- 172.21.15.42:0/111668143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8106e80 msgr2=0x7f0be81092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 --2- 172.21.15.42:0/111668143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8106e80 0x7f0be81092d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009ca0 tx=0x7f0be002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 -- 172.21.15.42:0/111668143 shutdown_connections 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 --2- 172.21.15.42:0/111668143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8106e80 0x7f0be81092d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 --2- 172.21.15.42:0/111668143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0be8104550 0x7f0be8106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 --2- 172.21.15.42:0/111668143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be8101c20 0x7f0be8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 -- 172.21.15.42:0/111668143 >> 172.21.15.42:0/111668143 conn(0x7f0be80fb630 msgr2=0x7f0be80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 -- 172.21.15.42:0/111668143 shutdown_connections 2024-03-12T04:12:58.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.164+0000 7f0bee301640 1 -- 172.21.15.42:0/111668143 wait complete. 2024-03-12T04:12:58.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 Processor -- start 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 -- start start 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8101c20 0x7f0be8101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0be8104550 0x7f0be80ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be8106e80 0x7f0be8100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8100870 con 0x7f0be8101c20 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be81009e0 con 0x7f0be8106e80 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bee301640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be81a11d0 con 0x7f0be8104550 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8101c20 0x7f0be8101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8101c20 0x7f0be8101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54536/0 (socket says 172.21.15.42:54536) 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 -- 172.21.15.42:0/3563159429 learned_addr learned my addr 172.21.15.42:0/3563159429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be77fe640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0be8104550 0x7f0be80ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0bec89e640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be8106e80 0x7f0be8100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 -- 172.21.15.42:0/3563159429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0be8104550 msgr2=0x7f0be80ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0be8104550 0x7f0be80ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 -- 172.21.15.42:0/3563159429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be8106e80 msgr2=0x7f0be8100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be8106e80 0x7f0be8100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.165+0000 7f0be7fff640 1 -- 172.21.15.42:0/3563159429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0009900 con 0x7f0be8101c20 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.166+0000 7f0be7fff640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8101c20 0x7f0be8101590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00ea20 tx=0x7f0bdc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.166+0000 7f0be57fa640 1 -- 172.21.15.42:0/3563159429 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc009700 con 0x7f0be8101c20 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.166+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be81a1480 con 0x7f0be8101c20 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.166+0000 7f0be57fa640 1 -- 172.21.15.42:0/3563159429 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0bdc00cd80 con 0x7f0be8101c20 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.166+0000 7f0be57fa640 1 -- 172.21.15.42:0/3563159429 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc018750 con 0x7f0be8101c20 2024-03-12T04:12:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.166+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be81a1990 con 0x7f0be8101c20 2024-03-12T04:12:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.167+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be80fd120 con 0x7f0be8101c20 2024-03-12T04:12:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.168+0000 7f0be57fa640 1 -- 172.21.15.42:0/3563159429 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0bdc010430 con 0x7f0be8101c20 2024-03-12T04:12:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.168+0000 7f0be57fa640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bd00a1a30 0x7f0bd00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.168+0000 7f0be77fe640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bd00a1a30 0x7f0bd00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.169+0000 7f0be57fa640 1 -- 172.21.15.42:0/3563159429 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0bdc014070 con 0x7f0be8101c20 2024-03-12T04:12:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.169+0000 7f0be77fe640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bd00a1a30 0x7f0bd00a3ef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0be8101300 tx=0x7f0bd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:58.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.173+0000 7f0be57fa640 1 -- 172.21.15.42:0/3563159429 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0bdc052d20 con 0x7f0be8101c20 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: pgmap v293: 97 pgs: 11 creating+peering, 82 active+clean, 4 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 741 B/s wr, 6 op/s 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/461127381' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-12T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:57 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1900361528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-12T04:12:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: pgmap v293: 97 pgs: 11 creating+peering, 82 active+clean, 4 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 741 B/s wr, 6 op/s 2024-03-12T04:12:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-12T04:12:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/461127381' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-12T04:12:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-12T04:12:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T04:12:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-12T04:12:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-12T04:12:58.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:57 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1900361528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-12T04:12:58.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:58.394+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f0be810ca60 con 0x7f0be8101c20 2024-03-12T04:12:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.040+0000 7f0be57fa640 1 -- 172.21.15.42:0/3563159429 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0be810ca60 con 0x7f0be8101c20 2024-03-12T04:12:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bd00a1a30 msgr2=0x7f0bd00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bd00a1a30 0x7f0bd00a3ef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0be8101300 tx=0x7f0bd8005f90 comp rx=0 tx=0).stop 2024-03-12T04:12:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8101c20 msgr2=0x7f0be8101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8101c20 0x7f0be8101590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00ea20 tx=0x7f0bdc00eee0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 shutdown_connections 2024-03-12T04:12:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bd00a1a30 0x7f0bd00a3ef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be8106e80 0x7f0be8100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0be8104550 0x7f0be80ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 --2- 172.21.15.42:0/3563159429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8101c20 0x7f0be8101590 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 >> 172.21.15.42:0/3563159429 conn(0x7f0be80fb630 msgr2=0x7f0be8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 shutdown_connections 2024-03-12T04:12:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.043+0000 7f0bee301640 1 -- 172.21.15.42:0/3563159429 wait complete. 2024-03-12T04:12:59.062 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-03-12T04:12:59.063 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-12T04:12:59.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.171+0000 7f91ae38c640 1 -- 172.21.15.42:0/800003741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8106e60 msgr2=0x7f91a81092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.171+0000 7f91ae38c640 1 --2- 172.21.15.42:0/800003741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8106e60 0x7f91a81092b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009ca0 tx=0x7f91a002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.171+0000 7f91ae38c640 1 -- 172.21.15.42:0/800003741 shutdown_connections 2024-03-12T04:12:59.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 --2- 172.21.15.42:0/800003741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8106e60 0x7f91a81092b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 --2- 172.21.15.42:0/800003741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a8104530 0x7f91a8106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 --2- 172.21.15.42:0/800003741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a8101c00 0x7f91a8103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 -- 172.21.15.42:0/800003741 >> 172.21.15.42:0/800003741 conn(0x7f91a80fb610 msgr2=0x7f91a80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 -- 172.21.15.42:0/800003741 shutdown_connections 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 -- 172.21.15.42:0/800003741 wait complete. 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 Processor -- start 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 -- start start 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8101c00 0x7f91a81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a8104530 0x7f91a8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a8106e60 0x7f91a819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819ecb0 con 0x7f91a8101c00 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819ee20 con 0x7f91a8104530 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.172+0000 7f91ae38c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819efe0 con 0x7f91a8106e60 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8101c00 0x7f91a81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8101c00 0x7f91a81971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54542/0 (socket says 172.21.15.42:54542) 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 -- 172.21.15.42:0/448224829 learned_addr learned my addr 172.21.15.42:0/448224829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a77fe640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a8104530 0x7f91a8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91ac929640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a8106e60 0x7f91a819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 -- 172.21.15.42:0/448224829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a8106e60 msgr2=0x7f91a819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a8106e60 0x7f91a819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 -- 172.21.15.42:0/448224829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a8104530 msgr2=0x7f91a8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a8104530 0x7f91a8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 -- 172.21.15.42:0/448224829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a0009900 con 0x7f91a8101c00 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a77fe640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a8104530 0x7f91a8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a7fff640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8101c00 0x7f91a81971c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f919800e9f0 tx=0x7f919800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a57fa640 1 -- 172.21.15.42:0/448224829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198009700 con 0x7f91a8101c00 2024-03-12T04:12:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a819f2c0 con 0x7f91a8101c00 2024-03-12T04:12:59.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a57fa640 1 -- 172.21.15.42:0/448224829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f919800cd80 con 0x7f91a8101c00 2024-03-12T04:12:59.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91a57fa640 1 -- 172.21.15.42:0/448224829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198018840 con 0x7f91a8101c00 2024-03-12T04:12:59.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.173+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a819f850 con 0x7f91a8101c00 2024-03-12T04:12:59.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.174+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a80fd100 con 0x7f91a8101c00 2024-03-12T04:12:59.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.176+0000 7f91a57fa640 1 -- 172.21.15.42:0/448224829 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f91980189a0 con 0x7f91a8101c00 2024-03-12T04:12:59.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.176+0000 7f91a57fa640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f91900a1a30 0x7f91900a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.176+0000 7f91a77fe640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f91900a1a30 0x7f91900a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.176+0000 7f91a57fa640 1 -- 172.21.15.42:0/448224829 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9198014070 con 0x7f91a8101c00 2024-03-12T04:12:59.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.177+0000 7f91a77fe640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f91900a1a30 0x7f91900a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91a81986d0 tx=0x7f919c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:59.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.180+0000 7f91a57fa640 1 -- 172.21.15.42:0/448224829 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9198096f00 con 0x7f91a8101c00 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1900361528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] up:active 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3563159429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-12T04:12:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-mon[32260]: pgmap v294: 97 pgs: 8 creating+peering, 89 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 7 op/s 2024-03-12T04:12:59.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.386+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f91a802ceb0 con 0x7f91900a1a30 2024-03-12T04:12:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.409+0000 7f91a57fa640 1 -- 172.21.15.42:0/448224829 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f91a802ceb0 con 0x7f91900a1a30 2024-03-12T04:12:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.411+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f91900a1a30 msgr2=0x7f91900a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.411+0000 7f91ae38c640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f91900a1a30 0x7f91900a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91a81986d0 tx=0x7f919c008040 comp rx=0 tx=0).stop 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.411+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8101c00 msgr2=0x7f91a81971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.411+0000 7f91ae38c640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8101c00 0x7f91a81971c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f919800e9f0 tx=0x7f919800eeb0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 shutdown_connections 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a8106e60 0x7f91a819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f91900a1a30 0x7f91900a3ef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a8104530 0x7f91a8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 --2- 172.21.15.42:0/448224829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8101c00 0x7f91a81971c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 >> 172.21.15.42:0/448224829 conn(0x7f91a80fb610 msgr2=0x7f91a8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 shutdown_connections 2024-03-12T04:12:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.412+0000 7f91ae38c640 1 -- 172.21.15.42:0/448224829 wait complete. 2024-03-12T04:12:59.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-12T04:12:59.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1900361528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-12T04:12:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] up:active 2024-03-12T04:12:59.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-12T04:12:59.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-12T04:12:59.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-12T04:12:59.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-12T04:12:59.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3563159429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-12T04:12:59.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-mon[36119]: pgmap v294: 97 pgs: 8 creating+peering, 89 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 7 op/s 2024-03-12T04:12:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1900361528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-12T04:12:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] up:active 2024-03-12T04:12:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-12T04:12:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-12T04:12:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-12T04:12:59.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-12T04:12:59.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3563159429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-12T04:12:59.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:12:59 smithi136 ceph-mon[35571]: pgmap v294: 97 pgs: 8 creating+peering, 89 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 7 op/s 2024-03-12T04:12:59.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 -- 172.21.15.42:0/4065579158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101c50 msgr2=0x7f5af41084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 --2- 172.21.15.42:0/4065579158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101c50 0x7f5af41084e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009ca0 tx=0x7f5aec02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 -- 172.21.15.42:0/4065579158 shutdown_connections 2024-03-12T04:12:59.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 --2- 172.21.15.42:0/4065579158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101c50 0x7f5af41084e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 --2- 172.21.15.42:0/4065579158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101290 0x7f5af4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 --2- 172.21.15.42:0/4065579158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af41000a0 0x7f5af41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 -- 172.21.15.42:0/4065579158 >> 172.21.15.42:0/4065579158 conn(0x7f5af40fb630 msgr2=0x7f5af40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 -- 172.21.15.42:0/4065579158 shutdown_connections 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 -- 172.21.15.42:0/4065579158 wait complete. 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 Processor -- start 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.530+0000 7f5af9adb640 1 -- start start 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af41000a0 0x7f5af410a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101290 0x7f5af4108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101c50 0x7f5af41094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4109c40 con 0x7f5af4101290 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4109db0 con 0x7f5af4101c50 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41a1140 con 0x7f5af41000a0 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101290 0x7f5af4108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101290 0x7f5af4108fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54580/0 (socket says 172.21.15.42:54580) 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 -- 172.21.15.42:0/2697072114 learned_addr learned my addr 172.21.15.42:0/2697072114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:59.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af3fff640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101c50 0x7f5af41094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 -- 172.21.15.42:0/2697072114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af41000a0 msgr2=0x7f5af410a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af41000a0 0x7f5af410a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 -- 172.21.15.42:0/2697072114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101c50 msgr2=0x7f5af41094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101c50 0x7f5af41094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 -- 172.21.15.42:0/2697072114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009900 con 0x7f5af4101290 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af2ffd640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101290 0x7f5af4108fa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800ea20 tx=0x7f5ae800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af0ff9640 1 -- 172.21.15.42:0/2697072114 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8009700 con 0x7f5af4101290 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af0ff9640 1 -- 172.21.15.42:0/2697072114 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ae800cd80 con 0x7f5af4101290 2024-03-12T04:12:59.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af41a13f0 con 0x7f5af4101290 2024-03-12T04:12:59.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af0ff9640 1 -- 172.21.15.42:0/2697072114 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8018710 con 0x7f5af4101290 2024-03-12T04:12:59.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.531+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af41a1900 con 0x7f5af4101290 2024-03-12T04:12:59.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.532+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af41073c0 con 0x7f5af4101290 2024-03-12T04:12:59.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.533+0000 7f5af0ff9640 1 -- 172.21.15.42:0/2697072114 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5ae8010430 con 0x7f5af4101290 2024-03-12T04:12:59.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.534+0000 7f5af0ff9640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5adc0a1a30 0x7f5adc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.534+0000 7f5af37fe640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5adc0a1a30 0x7f5adc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.534+0000 7f5af0ff9640 1 -- 172.21.15.42:0/2697072114 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ae8014070 con 0x7f5af4101290 2024-03-12T04:12:59.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.534+0000 7f5af37fe640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5adc0a1a30 0x7f5adc0a3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5af41010f0 tx=0x7f5ae4005c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:59.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.539+0000 7f5af0ff9640 1 -- 172.21.15.42:0/2697072114 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ae8052d20 con 0x7f5af4101290 2024-03-12T04:12:59.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.748+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f5af402ceb0 con 0x7f5adc0a1a30 2024-03-12T04:12:59.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.770+0000 7f5af0ff9640 1 -- 172.21.15.42:0/2697072114 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5af402ceb0 con 0x7f5adc0a1a30 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5adc0a1a30 msgr2=0x7f5adc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5adc0a1a30 0x7f5adc0a3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5af41010f0 tx=0x7f5ae4005c10 comp rx=0 tx=0).stop 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101290 msgr2=0x7f5af4108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101290 0x7f5af4108fa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800ea20 tx=0x7f5ae800eee0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 shutdown_connections 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5adc0a1a30 0x7f5adc0a3ef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101c50 0x7f5af41094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5af4101290 0x7f5af4108fa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 --2- 172.21.15.42:0/2697072114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af41000a0 0x7f5af410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 >> 172.21.15.42:0/2697072114 conn(0x7f5af40fb630 msgr2=0x7f5af4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 shutdown_connections 2024-03-12T04:12:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.772+0000 7f5af9adb640 1 -- 172.21.15.42:0/2697072114 wait complete. 2024-03-12T04:12:59.784 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:12:59.391+0000 7efcafe81700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:12:59.785 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:12:59.391+0000 7efcafe81700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:12:59.785 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:12:59.391+0000 7efcafe81700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:12:59.785 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:12:59.391+0000 7efcafe81700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:12:59.785 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:12:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:12:59.391+0000 7efcafe81700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:12:59.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.890+0000 7f52d1c88640 1 -- 172.21.15.42:0/1461412500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 msgr2=0x7f52cc1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.890+0000 7f52d1c88640 1 --2- 172.21.15.42:0/1461412500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc1092f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009ca0 tx=0x7f52c402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.890+0000 7f52d1c88640 1 -- 172.21.15.42:0/1461412500 shutdown_connections 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.890+0000 7f52d1c88640 1 --2- 172.21.15.42:0/1461412500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc1092f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 --2- 172.21.15.42:0/1461412500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52cc1045d0 0x7f52cc1069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 --2- 172.21.15.42:0/1461412500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52cc101ca0 0x7f52cc104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 -- 172.21.15.42:0/1461412500 >> 172.21.15.42:0/1461412500 conn(0x7f52cc0fb610 msgr2=0x7f52cc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 -- 172.21.15.42:0/1461412500 shutdown_connections 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 -- 172.21.15.42:0/1461412500 wait complete. 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 Processor -- start 2024-03-12T04:12:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 -- start start 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52cc101ca0 0x7f52cc101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52cc1045d0 0x7f52cc0ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1008f0 con 0x7f52cc106f00 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc100a60 con 0x7f52cc1045d0 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.891+0000 7f52d1c88640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1a11d0 con 0x7f52cc101ca0 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc100190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54584/0 (socket says 172.21.15.42:54584) 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 -- 172.21.15.42:0/786614187 learned_addr learned my addr 172.21.15.42:0/786614187 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52caffd640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52cc1045d0 0x7f52cc0ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cb7fe640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52cc101ca0 0x7f52cc101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 -- 172.21.15.42:0/786614187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52cc101ca0 msgr2=0x7f52cc101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52cc101ca0 0x7f52cc101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 -- 172.21.15.42:0/786614187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52cc1045d0 msgr2=0x7f52cc0ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52cc1045d0 0x7f52cc0ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 -- 172.21.15.42:0/786614187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009900 con 0x7f52cc106f00 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52caffd640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52cc1045d0 0x7f52cc0ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52cbfff640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc100190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f52c4033000 tx=0x7f52c4032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4034e70 con 0x7f52cc106f00 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f52c4003e20 con 0x7f52cc106f00 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc1a1420 con 0x7f52cc106f00 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4004880 con 0x7f52cc106f00 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.892+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc1a1920 con 0x7f52cc106f00 2024-03-12T04:12:59.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.893+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc0fd100 con 0x7f52cc106f00 2024-03-12T04:12:59.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.894+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f52c403b040 con 0x7f52cc106f00 2024-03-12T04:12:59.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.895+0000 7f52c8ff9640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52b40a1ab0 0x7f52b40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:12:59.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.895+0000 7f52cb7fe640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52b40a1ab0 0x7f52b40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:12:59.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.895+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52c40ef9c0 con 0x7f52cc106f00 2024-03-12T04:12:59.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.895+0000 7f52cb7fe640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52b40a1ab0 0x7f52b40a3f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009aa0 tx=0x7f52bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:12:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:12:59.899+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52c40b8d20 con 0x7f52cc106f00 2024-03-12T04:13:00.044 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: 2024-03-12T04:12:59.751+0000 7f6f07212700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-12T04:13:00.044 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: 2024-03-12T04:12:59.751+0000 7f6f07212700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-12T04:13:00.044 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:12:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: 2024-03-12T04:12:59.751+0000 7f6f07212700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-12T04:13:00.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.058+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f52c400a040 con 0x7f52cc106f00 2024-03-12T04:13:00.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.119+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f52cc02ceb0 con 0x7f52b40a1ab0 2024-03-12T04:13:00.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.124+0000 7f52c8ff9640 1 -- 172.21.15.42:0/786614187 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f52cc02ceb0 con 0x7f52b40a1ab0 2024-03-12T04:13:00.126 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_0/42c61c3a-49f9-4165-a046-fc4ded319250 2024-03-12T04:13:00.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.125+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52b40a1ab0 msgr2=0x7f52b40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.125+0000 7f52d1c88640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52b40a1ab0 0x7f52b40a3f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009aa0 tx=0x7f52bc005f90 comp rx=0 tx=0).stop 2024-03-12T04:13:00.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.125+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 msgr2=0x7f52cc100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.125+0000 7f52d1c88640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc100190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f52c4033000 tx=0x7f52c4032980 comp rx=0 tx=0).stop 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 shutdown_connections 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52cc106f00 0x7f52cc100190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52b40a1ab0 0x7f52b40a3f70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52cc1045d0 0x7f52cc0ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 --2- 172.21.15.42:0/786614187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52cc101ca0 0x7f52cc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 >> 172.21.15.42:0/786614187 conn(0x7f52cc0fb610 msgr2=0x7f52cc102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 shutdown_connections 2024-03-12T04:13:00.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.126+0000 7f52d1c88640 1 -- 172.21.15.42:0/786614187 wait complete. 2024-03-12T04:13:00.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-12T04:13:00.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- 172.21.15.42:0/3129180028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101c30 msgr2=0x7f2cec1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 --2- 172.21.15.42:0/3129180028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101c30 0x7f2cec1084c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009ca0 tx=0x7f2ce402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- 172.21.15.42:0/3129180028 shutdown_connections 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 --2- 172.21.15.42:0/3129180028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101c30 0x7f2cec1084c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 --2- 172.21.15.42:0/3129180028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec101270 0x7f2cec1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 --2- 172.21.15.42:0/3129180028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cec100080 0x7f2cec1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- 172.21.15.42:0/3129180028 >> 172.21.15.42:0/3129180028 conn(0x7f2cec0fb610 msgr2=0x7f2cec0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- 172.21.15.42:0/3129180028 shutdown_connections 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- 172.21.15.42:0/3129180028 wait complete. 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 Processor -- start 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- start start 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cec100080 0x7f2cec196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101270 0x7f2cec196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec101c30 0x7f2cec19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec19db10 con 0x7f2cec101270 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec19dc80 con 0x7f2cec101c30 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cf3264640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec19de40 con 0x7f2cec100080 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.247+0000 7f2cebfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101270 0x7f2cec196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101270 0x7f2cec196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54612/0 (socket says 172.21.15.42:54612) 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 -- 172.21.15.42:0/1566742151 learned_addr learned my addr 172.21.15.42:0/1566742151 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 -- 172.21.15.42:0/1566742151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cec100080 msgr2=0x7f2cec196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cf1000640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cec100080 0x7f2cec196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cec100080 0x7f2cec196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 -- 172.21.15.42:0/1566742151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec101c30 msgr2=0x7f2cec19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec101c30 0x7f2cec19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 -- 172.21.15.42:0/1566742151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009900 con 0x7f2cec101270 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cebfff640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101270 0x7f2cec196560 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009970 tx=0x7f2cd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2ce9ffb640 1 -- 172.21.15.42:0/1566742151 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8012070 con 0x7f2cec101270 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2ce9ffb640 1 -- 172.21.15.42:0/1566742151 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cd8007d10 con 0x7f2cec101270 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec068340 con 0x7f2cec101270 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2ce9ffb640 1 -- 172.21.15.42:0/1566742151 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800ca20 con 0x7f2cec101270 2024-03-12T04:13:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.248+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec068850 con 0x7f2cec101270 2024-03-12T04:13:00.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.249+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec067f70 con 0x7f2cec101270 2024-03-12T04:13:00.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.250+0000 7f2ce9ffb640 1 -- 172.21.15.42:0/1566742151 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f2cd8016070 con 0x7f2cec101270 2024-03-12T04:13:00.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.251+0000 7f2ce9ffb640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cdc0a1ac0 0x7f2cdc0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.251+0000 7f2cf1000640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cdc0a1ac0 0x7f2cdc0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.251+0000 7f2ce9ffb640 1 -- 172.21.15.42:0/1566742151 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2cd80ce4b0 con 0x7f2cec101270 2024-03-12T04:13:00.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.252+0000 7f2cf1000640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cdc0a1ac0 0x7f2cdc0a3f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00058c0 tx=0x7f2ce0005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:00.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.256+0000 7f2ce9ffb640 1 -- 172.21.15.42:0/1566742151 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cd8097710 con 0x7f2cec101270 2024-03-12T04:13:00.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3563159429' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-12T04:13:00.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] up:active 2024-03-12T04:13:00.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-12T04:13:00.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-12T04:13:00.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-12T04:13:00.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] up:active 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: Cluster is now healthy 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-12T04:13:00.317 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:00 smithi042 ceph-mon[32260]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3563159429' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] up:active 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] up:active 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-12T04:13:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-12T04:13:00.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: Cluster is now healthy 2024-03-12T04:13:00.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-03-12T04:13:00.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-12T04:13:00.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:00 smithi119 ceph-mon[36119]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-12T04:13:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3563159429' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-12T04:13:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] up:active 2024-03-12T04:13:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-12T04:13:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] up:active 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: Cluster is now healthy 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-12T04:13:00.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:00 smithi136 ceph-mon[35571]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-12T04:13:00.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.466+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cec107070 con 0x7f2cdc0a1ac0 2024-03-12T04:13:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.491+0000 7f2ce9ffb640 1 -- 172.21.15.42:0/1566742151 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2cec107070 con 0x7f2cdc0a1ac0 2024-03-12T04:13:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.493+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cdc0a1ac0 msgr2=0x7f2cdc0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.493+0000 7f2cf3264640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cdc0a1ac0 0x7f2cdc0a3f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00058c0 tx=0x7f2ce0005c80 comp rx=0 tx=0).stop 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.493+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101270 msgr2=0x7f2cec196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.493+0000 7f2cf3264640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101270 0x7f2cec196560 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009970 tx=0x7f2cd8009e30 comp rx=0 tx=0).stop 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.493+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 shutdown_connections 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.493+0000 7f2cf3264640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2cdc0a1ac0 0x7f2cdc0a3f80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.494+0000 7f2cf3264640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec101c30 0x7f2cec19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.494+0000 7f2cf3264640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cec101270 0x7f2cec196560 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.494+0000 7f2cf3264640 1 --2- 172.21.15.42:0/1566742151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cec100080 0x7f2cec196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.494+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 >> 172.21.15.42:0/1566742151 conn(0x7f2cec0fb610 msgr2=0x7f2cec1045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.494+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 shutdown_connections 2024-03-12T04:13:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.494+0000 7f2cf3264640 1 -- 172.21.15.42:0/1566742151 wait complete. 2024-03-12T04:13:00.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-12T04:13:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 -- 172.21.15.42:0/1365672969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102380 msgr2=0x7f09c0102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 --2- 172.21.15.42:0/1365672969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102380 0x7f09c0102800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402bc30 comp rx=0 tx=0).stop 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 -- 172.21.15.42:0/1365672969 shutdown_connections 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 --2- 172.21.15.42:0/1365672969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c0102d40 0x7f09c01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 --2- 172.21.15.42:0/1365672969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102380 0x7f09c0102800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 --2- 172.21.15.42:0/1365672969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c0101190 0x7f09c01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 -- 172.21.15.42:0/1365672969 >> 172.21.15.42:0/1365672969 conn(0x7f09c00fc720 msgr2=0x7f09c00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 -- 172.21.15.42:0/1365672969 shutdown_connections 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 -- 172.21.15.42:0/1365672969 wait complete. 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 Processor -- start 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.612+0000 7f09c680e640 1 -- start start 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c0101190 0x7f09c0197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c0102380 0x7f09c01976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102d40 0x7f09c019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019ec80 con 0x7f09c0102d40 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019edf0 con 0x7f09c0101190 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019efb0 con 0x7f09c0102380 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102d40 0x7f09c019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102d40 0x7f09c019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54618/0 (socket says 172.21.15.42:54618) 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 -- 172.21.15.42:0/2519501598 learned_addr learned my addr 172.21.15.42:0/2519501598 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09bffff640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c0101190 0x7f09c0197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09bf7fe640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c0102380 0x7f09c01976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 -- 172.21.15.42:0/2519501598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c0102380 msgr2=0x7f09c01976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c0102380 0x7f09c01976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 -- 172.21.15.42:0/2519501598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c0101190 msgr2=0x7f09c0197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c0101190 0x7f09c0197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 -- 172.21.15.42:0/2519501598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4009560 con 0x7f09c0102d40 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09bf7fe640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c0102380 0x7f09c01976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c4dab640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102d40 0x7f09c019e740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09b800cce0 tx=0x7f09b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09bd7fa640 1 -- 172.21.15.42:0/2519501598 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b800f450 con 0x7f09c0102d40 2024-03-12T04:13:00.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c019f290 con 0x7f09c0102d40 2024-03-12T04:13:00.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09bd7fa640 1 -- 172.21.15.42:0/2519501598 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09b800fa60 con 0x7f09c0102d40 2024-03-12T04:13:00.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09bd7fa640 1 -- 172.21.15.42:0/2519501598 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8018b10 con 0x7f09c0102d40 2024-03-12T04:13:00.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.613+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c019f7d0 con 0x7f09c0102d40 2024-03-12T04:13:00.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.615+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c0108550 con 0x7f09c0102d40 2024-03-12T04:13:00.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.616+0000 7f09bd7fa640 1 -- 172.21.15.42:0/2519501598 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f09b800fbd0 con 0x7f09c0102d40 2024-03-12T04:13:00.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.616+0000 7f09bd7fa640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f09a80a1ac0 0x7f09a80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.616+0000 7f09bffff640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f09a80a1ac0 0x7f09a80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.616+0000 7f09bd7fa640 1 -- 172.21.15.42:0/2519501598 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09b80cd8e0 con 0x7f09c0102d40 2024-03-12T04:13:00.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.617+0000 7f09bffff640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f09a80a1ac0 0x7f09a80a3f80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09c01021e0 tx=0x7f09b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:00.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.621+0000 7f09bd7fa640 1 -- 172.21.15.42:0/2519501598 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09b8096b40 con 0x7f09c0102d40 2024-03-12T04:13:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.830+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f09c002ceb0 con 0x7f09a80a1ac0 2024-03-12T04:13:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.834+0000 7f09bd7fa640 1 -- 172.21.15.42:0/2519501598 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f09c002ceb0 con 0x7f09a80a1ac0 2024-03-12T04:13:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_1/9a2f96af-7fa6-40af-8433-1699a7f358aa 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f09a80a1ac0 msgr2=0x7f09a80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f09a80a1ac0 0x7f09a80a3f80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09c01021e0 tx=0x7f09b0008040 comp rx=0 tx=0).stop 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102d40 msgr2=0x7f09c019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102d40 0x7f09c019e740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09b800cce0 tx=0x7f09b800a620 comp rx=0 tx=0).stop 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 shutdown_connections 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c0102d40 0x7f09c019e740 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c0102380 0x7f09c01976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f09a80a1ac0 0x7f09a80a3f80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 --2- 172.21.15.42:0/2519501598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c0101190 0x7f09c0197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 >> 172.21.15.42:0/2519501598 conn(0x7f09c00fc720 msgr2=0x7f09c0105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 shutdown_connections 2024-03-12T04:13:00.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.836+0000 7f09c680e640 1 -- 172.21.15.42:0/2519501598 wait complete. 2024-03-12T04:13:00.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:13:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 -- 172.21.15.42:0/1547165943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 msgr2=0x7fcc881092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/1547165943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc881092f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009ca0 tx=0x7fcc8002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 -- 172.21.15.42:0/1547165943 shutdown_connections 2024-03-12T04:13:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/1547165943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc881092f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/1547165943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc88104570 0x7fcc88106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/1547165943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc88101c40 0x7fcc88104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 -- 172.21.15.42:0/1547165943 >> 172.21.15.42:0/1547165943 conn(0x7fcc880fb630 msgr2=0x7fcc880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 -- 172.21.15.42:0/1547165943 shutdown_connections 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.950+0000 7fcc8d36b640 1 -- 172.21.15.42:0/1547165943 wait complete. 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 Processor -- start 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 -- start start 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc88101c40 0x7fcc881015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc88104570 0x7fcc880ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc88100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc88100890 con 0x7fcc88106ea0 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc88100a00 con 0x7fcc88104570 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc8d36b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc881a1220 con 0x7fcc88101c40 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc88100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc88100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54634/0 (socket says 172.21.15.42:54634) 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 -- 172.21.15.42:0/630596108 learned_addr learned my addr 172.21.15.42:0/630596108 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc7ffff640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc88104570 0x7fcc880ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc86ffd640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc88101c40 0x7fcc881015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 -- 172.21.15.42:0/630596108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc88101c40 msgr2=0x7fcc881015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc88101c40 0x7fcc881015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 -- 172.21.15.42:0/630596108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc88104570 msgr2=0x7fcc880ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc88104570 0x7fcc880ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc877fe640 1 -- 172.21.15.42:0/630596108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80009900 con 0x7fcc88106ea0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.951+0000 7fcc7ffff640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc88104570 0x7fcc880ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.952+0000 7fcc877fe640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc88100130 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcc80033000 tx=0x7fcc80032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.952+0000 7fcc84ff9640 1 -- 172.21.15.42:0/630596108 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80034e70 con 0x7fcc88106ea0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.952+0000 7fcc84ff9640 1 -- 172.21.15.42:0/630596108 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc80003e20 con 0x7fcc88106ea0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.952+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc881a1470 con 0x7fcc88106ea0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.952+0000 7fcc84ff9640 1 -- 172.21.15.42:0/630596108 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8003d3f0 con 0x7fcc88106ea0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.952+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc881a19a0 con 0x7fcc88106ea0 2024-03-12T04:13:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.953+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc880fd120 con 0x7fcc88106ea0 2024-03-12T04:13:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.954+0000 7fcc84ff9640 1 -- 172.21.15.42:0/630596108 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fcc8003b040 con 0x7fcc88106ea0 2024-03-12T04:13:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.954+0000 7fcc84ff9640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcc740a1ac0 0x7fcc740a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:00.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.955+0000 7fcc86ffd640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcc740a1ac0 0x7fcc740a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:00.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.955+0000 7fcc84ff9640 1 -- 172.21.15.42:0/630596108 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc800efe50 con 0x7fcc88106ea0 2024-03-12T04:13:00.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.955+0000 7fcc86ffd640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcc740a1ac0 0x7fcc740a3f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800a810 tx=0x7fcc78008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:00.959+0000 7fcc84ff9640 1 -- 172.21.15.42:0/630596108 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc800b91c0 con 0x7fcc88106ea0 2024-03-12T04:13:01.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.187+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc8802ce40 con 0x7fcc88106ea0 2024-03-12T04:13:01.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.187+0000 7fcc84ff9640 1 -- 172.21.15.42:0/630596108 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fcc800bc880 con 0x7fcc88106ea0 2024-03-12T04:13:01.190 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:13:01.190 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:13:01.029979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:13:01.190 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcc740a1ac0 msgr2=0x7fcc740a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcc740a1ac0 0x7fcc740a3f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800a810 tx=0x7fcc78008040 comp rx=0 tx=0).stop 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 msgr2=0x7fcc88100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc88100130 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcc80033000 tx=0x7fcc80032980 comp rx=0 tx=0).stop 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 shutdown_connections 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc88106ea0 0x7fcc88100130 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcc740a1ac0 0x7fcc740a3f80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc88104570 0x7fcc880ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 --2- 172.21.15.42:0/630596108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc88101c40 0x7fcc881015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 >> 172.21.15.42:0/630596108 conn(0x7fcc880fb630 msgr2=0x7fcc88102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:01.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 shutdown_connections 2024-03-12T04:13:01.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.189+0000 7fcc8d36b640 1 -- 172.21.15.42:0/630596108 wait complete. 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: pgmap v295: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s wr, 17 op/s 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] up:active 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-12T04:13:01.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:01 smithi042 ceph-mon[32260]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-12T04:13:01.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:13:01.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.303+0000 7fd3955c9640 1 -- 172.21.15.42:0/2576952609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390106f20 msgr2=0x7fd390109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.303+0000 7fd3955c9640 1 --2- 172.21.15.42:0/2576952609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390106f20 0x7fd390109310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd380009ca0 tx=0x7fd38002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.303+0000 7fd3955c9640 1 -- 172.21.15.42:0/2576952609 shutdown_connections 2024-03-12T04:13:01.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.303+0000 7fd3955c9640 1 --2- 172.21.15.42:0/2576952609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390106f20 0x7fd390109310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.303+0000 7fd3955c9640 1 --2- 172.21.15.42:0/2576952609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3901045f0 0x7fd3901069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.303+0000 7fd3955c9640 1 --2- 172.21.15.42:0/2576952609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd390101cc0 0x7fd3901040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.303+0000 7fd3955c9640 1 -- 172.21.15.42:0/2576952609 >> 172.21.15.42:0/2576952609 conn(0x7fd3900fb610 msgr2=0x7fd3900fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 -- 172.21.15.42:0/2576952609 shutdown_connections 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 -- 172.21.15.42:0/2576952609 wait complete. 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 Processor -- start 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 -- start start 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390101cc0 0x7fd390197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3901045f0 0x7fd390197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd390106f20 0x7fd39019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019eb00 con 0x7fd390101cc0 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019ec70 con 0x7fd390106f20 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.304+0000 7fd3955c9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019ee30 con 0x7fd3901045f0 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390101cc0 0x7fd390197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390101cc0 0x7fd390197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54650/0 (socket says 172.21.15.42:54650) 2024-03-12T04:13:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 -- 172.21.15.42:0/151819899 learned_addr learned my addr 172.21.15.42:0/151819899 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 -- 172.21.15.42:0/151819899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3901045f0 msgr2=0x7fd390197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3901045f0 0x7fd390197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38f7fe640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd390106f20 0x7fd39019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 -- 172.21.15.42:0/151819899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd390106f20 msgr2=0x7fd39019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd390106f20 0x7fd39019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 -- 172.21.15.42:0/151819899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd380009900 con 0x7fd390101cc0 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd38effd640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390101cc0 0x7fd390197010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c920 tx=0x7fd38800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd387fff640 1 -- 172.21.15.42:0/151819899 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388007c70 con 0x7fd390101cc0 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39019f110 con 0x7fd390101cc0 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd387fff640 1 -- 172.21.15.42:0/151819899 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd388007dd0 con 0x7fd390101cc0 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd387fff640 1 -- 172.21.15.42:0/151819899 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3880187c0 con 0x7fd390101cc0 2024-03-12T04:13:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.305+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39019f620 con 0x7fd390101cc0 2024-03-12T04:13:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.306+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3900fd100 con 0x7fd390101cc0 2024-03-12T04:13:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.307+0000 7fd387fff640 1 -- 172.21.15.42:0/151819899 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fd38800f450 con 0x7fd390101cc0 2024-03-12T04:13:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.308+0000 7fd387fff640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd37c0a1ac0 0x7fd37c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.308+0000 7fd38e7fc640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd37c0a1ac0 0x7fd37c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.308+0000 7fd387fff640 1 -- 172.21.15.42:0/151819899 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3880cd990 con 0x7fd390101cc0 2024-03-12T04:13:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.308+0000 7fd38e7fc640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd37c0a1ac0 0x7fd37c0a3f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd390198520 tx=0x7fd378005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.311+0000 7fd387fff640 1 -- 172.21.15.42:0/151819899 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd388096b80 con 0x7fd390101cc0 2024-03-12T04:13:01.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-12T04:13:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: pgmap v295: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s wr, 17 op/s 2024-03-12T04:13:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] up:active 2024-03-12T04:13:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-12T04:13:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:01 smithi119 ceph-mon[36119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-12T04:13:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-12T04:13:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:13:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: pgmap v295: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s wr, 17 op/s 2024-03-12T04:13:01.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] up:active 2024-03-12T04:13:01.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-12T04:13:01.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:01 smithi136 ceph-mon[35571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-12T04:13:01.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.494+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd39010cd30 con 0x7fd390101cc0 2024-03-12T04:13:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.495+0000 7fd387fff640 1 -- 172.21.15.42:0/151819899 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22219 (secure 0 0 0) 0x7fd39010cd30 con 0x7fd390101cc0 2024-03-12T04:13:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:13:01.498 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:55.769086+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:13:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.497+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd37c0a1ac0 msgr2=0x7fd37c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.497+0000 7fd3955c9640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd37c0a1ac0 0x7fd37c0a3f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd390198520 tx=0x7fd378005c70 comp rx=0 tx=0).stop 2024-03-12T04:13:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.497+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390101cc0 msgr2=0x7fd390197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390101cc0 0x7fd390197010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c920 tx=0x7fd38800cde0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 shutdown_connections 2024-03-12T04:13:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd37c0a1ac0 0x7fd37c0a3f80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd390106f20 0x7fd39019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3901045f0 0x7fd390197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 --2- 172.21.15.42:0/151819899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd390101cc0 0x7fd390197010 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 >> 172.21.15.42:0/151819899 conn(0x7fd3900fb610 msgr2=0x7fd390102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 shutdown_connections 2024-03-12T04:13:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.498+0000 7fd3955c9640 1 -- 172.21.15.42:0/151819899 wait complete. 2024-03-12T04:13:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-12T04:13:01.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T04:13:01.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 -- 172.21.15.42:0/256892190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac1000a0 msgr2=0x7f19ac1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 --2- 172.21.15.42:0/256892190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac1000a0 0x7f19ac1004c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009900 tx=0x7f19a802bc30 comp rx=0 tx=0).stop 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 -- 172.21.15.42:0/256892190 shutdown_connections 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 --2- 172.21.15.42:0/256892190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ac101c50 0x7f19ac1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 --2- 172.21.15.42:0/256892190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac101290 0x7f19ac101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 --2- 172.21.15.42:0/256892190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac1000a0 0x7f19ac1004c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 -- 172.21.15.42:0/256892190 >> 172.21.15.42:0/256892190 conn(0x7f19ac0fb610 msgr2=0x7f19ac0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 -- 172.21.15.42:0/256892190 shutdown_connections 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 -- 172.21.15.42:0/256892190 wait complete. 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 Processor -- start 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.612+0000 7f19b46fb640 1 -- start start 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ac1000a0 0x7f19ac1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac101290 0x7f19ac197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac101c50 0x7f19ac19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac19ecd0 con 0x7f19ac101c50 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac19ee40 con 0x7f19ac1000a0 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac19f000 con 0x7f19ac101290 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac101c50 0x7f19ac19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac101c50 0x7f19ac19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35816/0 (socket says 172.21.15.42:35816) 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 -- 172.21.15.42:0/2539310649 learned_addr learned my addr 172.21.15.42:0/2539310649 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2497640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ac1000a0 0x7f19ac1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 -- 172.21.15.42:0/2539310649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac101290 msgr2=0x7f19ac197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac101290 0x7f19ac197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 -- 172.21.15.42:0/2539310649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ac1000a0 msgr2=0x7f19ac1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ac1000a0 0x7f19ac1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 -- 172.21.15.42:0/2539310649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8009560 con 0x7f19ac101c50 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2497640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ac1000a0 0x7f19ac1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b2c98640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac101c50 0x7f19ac19e790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b970 tx=0x7f19a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19a37fe640 1 -- 172.21.15.42:0/2539310649 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a400cb40 con 0x7f19ac101c50 2024-03-12T04:13:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac19f2e0 con 0x7f19ac101c50 2024-03-12T04:13:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19a37fe640 1 -- 172.21.15.42:0/2539310649 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19a400cca0 con 0x7f19ac101c50 2024-03-12T04:13:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19a37fe640 1 -- 172.21.15.42:0/2539310649 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40076e0 con 0x7f19ac101c50 2024-03-12T04:13:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.613+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac19f870 con 0x7f19ac101c50 2024-03-12T04:13:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.614+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac1073f0 con 0x7f19ac101c50 2024-03-12T04:13:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.615+0000 7f19a37fe640 1 -- 172.21.15.42:0/2539310649 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f19a4014440 con 0x7f19ac101c50 2024-03-12T04:13:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.616+0000 7f19a37fe640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f19980a1ac0 0x7f19980a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.616+0000 7f19b2497640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f19980a1ac0 0x7f19980a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.616+0000 7f19a37fe640 1 -- 172.21.15.42:0/2539310649 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19a40cd1b0 con 0x7f19ac101c50 2024-03-12T04:13:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.616+0000 7f19b2497640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f19980a1ac0 0x7f19980a3f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f19a8036040 tx=0x7f19a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.619+0000 7f19a37fe640 1 -- 172.21.15.42:0/2539310649 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19a4096420 con 0x7f19ac101c50 2024-03-12T04:13:01.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.800+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19ac02ceb0 con 0x7f19980a1ac0 2024-03-12T04:13:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.808+0000 7f19a37fe640 1 -- 172.21.15.42:0/2539310649 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258905 (secure 0 0 0) 0x7f19ac02ceb0 con 0x7f19980a1ac0 2024-03-12T04:13:01.813 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:13:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f19980a1ac0 msgr2=0x7f19980a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f19980a1ac0 0x7f19980a3f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f19a8036040 tx=0x7f19a8000bc0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac101c50 msgr2=0x7f19ac19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac101c50 0x7f19ac19e790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b970 tx=0x7f19a400be30 comp rx=0 tx=0).stop 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 shutdown_connections 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19ac101c50 0x7f19ac19e790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac101290 0x7f19ac197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f19980a1ac0 0x7f19980a3f80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 --2- 172.21.15.42:0/2539310649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ac1000a0 0x7f19ac1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.811+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 >> 172.21.15.42:0/2539310649 conn(0x7f19ac0fb610 msgr2=0x7f19ac1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.812+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 shutdown_connections 2024-03-12T04:13:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.812+0000 7f19b46fb640 1 -- 172.21.15.42:0/2539310649 wait complete. 2024-03-12T04:13:01.825 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-03-12T04:13:00.899408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597464,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":308,"num_write_kb":2979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":247,"ondisk_log_size":247,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":261,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373004,"kb_used_data":11328,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568820,"statfs":{"total":1151940427776,"available":1151558471680,"internally_reserved":0,"allocated":11599872,"data_stored":7174848,"data_compressed":108960,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7096,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.140985"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.121053+0000","last_change":"2024-03-12T04:12:54.788665+0000","last_active":"2024-03-12T04:12:56.121053+0000","last_peered":"2024-03-12T04:12:56.121053+0000","last_clean":"2024-03-12T04:12:56.121053+0000","last_became_active":"2024-03-12T04:12:54.788334+0000","last_became_peered":"2024-03-12T04:12:54.788334+0000","last_unstale":"2024-03-12T04:12:56.121053+0000","last_undegraded":"2024-03-12T04:12:56.121053+0000","last_fullsized":"2024-03-12T04:12:56.121053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:02.442543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775682+0000","last_change":"2024-03-12T04:12:54.791942+0000","last_active":"2024-03-12T04:12:55.775682+0000","last_peered":"2024-03-12T04:12:55.775682+0000","last_clean":"2024-03-12T04:12:55.775682+0000","last_became_active":"2024-03-12T04:12:54.791579+0000","last_became_peered":"2024-03-12T04:12:54.791579+0000","last_unstale":"2024-03-12T04:12:55.775682+0000","last_undegraded":"2024-03-12T04:12:55.775682+0000","last_fullsized":"2024-03-12T04:12:55.775682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:51:08.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775581+0000","last_change":"2024-03-12T04:12:54.793238+0000","last_active":"2024-03-12T04:12:55.775581+0000","last_peered":"2024-03-12T04:12:55.775581+0000","last_clean":"2024-03-12T04:12:55.775581+0000","last_became_active":"2024-03-12T04:12:54.792877+0000","last_became_peered":"2024-03-12T04:12:54.792877+0000","last_unstale":"2024-03-12T04:12:55.775581+0000","last_undegraded":"2024-03-12T04:12:55.775581+0000","last_fullsized":"2024-03-12T04:12:55.775581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:36:46.914204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120602+0000","last_change":"2024-03-12T04:12:54.797573+0000","last_active":"2024-03-12T04:12:56.120602+0000","last_peered":"2024-03-12T04:12:56.120602+0000","last_clean":"2024-03-12T04:12:56.120602+0000","last_became_active":"2024-03-12T04:12:54.796939+0000","last_became_peered":"2024-03-12T04:12:54.796939+0000","last_unstale":"2024-03-12T04:12:56.120602+0000","last_undegraded":"2024-03-12T04:12:56.120602+0000","last_fullsized":"2024-03-12T04:12:56.120602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:34:58.047492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775990+0000","last_change":"2024-03-12T04:12:54.780595+0000","last_active":"2024-03-12T04:12:55.775990+0000","last_peered":"2024-03-12T04:12:55.775990+0000","last_clean":"2024-03-12T04:12:55.775990+0000","last_became_active":"2024-03-12T04:12:54.780256+0000","last_became_peered":"2024-03-12T04:12:54.780256+0000","last_unstale":"2024-03-12T04:12:55.775990+0000","last_undegraded":"2024-03-12T04:12:55.775990+0000","last_fullsized":"2024-03-12T04:12:55.775990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:42.492006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776238+0000","last_change":"2024-03-12T04:12:54.781573+0000","last_active":"2024-03-12T04:12:55.776238+0000","last_peered":"2024-03-12T04:12:55.776238+0000","last_clean":"2024-03-12T04:12:55.776238+0000","last_became_active":"2024-03-12T04:12:54.781230+0000","last_became_peered":"2024-03-12T04:12:54.781230+0000","last_unstale":"2024-03-12T04:12:55.776238+0000","last_undegraded":"2024-03-12T04:12:55.776238+0000","last_fullsized":"2024-03-12T04:12:55.776238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:16.869952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393320+0000","last_change":"2024-03-12T04:12:54.781510+0000","last_active":"2024-03-12T04:12:56.393320+0000","last_peered":"2024-03-12T04:12:56.393320+0000","last_clean":"2024-03-12T04:12:56.393320+0000","last_became_active":"2024-03-12T04:12:54.781141+0000","last_became_peered":"2024-03-12T04:12:54.781141+0000","last_unstale":"2024-03-12T04:12:56.393320+0000","last_undegraded":"2024-03-12T04:12:56.393320+0000","last_fullsized":"2024-03-12T04:12:56.393320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:42:05.055308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120330+0000","last_change":"2024-03-12T04:12:54.782541+0000","last_active":"2024-03-12T04:12:56.120330+0000","last_peered":"2024-03-12T04:12:56.120330+0000","last_clean":"2024-03-12T04:12:56.120330+0000","last_became_active":"2024-03-12T04:12:54.781582+0000","last_became_peered":"2024-03-12T04:12:54.781582+0000","last_unstale":"2024-03-12T04:12:56.120330+0000","last_undegraded":"2024-03-12T04:12:56.120330+0000","last_fullsized":"2024-03-12T04:12:56.120330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:15.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120792+0000","last_change":"2024-03-12T04:12:54.788201+0000","last_active":"2024-03-12T04:12:56.120792+0000","last_peered":"2024-03-12T04:12:56.120792+0000","last_clean":"2024-03-12T04:12:56.120792+0000","last_became_active":"2024-03-12T04:12:54.787875+0000","last_became_peered":"2024-03-12T04:12:54.787875+0000","last_unstale":"2024-03-12T04:12:56.120792+0000","last_undegraded":"2024-03-12T04:12:56.120792+0000","last_fullsized":"2024-03-12T04:12:56.120792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:40:39.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776604+0000","last_change":"2024-03-12T04:12:54.789748+0000","last_active":"2024-03-12T04:12:55.776604+0000","last_peered":"2024-03-12T04:12:55.776604+0000","last_clean":"2024-03-12T04:12:55.776604+0000","last_became_active":"2024-03-12T04:12:54.788838+0000","last_became_peered":"2024-03-12T04:12:54.788838+0000","last_unstale":"2024-03-12T04:12:55.776604+0000","last_undegraded":"2024-03-12T04:12:55.776604+0000","last_fullsized":"2024-03-12T04:12:55.776604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:39:51.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393582+0000","last_change":"2024-03-12T04:12:54.793101+0000","last_active":"2024-03-12T04:12:56.393582+0000","last_peered":"2024-03-12T04:12:56.393582+0000","last_clean":"2024-03-12T04:12:56.393582+0000","last_became_active":"2024-03-12T04:12:54.792488+0000","last_became_peered":"2024-03-12T04:12:54.792488+0000","last_unstale":"2024-03-12T04:12:56.393582+0000","last_undegraded":"2024-03-12T04:12:56.393582+0000","last_fullsized":"2024-03-12T04:12:56.393582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:33.862011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120600+0000","last_change":"2024-03-12T04:12:54.797953+0000","last_active":"2024-03-12T04:12:56.120600+0000","last_peered":"2024-03-12T04:12:56.120600+0000","last_clean":"2024-03-12T04:12:56.120600+0000","last_became_active":"2024-03-12T04:12:54.797413+0000","last_became_peered":"2024-03-12T04:12:54.797413+0000","last_unstale":"2024-03-12T04:12:56.120600+0000","last_undegraded":"2024-03-12T04:12:56.120600+0000","last_fullsized":"2024-03-12T04:12:56.120600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:18:01.140872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120135+0000","last_change":"2024-03-12T04:12:54.775570+0000","last_active":"2024-03-12T04:12:56.120135+0000","last_peered":"2024-03-12T04:12:56.120135+0000","last_clean":"2024-03-12T04:12:56.120135+0000","last_became_active":"2024-03-12T04:12:54.775186+0000","last_became_peered":"2024-03-12T04:12:54.775186+0000","last_unstale":"2024-03-12T04:12:56.120135+0000","last_undegraded":"2024-03-12T04:12:56.120135+0000","last_fullsized":"2024-03-12T04:12:56.120135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:26.004458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775390+0000","last_change":"2024-03-12T04:12:54.780859+0000","last_active":"2024-03-12T04:12:55.775390+0000","last_peered":"2024-03-12T04:12:55.775390+0000","last_clean":"2024-03-12T04:12:55.775390+0000","last_became_active":"2024-03-12T04:12:54.780484+0000","last_became_peered":"2024-03-12T04:12:54.780484+0000","last_unstale":"2024-03-12T04:12:55.775390+0000","last_undegraded":"2024-03-12T04:12:55.775390+0000","last_fullsized":"2024-03-12T04:12:55.775390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:24:00.575324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775375+0000","last_change":"2024-03-12T04:12:54.796087+0000","last_active":"2024-03-12T04:12:55.775375+0000","last_peered":"2024-03-12T04:12:55.775375+0000","last_clean":"2024-03-12T04:12:55.775375+0000","last_became_active":"2024-03-12T04:12:54.793702+0000","last_became_peered":"2024-03-12T04:12:54.793702+0000","last_unstale":"2024-03-12T04:12:55.775375+0000","last_undegraded":"2024-03-12T04:12:55.775375+0000","last_fullsized":"2024-03-12T04:12:55.775375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:09:27.334791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120170+0000","last_change":"2024-03-12T04:12:54.783146+0000","last_active":"2024-03-12T04:12:56.120170+0000","last_peered":"2024-03-12T04:12:56.120170+0000","last_clean":"2024-03-12T04:12:56.120170+0000","last_became_active":"2024-03-12T04:12:54.782834+0000","last_became_peered":"2024-03-12T04:12:54.782834+0000","last_unstale":"2024-03-12T04:12:56.120170+0000","last_undegraded":"2024-03-12T04:12:56.120170+0000","last_fullsized":"2024-03-12T04:12:56.120170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:35:11.126629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.121227+0000","last_change":"2024-03-12T04:12:54.794879+0000","last_active":"2024-03-12T04:12:56.121227+0000","last_peered":"2024-03-12T04:12:56.121227+0000","last_clean":"2024-03-12T04:12:56.121227+0000","last_became_active":"2024-03-12T04:12:54.794251+0000","last_became_peered":"2024-03-12T04:12:54.794251+0000","last_unstale":"2024-03-12T04:12:56.121227+0000","last_undegraded":"2024-03-12T04:12:56.121227+0000","last_fullsized":"2024-03-12T04:12:56.121227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:33.754426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119871+0000","last_change":"2024-03-12T04:12:54.791301+0000","last_active":"2024-03-12T04:12:56.119871+0000","last_peered":"2024-03-12T04:12:56.119871+0000","last_clean":"2024-03-12T04:12:56.119871+0000","last_became_active":"2024-03-12T04:12:54.790850+0000","last_became_peered":"2024-03-12T04:12:54.790850+0000","last_unstale":"2024-03-12T04:12:56.119871+0000","last_undegraded":"2024-03-12T04:12:56.119871+0000","last_fullsized":"2024-03-12T04:12:56.119871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:26:04.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120305+0000","last_change":"2024-03-12T04:12:54.793419+0000","last_active":"2024-03-12T04:12:56.120305+0000","last_peered":"2024-03-12T04:12:56.120305+0000","last_clean":"2024-03-12T04:12:56.120305+0000","last_became_active":"2024-03-12T04:12:54.793080+0000","last_became_peered":"2024-03-12T04:12:54.793080+0000","last_unstale":"2024-03-12T04:12:56.120305+0000","last_undegraded":"2024-03-12T04:12:56.120305+0000","last_fullsized":"2024-03-12T04:12:56.120305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:46:26.506899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'11","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.058934+0000","last_change":"2024-03-12T04:12:53.774893+0000","last_active":"2024-03-12T04:12:59.058934+0000","last_peered":"2024-03-12T04:12:59.058934+0000","last_clean":"2024-03-12T04:12:59.058934+0000","last_became_active":"2024-03-12T04:12:53.773801+0000","last_became_peered":"2024-03-12T04:12:53.773801+0000","last_unstale":"2024-03-12T04:12:59.058934+0000","last_undegraded":"2024-03-12T04:12:59.058934+0000","last_fullsized":"2024-03-12T04:12:59.058934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:30:19.995765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120353+0000","last_change":"2024-03-12T04:12:54.786795+0000","last_active":"2024-03-12T04:12:56.120353+0000","last_peered":"2024-03-12T04:12:56.120353+0000","last_clean":"2024-03-12T04:12:56.120353+0000","last_became_active":"2024-03-12T04:12:54.786481+0000","last_became_peered":"2024-03-12T04:12:54.786481+0000","last_unstale":"2024-03-12T04:12:56.120353+0000","last_undegraded":"2024-03-12T04:12:56.120353+0000","last_fullsized":"2024-03-12T04:12:56.120353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:06.849644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120687+0000","last_change":"2024-03-12T04:12:53.779582+0000","last_active":"2024-03-12T04:12:56.120687+0000","last_peered":"2024-03-12T04:12:56.120687+0000","last_clean":"2024-03-12T04:12:56.120687+0000","last_became_active":"2024-03-12T04:12:53.779062+0000","last_became_peered":"2024-03-12T04:12:53.779062+0000","last_unstale":"2024-03-12T04:12:56.120687+0000","last_undegraded":"2024-03-12T04:12:56.120687+0000","last_fullsized":"2024-03-12T04:12:56.120687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:44:01.065183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775737+0000","last_change":"2024-03-12T04:12:54.789198+0000","last_active":"2024-03-12T04:12:55.775737+0000","last_peered":"2024-03-12T04:12:55.775737+0000","last_clean":"2024-03-12T04:12:55.775737+0000","last_became_active":"2024-03-12T04:12:54.788213+0000","last_became_peered":"2024-03-12T04:12:54.788213+0000","last_unstale":"2024-03-12T04:12:55.775737+0000","last_undegraded":"2024-03-12T04:12:55.775737+0000","last_fullsized":"2024-03-12T04:12:55.775737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:24:36.561553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776364+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:12:55.776364+0000","last_peered":"2024-03-12T04:12:55.776364+0000","last_clean":"2024-03-12T04:12:55.776364+0000","last_became_active":"2024-03-12T04:12:53.771178+0000","last_became_peered":"2024-03-12T04:12:53.771178+0000","last_unstale":"2024-03-12T04:12:55.776364+0000","last_undegraded":"2024-03-12T04:12:55.776364+0000","last_fullsized":"2024-03-12T04:12:55.776364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:12.882128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119957+0000","last_change":"2024-03-12T04:12:54.799758+0000","last_active":"2024-03-12T04:12:56.119957+0000","last_peered":"2024-03-12T04:12:56.119957+0000","last_clean":"2024-03-12T04:12:56.119957+0000","last_became_active":"2024-03-12T04:12:54.799338+0000","last_became_peered":"2024-03-12T04:12:54.799338+0000","last_unstale":"2024-03-12T04:12:56.119957+0000","last_undegraded":"2024-03-12T04:12:56.119957+0000","last_fullsized":"2024-03-12T04:12:56.119957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:32:54.087890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776196+0000","last_change":"2024-03-12T04:12:53.774390+0000","last_active":"2024-03-12T04:12:55.776196+0000","last_peered":"2024-03-12T04:12:55.776196+0000","last_clean":"2024-03-12T04:12:55.776196+0000","last_became_active":"2024-03-12T04:12:53.774042+0000","last_became_peered":"2024-03-12T04:12:53.774042+0000","last_unstale":"2024-03-12T04:12:55.776196+0000","last_undegraded":"2024-03-12T04:12:55.776196+0000","last_fullsized":"2024-03-12T04:12:55.776196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:37.361505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.121102+0000","last_change":"2024-03-12T04:12:54.783478+0000","last_active":"2024-03-12T04:12:56.121102+0000","last_peered":"2024-03-12T04:12:56.121102+0000","last_clean":"2024-03-12T04:12:56.121102+0000","last_became_active":"2024-03-12T04:12:54.783080+0000","last_became_peered":"2024-03-12T04:12:54.783080+0000","last_unstale":"2024-03-12T04:12:56.121102+0000","last_undegraded":"2024-03-12T04:12:56.121102+0000","last_fullsized":"2024-03-12T04:12:56.121102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:57:48.639090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.042994+0000","last_change":"2024-03-12T04:12:53.772182+0000","last_active":"2024-03-12T04:12:57.042994+0000","last_peered":"2024-03-12T04:12:57.042994+0000","last_clean":"2024-03-12T04:12:57.042994+0000","last_became_active":"2024-03-12T04:12:53.771858+0000","last_became_peered":"2024-03-12T04:12:53.771858+0000","last_unstale":"2024-03-12T04:12:57.042994+0000","last_undegraded":"2024-03-12T04:12:57.042994+0000","last_fullsized":"2024-03-12T04:12:57.042994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:42:41.466311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776120+0000","last_change":"2024-03-12T04:12:54.779869+0000","last_active":"2024-03-12T04:12:55.776120+0000","last_peered":"2024-03-12T04:12:55.776120+0000","last_clean":"2024-03-12T04:12:55.776120+0000","last_became_active":"2024-03-12T04:12:54.779500+0000","last_became_peered":"2024-03-12T04:12:54.779500+0000","last_unstale":"2024-03-12T04:12:55.776120+0000","last_undegraded":"2024-03-12T04:12:55.776120+0000","last_fullsized":"2024-03-12T04:12:55.776120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:29.538570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:58.047421+0000","last_change":"2024-03-12T04:12:53.774056+0000","last_active":"2024-03-12T04:12:58.047421+0000","last_peered":"2024-03-12T04:12:58.047421+0000","last_clean":"2024-03-12T04:12:58.047421+0000","last_became_active":"2024-03-12T04:12:53.773675+0000","last_became_peered":"2024-03-12T04:12:53.773675+0000","last_unstale":"2024-03-12T04:12:58.047421+0000","last_undegraded":"2024-03-12T04:12:58.047421+0000","last_fullsized":"2024-03-12T04:12:58.047421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:33.576947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120839+0000","last_change":"2024-03-12T04:12:54.790583+0000","last_active":"2024-03-12T04:12:56.120839+0000","last_peered":"2024-03-12T04:12:56.120839+0000","last_clean":"2024-03-12T04:12:56.120839+0000","last_became_active":"2024-03-12T04:12:54.788364+0000","last_became_peered":"2024-03-12T04:12:54.788364+0000","last_unstale":"2024-03-12T04:12:56.120839+0000","last_undegraded":"2024-03-12T04:12:56.120839+0000","last_fullsized":"2024-03-12T04:12:56.120839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:05:13.542279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119451+0000","last_change":"2024-03-12T04:12:53.769619+0000","last_active":"2024-03-12T04:12:56.119451+0000","last_peered":"2024-03-12T04:12:56.119451+0000","last_clean":"2024-03-12T04:12:56.119451+0000","last_became_active":"2024-03-12T04:12:53.769293+0000","last_became_peered":"2024-03-12T04:12:53.769293+0000","last_unstale":"2024-03-12T04:12:56.119451+0000","last_undegraded":"2024-03-12T04:12:56.119451+0000","last_fullsized":"2024-03-12T04:12:56.119451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:44.618499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120232+0000","last_change":"2024-03-12T04:12:54.776493+0000","last_active":"2024-03-12T04:12:56.120232+0000","last_peered":"2024-03-12T04:12:56.120232+0000","last_clean":"2024-03-12T04:12:56.120232+0000","last_became_active":"2024-03-12T04:12:54.775967+0000","last_became_peered":"2024-03-12T04:12:54.775967+0000","last_unstale":"2024-03-12T04:12:56.120232+0000","last_undegraded":"2024-03-12T04:12:56.120232+0000","last_fullsized":"2024-03-12T04:12:56.120232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:19:46.824145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.041412+0000","last_change":"2024-03-12T04:12:53.773523+0000","last_active":"2024-03-12T04:12:57.041412+0000","last_peered":"2024-03-12T04:12:57.041412+0000","last_clean":"2024-03-12T04:12:57.041412+0000","last_became_active":"2024-03-12T04:12:53.773161+0000","last_became_peered":"2024-03-12T04:12:53.773161+0000","last_unstale":"2024-03-12T04:12:57.041412+0000","last_undegraded":"2024-03-12T04:12:57.041412+0000","last_fullsized":"2024-03-12T04:12:57.041412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:41:40.810994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120661+0000","last_change":"2024-03-12T04:12:54.791688+0000","last_active":"2024-03-12T04:12:56.120661+0000","last_peered":"2024-03-12T04:12:56.120661+0000","last_clean":"2024-03-12T04:12:56.120661+0000","last_became_active":"2024-03-12T04:12:54.790854+0000","last_became_peered":"2024-03-12T04:12:54.790854+0000","last_unstale":"2024-03-12T04:12:56.120661+0000","last_undegraded":"2024-03-12T04:12:56.120661+0000","last_fullsized":"2024-03-12T04:12:56.120661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:42:50.343524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.039970+0000","last_change":"2024-03-12T04:12:53.777534+0000","last_active":"2024-03-12T04:12:57.039970+0000","last_peered":"2024-03-12T04:12:57.039970+0000","last_clean":"2024-03-12T04:12:57.039970+0000","last_became_active":"2024-03-12T04:12:53.777201+0000","last_became_peered":"2024-03-12T04:12:53.777201+0000","last_unstale":"2024-03-12T04:12:57.039970+0000","last_undegraded":"2024-03-12T04:12:57.039970+0000","last_fullsized":"2024-03-12T04:12:57.039970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:54.708892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120679+0000","last_change":"2024-03-12T04:12:54.798962+0000","last_active":"2024-03-12T04:12:56.120679+0000","last_peered":"2024-03-12T04:12:56.120679+0000","last_clean":"2024-03-12T04:12:56.120679+0000","last_became_active":"2024-03-12T04:12:54.798650+0000","last_became_peered":"2024-03-12T04:12:54.798650+0000","last_unstale":"2024-03-12T04:12:56.120679+0000","last_undegraded":"2024-03-12T04:12:56.120679+0000","last_fullsized":"2024-03-12T04:12:56.120679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:05:29.270242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393602+0000","last_change":"2024-03-12T04:12:53.777791+0000","last_active":"2024-03-12T04:12:56.393602+0000","last_peered":"2024-03-12T04:12:56.393602+0000","last_clean":"2024-03-12T04:12:56.393602+0000","last_became_active":"2024-03-12T04:12:53.777430+0000","last_became_peered":"2024-03-12T04:12:53.777430+0000","last_unstale":"2024-03-12T04:12:56.393602+0000","last_undegraded":"2024-03-12T04:12:56.393602+0000","last_fullsized":"2024-03-12T04:12:56.393602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:38:21.858857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775707+0000","last_change":"2024-03-12T04:12:54.784710+0000","last_active":"2024-03-12T04:12:55.775707+0000","last_peered":"2024-03-12T04:12:55.775707+0000","last_clean":"2024-03-12T04:12:55.775707+0000","last_became_active":"2024-03-12T04:12:54.784342+0000","last_became_peered":"2024-03-12T04:12:54.784342+0000","last_unstale":"2024-03-12T04:12:55.775707+0000","last_undegraded":"2024-03-12T04:12:55.775707+0000","last_fullsized":"2024-03-12T04:12:55.775707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:21.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.040427+0000","last_change":"2024-03-12T04:12:53.779289+0000","last_active":"2024-03-12T04:12:57.040427+0000","last_peered":"2024-03-12T04:12:57.040427+0000","last_clean":"2024-03-12T04:12:57.040427+0000","last_became_active":"2024-03-12T04:12:53.778894+0000","last_became_peered":"2024-03-12T04:12:53.778894+0000","last_unstale":"2024-03-12T04:12:57.040427+0000","last_undegraded":"2024-03-12T04:12:57.040427+0000","last_fullsized":"2024-03-12T04:12:57.040427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:13:33.328977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120160+0000","last_change":"2024-03-12T04:12:54.794102+0000","last_active":"2024-03-12T04:12:56.120160+0000","last_peered":"2024-03-12T04:12:56.120160+0000","last_clean":"2024-03-12T04:12:56.120160+0000","last_became_active":"2024-03-12T04:12:54.793789+0000","last_became_peered":"2024-03-12T04:12:54.793789+0000","last_unstale":"2024-03-12T04:12:56.120160+0000","last_undegraded":"2024-03-12T04:12:56.120160+0000","last_fullsized":"2024-03-12T04:12:56.120160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:55:33.632297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.039470+0000","last_change":"2024-03-12T04:12:53.774847+0000","last_active":"2024-03-12T04:12:57.039470+0000","last_peered":"2024-03-12T04:12:57.039470+0000","last_clean":"2024-03-12T04:12:57.039470+0000","last_became_active":"2024-03-12T04:12:53.774457+0000","last_became_peered":"2024-03-12T04:12:53.774457+0000","last_unstale":"2024-03-12T04:12:57.039470+0000","last_undegraded":"2024-03-12T04:12:57.039470+0000","last_fullsized":"2024-03-12T04:12:57.039470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:17.350636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776530+0000","last_change":"2024-03-12T04:12:54.783671+0000","last_active":"2024-03-12T04:12:55.776530+0000","last_peered":"2024-03-12T04:12:55.776530+0000","last_clean":"2024-03-12T04:12:55.776530+0000","last_became_active":"2024-03-12T04:12:54.782919+0000","last_became_peered":"2024-03-12T04:12:54.782919+0000","last_unstale":"2024-03-12T04:12:55.776530+0000","last_undegraded":"2024-03-12T04:12:55.776530+0000","last_fullsized":"2024-03-12T04:12:55.776530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:38:19.445096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:58.045694+0000","last_change":"2024-03-12T04:12:53.773779+0000","last_active":"2024-03-12T04:12:58.045694+0000","last_peered":"2024-03-12T04:12:58.045694+0000","last_clean":"2024-03-12T04:12:58.045694+0000","last_became_active":"2024-03-12T04:12:53.772507+0000","last_became_peered":"2024-03-12T04:12:53.772507+0000","last_unstale":"2024-03-12T04:12:58.045694+0000","last_undegraded":"2024-03-12T04:12:58.045694+0000","last_fullsized":"2024-03-12T04:12:58.045694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:36:39.265551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119602+0000","last_change":"2024-03-12T04:12:54.779319+0000","last_active":"2024-03-12T04:12:56.119602+0000","last_peered":"2024-03-12T04:12:56.119602+0000","last_clean":"2024-03-12T04:12:56.119602+0000","last_became_active":"2024-03-12T04:12:54.778927+0000","last_became_peered":"2024-03-12T04:12:54.778927+0000","last_unstale":"2024-03-12T04:12:56.119602+0000","last_undegraded":"2024-03-12T04:12:56.119602+0000","last_fullsized":"2024-03-12T04:12:56.119602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:20:53.947257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.041471+0000","last_change":"2024-03-12T04:12:53.774754+0000","last_active":"2024-03-12T04:12:57.041471+0000","last_peered":"2024-03-12T04:12:57.041471+0000","last_clean":"2024-03-12T04:12:57.041471+0000","last_became_active":"2024-03-12T04:12:53.774395+0000","last_became_peered":"2024-03-12T04:12:53.774395+0000","last_unstale":"2024-03-12T04:12:57.041471+0000","last_undegraded":"2024-03-12T04:12:57.041471+0000","last_fullsized":"2024-03-12T04:12:57.041471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:13.286204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393710+0000","last_change":"2024-03-12T04:12:54.786891+0000","last_active":"2024-03-12T04:12:56.393710+0000","last_peered":"2024-03-12T04:12:56.393710+0000","last_clean":"2024-03-12T04:12:56.393710+0000","last_became_active":"2024-03-12T04:12:54.786591+0000","last_became_peered":"2024-03-12T04:12:54.786591+0000","last_unstale":"2024-03-12T04:12:56.393710+0000","last_undegraded":"2024-03-12T04:12:56.393710+0000","last_fullsized":"2024-03-12T04:12:56.393710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:16:09.188557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.056764+0000","last_change":"2024-03-12T04:12:53.769839+0000","last_active":"2024-03-12T04:12:59.056764+0000","last_peered":"2024-03-12T04:12:59.056764+0000","last_clean":"2024-03-12T04:12:59.056764+0000","last_became_active":"2024-03-12T04:12:53.769486+0000","last_became_peered":"2024-03-12T04:12:53.769486+0000","last_unstale":"2024-03-12T04:12:59.056764+0000","last_undegraded":"2024-03-12T04:12:59.056764+0000","last_fullsized":"2024-03-12T04:12:59.056764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:46.050018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393344+0000","last_change":"2024-03-12T04:12:54.782447+0000","last_active":"2024-03-12T04:12:56.393344+0000","last_peered":"2024-03-12T04:12:56.393344+0000","last_clean":"2024-03-12T04:12:56.393344+0000","last_became_active":"2024-03-12T04:12:54.782091+0000","last_became_peered":"2024-03-12T04:12:54.782091+0000","last_unstale":"2024-03-12T04:12:56.393344+0000","last_undegraded":"2024-03-12T04:12:56.393344+0000","last_fullsized":"2024-03-12T04:12:56.393344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:22.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775825+0000","last_change":"2024-03-12T04:12:53.777303+0000","last_active":"2024-03-12T04:12:55.775825+0000","last_peered":"2024-03-12T04:12:55.775825+0000","last_clean":"2024-03-12T04:12:55.775825+0000","last_became_active":"2024-03-12T04:12:53.776987+0000","last_became_peered":"2024-03-12T04:12:53.776987+0000","last_unstale":"2024-03-12T04:12:55.775825+0000","last_undegraded":"2024-03-12T04:12:55.775825+0000","last_fullsized":"2024-03-12T04:12:55.775825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:24:33.590904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120868+0000","last_change":"2024-03-12T04:12:54.784282+0000","last_active":"2024-03-12T04:12:56.120868+0000","last_peered":"2024-03-12T04:12:56.120868+0000","last_clean":"2024-03-12T04:12:56.120868+0000","last_became_active":"2024-03-12T04:12:54.783953+0000","last_became_peered":"2024-03-12T04:12:54.783953+0000","last_unstale":"2024-03-12T04:12:56.120868+0000","last_undegraded":"2024-03-12T04:12:56.120868+0000","last_fullsized":"2024-03-12T04:12:56.120868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:14:02.182546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776348+0000","last_change":"2024-03-12T04:12:53.770230+0000","last_active":"2024-03-12T04:12:55.776348+0000","last_peered":"2024-03-12T04:12:55.776348+0000","last_clean":"2024-03-12T04:12:55.776348+0000","last_became_active":"2024-03-12T04:12:53.768817+0000","last_became_peered":"2024-03-12T04:12:53.768817+0000","last_unstale":"2024-03-12T04:12:55.776348+0000","last_undegraded":"2024-03-12T04:12:55.776348+0000","last_fullsized":"2024-03-12T04:12:55.776348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:09:12.656489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120172+0000","last_change":"2024-03-12T04:12:54.785654+0000","last_active":"2024-03-12T04:12:56.120172+0000","last_peered":"2024-03-12T04:12:56.120172+0000","last_clean":"2024-03-12T04:12:56.120172+0000","last_became_active":"2024-03-12T04:12:54.785244+0000","last_became_peered":"2024-03-12T04:12:54.785244+0000","last_unstale":"2024-03-12T04:12:56.120172+0000","last_undegraded":"2024-03-12T04:12:56.120172+0000","last_fullsized":"2024-03-12T04:12:56.120172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:00:41.961637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120284+0000","last_change":"2024-03-12T04:12:53.780962+0000","last_active":"2024-03-12T04:12:56.120284+0000","last_peered":"2024-03-12T04:12:56.120284+0000","last_clean":"2024-03-12T04:12:56.120284+0000","last_became_active":"2024-03-12T04:12:53.780623+0000","last_became_peered":"2024-03-12T04:12:53.780623+0000","last_unstale":"2024-03-12T04:12:56.120284+0000","last_undegraded":"2024-03-12T04:12:56.120284+0000","last_fullsized":"2024-03-12T04:12:56.120284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776520+0000","last_change":"2024-03-12T04:12:54.775065+0000","last_active":"2024-03-12T04:12:55.776520+0000","last_peered":"2024-03-12T04:12:55.776520+0000","last_clean":"2024-03-12T04:12:55.776520+0000","last_became_active":"2024-03-12T04:12:54.774686+0000","last_became_peered":"2024-03-12T04:12:54.774686+0000","last_unstale":"2024-03-12T04:12:55.776520+0000","last_undegraded":"2024-03-12T04:12:55.776520+0000","last_fullsized":"2024-03-12T04:12:55.776520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:56:25.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393624+0000","last_change":"2024-03-12T04:12:53.777702+0000","last_active":"2024-03-12T04:12:56.393624+0000","last_peered":"2024-03-12T04:12:56.393624+0000","last_clean":"2024-03-12T04:12:56.393624+0000","last_became_active":"2024-03-12T04:12:53.777346+0000","last_became_peered":"2024-03-12T04:12:53.777346+0000","last_unstale":"2024-03-12T04:12:56.393624+0000","last_undegraded":"2024-03-12T04:12:56.393624+0000","last_fullsized":"2024-03-12T04:12:56.393624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:09.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776500+0000","last_change":"2024-03-12T04:12:54.789087+0000","last_active":"2024-03-12T04:12:55.776500+0000","last_peered":"2024-03-12T04:12:55.776500+0000","last_clean":"2024-03-12T04:12:55.776500+0000","last_became_active":"2024-03-12T04:12:54.788735+0000","last_became_peered":"2024-03-12T04:12:54.788735+0000","last_unstale":"2024-03-12T04:12:55.776500+0000","last_undegraded":"2024-03-12T04:12:55.776500+0000","last_fullsized":"2024-03-12T04:12:55.776500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:14.947137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120170+0000","last_change":"2024-03-12T04:12:54.789974+0000","last_active":"2024-03-12T04:12:56.120170+0000","last_peered":"2024-03-12T04:12:56.120170+0000","last_clean":"2024-03-12T04:12:56.120170+0000","last_became_active":"2024-03-12T04:12:54.789578+0000","last_became_peered":"2024-03-12T04:12:54.789578+0000","last_unstale":"2024-03-12T04:12:56.120170+0000","last_undegraded":"2024-03-12T04:12:56.120170+0000","last_fullsized":"2024-03-12T04:12:56.120170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:22:25.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119350+0000","last_change":"2024-03-12T04:12:53.773601+0000","last_active":"2024-03-12T04:12:56.119350+0000","last_peered":"2024-03-12T04:12:56.119350+0000","last_clean":"2024-03-12T04:12:56.119350+0000","last_became_active":"2024-03-12T04:12:53.773264+0000","last_became_peered":"2024-03-12T04:12:53.773264+0000","last_unstale":"2024-03-12T04:12:56.119350+0000","last_undegraded":"2024-03-12T04:12:56.119350+0000","last_fullsized":"2024-03-12T04:12:56.119350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:07:03.580157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120285+0000","last_change":"2024-03-12T04:12:54.782496+0000","last_active":"2024-03-12T04:12:56.120285+0000","last_peered":"2024-03-12T04:12:56.120285+0000","last_clean":"2024-03-12T04:12:56.120285+0000","last_became_active":"2024-03-12T04:12:54.782014+0000","last_became_peered":"2024-03-12T04:12:54.782014+0000","last_unstale":"2024-03-12T04:12:56.120285+0000","last_undegraded":"2024-03-12T04:12:56.120285+0000","last_fullsized":"2024-03-12T04:12:56.120285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:08.580655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.057141+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:12:59.057141+0000","last_peered":"2024-03-12T04:12:59.057141+0000","last_clean":"2024-03-12T04:12:59.057141+0000","last_became_active":"2024-03-12T04:12:53.770855+0000","last_became_peered":"2024-03-12T04:12:53.770855+0000","last_unstale":"2024-03-12T04:12:59.057141+0000","last_undegraded":"2024-03-12T04:12:59.057141+0000","last_fullsized":"2024-03-12T04:12:59.057141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:32:33.969240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393415+0000","last_change":"2024-03-12T04:11:37.642780+0000","last_active":"2024-03-12T04:12:56.393415+0000","last_peered":"2024-03-12T04:12:56.393415+0000","last_clean":"2024-03-12T04:12:56.393415+0000","last_became_active":"2024-03-12T04:11:37.642145+0000","last_became_peered":"2024-03-12T04:11:37.642145+0000","last_unstale":"2024-03-12T04:12:56.393415+0000","last_undegraded":"2024-03-12T04:12:56.393415+0000","last_fullsized":"2024-03-12T04:12:56.393415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_clean_scrub_stamp":"2024-03-12T04:08:09.604393+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:46:36.666483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119402+0000","last_change":"2024-03-12T04:12:54.790464+0000","last_active":"2024-03-12T04:12:56.119402+0000","last_peered":"2024-03-12T04:12:56.119402+0000","last_clean":"2024-03-12T04:12:56.119402+0000","last_became_active":"2024-03-12T04:12:54.788440+0000","last_became_peered":"2024-03-12T04:12:54.788440+0000","last_unstale":"2024-03-12T04:12:56.119402+0000","last_undegraded":"2024-03-12T04:12:56.119402+0000","last_fullsized":"2024-03-12T04:12:56.119402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:54.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393427+0000","last_change":"2024-03-12T04:12:53.773334+0000","last_active":"2024-03-12T04:12:56.393427+0000","last_peered":"2024-03-12T04:12:56.393427+0000","last_clean":"2024-03-12T04:12:56.393427+0000","last_became_active":"2024-03-12T04:12:53.772759+0000","last_became_peered":"2024-03-12T04:12:53.772759+0000","last_unstale":"2024-03-12T04:12:56.393427+0000","last_undegraded":"2024-03-12T04:12:56.393427+0000","last_fullsized":"2024-03-12T04:12:56.393427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:03:17.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776143+0000","last_change":"2024-03-12T04:12:53.777035+0000","last_active":"2024-03-12T04:12:55.776143+0000","last_peered":"2024-03-12T04:12:55.776143+0000","last_clean":"2024-03-12T04:12:55.776143+0000","last_became_active":"2024-03-12T04:12:53.776587+0000","last_became_peered":"2024-03-12T04:12:53.776587+0000","last_unstale":"2024-03-12T04:12:55.776143+0000","last_undegraded":"2024-03-12T04:12:55.776143+0000","last_fullsized":"2024-03-12T04:12:55.776143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:21.390525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120113+0000","last_change":"2024-03-12T04:12:54.790835+0000","last_active":"2024-03-12T04:12:56.120113+0000","last_peered":"2024-03-12T04:12:56.120113+0000","last_clean":"2024-03-12T04:12:56.120113+0000","last_became_active":"2024-03-12T04:12:54.790512+0000","last_became_peered":"2024-03-12T04:12:54.790512+0000","last_unstale":"2024-03-12T04:12:56.120113+0000","last_undegraded":"2024-03-12T04:12:56.120113+0000","last_fullsized":"2024-03-12T04:12:56.120113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:37:58.029450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.059728+0000","last_change":"2024-03-12T04:12:53.774604+0000","last_active":"2024-03-12T04:12:59.059728+0000","last_peered":"2024-03-12T04:12:59.059728+0000","last_clean":"2024-03-12T04:12:59.059728+0000","last_became_active":"2024-03-12T04:12:53.774270+0000","last_became_peered":"2024-03-12T04:12:53.774270+0000","last_unstale":"2024-03-12T04:12:59.059728+0000","last_undegraded":"2024-03-12T04:12:59.059728+0000","last_fullsized":"2024-03-12T04:12:59.059728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:23.635179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120069+0000","last_change":"2024-03-12T04:12:54.786259+0000","last_active":"2024-03-12T04:12:56.120069+0000","last_peered":"2024-03-12T04:12:56.120069+0000","last_clean":"2024-03-12T04:12:56.120069+0000","last_became_active":"2024-03-12T04:12:54.783114+0000","last_became_peered":"2024-03-12T04:12:54.783114+0000","last_unstale":"2024-03-12T04:12:56.120069+0000","last_undegraded":"2024-03-12T04:12:56.120069+0000","last_fullsized":"2024-03-12T04:12:56.120069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:53.036505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.042079+0000","last_change":"2024-03-12T04:12:53.779618+0000","last_active":"2024-03-12T04:12:57.042079+0000","last_peered":"2024-03-12T04:12:57.042079+0000","last_clean":"2024-03-12T04:12:57.042079+0000","last_became_active":"2024-03-12T04:12:53.779264+0000","last_became_peered":"2024-03-12T04:12:53.779264+0000","last_unstale":"2024-03-12T04:12:57.042079+0000","last_undegraded":"2024-03-12T04:12:57.042079+0000","last_fullsized":"2024-03-12T04:12:57.042079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:57.633014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775649+0000","last_change":"2024-03-12T04:12:54.789012+0000","last_active":"2024-03-12T04:12:55.775649+0000","last_peered":"2024-03-12T04:12:55.775649+0000","last_clean":"2024-03-12T04:12:55.775649+0000","last_became_active":"2024-03-12T04:12:54.788441+0000","last_became_peered":"2024-03-12T04:12:54.788441+0000","last_unstale":"2024-03-12T04:12:55.775649+0000","last_undegraded":"2024-03-12T04:12:55.775649+0000","last_fullsized":"2024-03-12T04:12:55.775649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:55.858926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.040979+0000","last_change":"2024-03-12T04:12:53.776621+0000","last_active":"2024-03-12T04:12:57.040979+0000","last_peered":"2024-03-12T04:12:57.040979+0000","last_clean":"2024-03-12T04:12:57.040979+0000","last_became_active":"2024-03-12T04:12:53.776313+0000","last_became_peered":"2024-03-12T04:12:53.776313+0000","last_unstale":"2024-03-12T04:12:57.040979+0000","last_undegraded":"2024-03-12T04:12:57.040979+0000","last_fullsized":"2024-03-12T04:12:57.040979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:43.536159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775980+0000","last_change":"2024-03-12T04:12:54.786929+0000","last_active":"2024-03-12T04:12:55.775980+0000","last_peered":"2024-03-12T04:12:55.775980+0000","last_clean":"2024-03-12T04:12:55.775980+0000","last_became_active":"2024-03-12T04:12:54.786580+0000","last_became_peered":"2024-03-12T04:12:54.786580+0000","last_unstale":"2024-03-12T04:12:55.775980+0000","last_undegraded":"2024-03-12T04:12:55.775980+0000","last_fullsized":"2024-03-12T04:12:55.775980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:58.955293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:13:00.034428+0000","last_change":"2024-03-12T04:12:53.778577+0000","last_active":"2024-03-12T04:13:00.034428+0000","last_peered":"2024-03-12T04:13:00.034428+0000","last_clean":"2024-03-12T04:13:00.034428+0000","last_became_active":"2024-03-12T04:12:53.778033+0000","last_became_peered":"2024-03-12T04:12:53.778033+0000","last_unstale":"2024-03-12T04:13:00.034428+0000","last_undegraded":"2024-03-12T04:13:00.034428+0000","last_fullsized":"2024-03-12T04:13:00.034428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:02.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119728+0000","last_change":"2024-03-12T04:12:54.785861+0000","last_active":"2024-03-12T04:12:56.119728+0000","last_peered":"2024-03-12T04:12:56.119728+0000","last_clean":"2024-03-12T04:12:56.119728+0000","last_became_active":"2024-03-12T04:12:54.784682+0000","last_became_peered":"2024-03-12T04:12:54.784682+0000","last_unstale":"2024-03-12T04:12:56.119728+0000","last_undegraded":"2024-03-12T04:12:56.119728+0000","last_fullsized":"2024-03-12T04:12:56.119728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:35:12.288309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:58.036286+0000","last_change":"2024-03-12T04:12:53.772865+0000","last_active":"2024-03-12T04:12:58.036286+0000","last_peered":"2024-03-12T04:12:58.036286+0000","last_clean":"2024-03-12T04:12:58.036286+0000","last_became_active":"2024-03-12T04:12:53.772502+0000","last_became_peered":"2024-03-12T04:12:53.772502+0000","last_unstale":"2024-03-12T04:12:58.036286+0000","last_undegraded":"2024-03-12T04:12:58.036286+0000","last_fullsized":"2024-03-12T04:12:58.036286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:40:24.112197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120550+0000","last_change":"2024-03-12T04:12:54.780722+0000","last_active":"2024-03-12T04:12:56.120550+0000","last_peered":"2024-03-12T04:12:56.120550+0000","last_clean":"2024-03-12T04:12:56.120550+0000","last_became_active":"2024-03-12T04:12:54.780404+0000","last_became_peered":"2024-03-12T04:12:54.780404+0000","last_unstale":"2024-03-12T04:12:56.120550+0000","last_undegraded":"2024-03-12T04:12:56.120550+0000","last_fullsized":"2024-03-12T04:12:56.120550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.068094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776032+0000","last_change":"2024-03-12T04:12:53.771471+0000","last_active":"2024-03-12T04:12:55.776032+0000","last_peered":"2024-03-12T04:12:55.776032+0000","last_clean":"2024-03-12T04:12:55.776032+0000","last_became_active":"2024-03-12T04:12:53.771147+0000","last_became_peered":"2024-03-12T04:12:53.771147+0000","last_unstale":"2024-03-12T04:12:55.776032+0000","last_undegraded":"2024-03-12T04:12:55.776032+0000","last_fullsized":"2024-03-12T04:12:55.776032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:46.159168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393503+0000","last_change":"2024-03-12T04:12:54.796253+0000","last_active":"2024-03-12T04:12:56.393503+0000","last_peered":"2024-03-12T04:12:56.393503+0000","last_clean":"2024-03-12T04:12:56.393503+0000","last_became_active":"2024-03-12T04:12:54.795656+0000","last_became_peered":"2024-03-12T04:12:54.795656+0000","last_unstale":"2024-03-12T04:12:56.393503+0000","last_undegraded":"2024-03-12T04:12:56.393503+0000","last_fullsized":"2024-03-12T04:12:56.393503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:39:32.788874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.059566+0000","last_change":"2024-03-12T04:12:53.780257+0000","last_active":"2024-03-12T04:12:59.059566+0000","last_peered":"2024-03-12T04:12:59.059566+0000","last_clean":"2024-03-12T04:12:59.059566+0000","last_became_active":"2024-03-12T04:12:53.779891+0000","last_became_peered":"2024-03-12T04:12:53.779891+0000","last_unstale":"2024-03-12T04:12:59.059566+0000","last_undegraded":"2024-03-12T04:12:59.059566+0000","last_fullsized":"2024-03-12T04:12:59.059566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:49:30.218372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393809+0000","last_change":"2024-03-12T04:12:54.792760+0000","last_active":"2024-03-12T04:12:56.393809+0000","last_peered":"2024-03-12T04:12:56.393809+0000","last_clean":"2024-03-12T04:12:56.393809+0000","last_became_active":"2024-03-12T04:12:54.792363+0000","last_became_peered":"2024-03-12T04:12:54.792363+0000","last_unstale":"2024-03-12T04:12:56.393809+0000","last_undegraded":"2024-03-12T04:12:56.393809+0000","last_fullsized":"2024-03-12T04:12:56.393809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:30:15.394318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120740+0000","last_change":"2024-03-12T04:12:53.775556+0000","last_active":"2024-03-12T04:12:56.120740+0000","last_peered":"2024-03-12T04:12:56.120740+0000","last_clean":"2024-03-12T04:12:56.120740+0000","last_became_active":"2024-03-12T04:12:53.775182+0000","last_became_peered":"2024-03-12T04:12:53.775182+0000","last_unstale":"2024-03-12T04:12:56.120740+0000","last_undegraded":"2024-03-12T04:12:56.120740+0000","last_fullsized":"2024-03-12T04:12:56.120740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:13:50.076672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393544+0000","last_change":"2024-03-12T04:12:54.795876+0000","last_active":"2024-03-12T04:12:56.393544+0000","last_peered":"2024-03-12T04:12:56.393544+0000","last_clean":"2024-03-12T04:12:56.393544+0000","last_became_active":"2024-03-12T04:12:54.793874+0000","last_became_peered":"2024-03-12T04:12:54.793874+0000","last_unstale":"2024-03-12T04:12:56.393544+0000","last_undegraded":"2024-03-12T04:12:56.393544+0000","last_fullsized":"2024-03-12T04:12:56.393544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:33:31.165974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120059+0000","last_change":"2024-03-12T04:12:53.774879+0000","last_active":"2024-03-12T04:12:56.120059+0000","last_peered":"2024-03-12T04:12:56.120059+0000","last_clean":"2024-03-12T04:12:56.120059+0000","last_became_active":"2024-03-12T04:12:53.774530+0000","last_became_peered":"2024-03-12T04:12:53.774530+0000","last_unstale":"2024-03-12T04:12:56.120059+0000","last_undegraded":"2024-03-12T04:12:56.120059+0000","last_fullsized":"2024-03-12T04:12:56.120059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:25.858165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393659+0000","last_change":"2024-03-12T04:12:54.790140+0000","last_active":"2024-03-12T04:12:56.393659+0000","last_peered":"2024-03-12T04:12:56.393659+0000","last_clean":"2024-03-12T04:12:56.393659+0000","last_became_active":"2024-03-12T04:12:54.789798+0000","last_became_peered":"2024-03-12T04:12:54.789798+0000","last_unstale":"2024-03-12T04:12:56.393659+0000","last_undegraded":"2024-03-12T04:12:56.393659+0000","last_fullsized":"2024-03-12T04:12:56.393659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:12:38.722638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119825+0000","last_change":"2024-03-12T04:12:54.785029+0000","last_active":"2024-03-12T04:12:56.119825+0000","last_peered":"2024-03-12T04:12:56.119825+0000","last_clean":"2024-03-12T04:12:56.119825+0000","last_became_active":"2024-03-12T04:12:54.784197+0000","last_became_peered":"2024-03-12T04:12:54.784197+0000","last_unstale":"2024-03-12T04:12:56.119825+0000","last_undegraded":"2024-03-12T04:12:56.119825+0000","last_fullsized":"2024-03-12T04:12:56.119825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:22:41.352093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120083+0000","last_change":"2024-03-12T04:12:54.793132+0000","last_active":"2024-03-12T04:12:56.120083+0000","last_peered":"2024-03-12T04:12:56.120083+0000","last_clean":"2024-03-12T04:12:56.120083+0000","last_became_active":"2024-03-12T04:12:54.792838+0000","last_became_peered":"2024-03-12T04:12:54.792838+0000","last_unstale":"2024-03-12T04:12:56.120083+0000","last_undegraded":"2024-03-12T04:12:56.120083+0000","last_fullsized":"2024-03-12T04:12:56.120083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:25:22.641963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119945+0000","last_change":"2024-03-12T04:12:54.794892+0000","last_active":"2024-03-12T04:12:56.119945+0000","last_peered":"2024-03-12T04:12:56.119945+0000","last_clean":"2024-03-12T04:12:56.119945+0000","last_became_active":"2024-03-12T04:12:54.792656+0000","last_became_peered":"2024-03-12T04:12:54.792656+0000","last_unstale":"2024-03-12T04:12:56.119945+0000","last_undegraded":"2024-03-12T04:12:56.119945+0000","last_fullsized":"2024-03-12T04:12:56.119945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:14.820628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775582+0000","last_change":"2024-03-12T04:12:54.785830+0000","last_active":"2024-03-12T04:12:55.775582+0000","last_peered":"2024-03-12T04:12:55.775582+0000","last_clean":"2024-03-12T04:12:55.775582+0000","last_became_active":"2024-03-12T04:12:54.785087+0000","last_became_peered":"2024-03-12T04:12:54.785087+0000","last_unstale":"2024-03-12T04:12:55.775582+0000","last_undegraded":"2024-03-12T04:12:55.775582+0000","last_fullsized":"2024-03-12T04:12:55.775582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:08:09.860311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119972+0000","last_change":"2024-03-12T04:12:54.787491+0000","last_active":"2024-03-12T04:12:56.119972+0000","last_peered":"2024-03-12T04:12:56.119972+0000","last_clean":"2024-03-12T04:12:56.119972+0000","last_became_active":"2024-03-12T04:12:54.787097+0000","last_became_peered":"2024-03-12T04:12:54.787097+0000","last_unstale":"2024-03-12T04:12:56.119972+0000","last_undegraded":"2024-03-12T04:12:56.119972+0000","last_fullsized":"2024-03-12T04:12:56.119972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:24:31.388053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120824+0000","last_change":"2024-03-12T04:12:54.781227+0000","last_active":"2024-03-12T04:12:56.120824+0000","last_peered":"2024-03-12T04:12:56.120824+0000","last_clean":"2024-03-12T04:12:56.120824+0000","last_became_active":"2024-03-12T04:12:54.780801+0000","last_became_peered":"2024-03-12T04:12:54.780801+0000","last_unstale":"2024-03-12T04:12:56.120824+0000","last_undegraded":"2024-03-12T04:12:56.120824+0000","last_fullsized":"2024-03-12T04:12:56.120824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:09:55.297944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120875+0000","last_change":"2024-03-12T04:12:54.796733+0000","last_active":"2024-03-12T04:12:56.120875+0000","last_peered":"2024-03-12T04:12:56.120875+0000","last_clean":"2024-03-12T04:12:56.120875+0000","last_became_active":"2024-03-12T04:12:54.796388+0000","last_became_peered":"2024-03-12T04:12:54.796388+0000","last_unstale":"2024-03-12T04:12:56.120875+0000","last_undegraded":"2024-03-12T04:12:56.120875+0000","last_fullsized":"2024-03-12T04:12:56.120875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:01.580286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120367+0000","last_change":"2024-03-12T04:12:54.787200+0000","last_active":"2024-03-12T04:12:56.120367+0000","last_peered":"2024-03-12T04:12:56.120367+0000","last_clean":"2024-03-12T04:12:56.120367+0000","last_became_active":"2024-03-12T04:12:54.786657+0000","last_became_peered":"2024-03-12T04:12:54.786657+0000","last_unstale":"2024-03-12T04:12:56.120367+0000","last_undegraded":"2024-03-12T04:12:56.120367+0000","last_fullsized":"2024-03-12T04:12:56.120367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:15:35.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119987+0000","last_change":"2024-03-12T04:12:54.785132+0000","last_active":"2024-03-12T04:12:56.119987+0000","last_peered":"2024-03-12T04:12:56.119987+0000","last_clean":"2024-03-12T04:12:56.119987+0000","last_became_active":"2024-03-12T04:12:54.782548+0000","last_became_peered":"2024-03-12T04:12:54.782548+0000","last_unstale":"2024-03-12T04:12:56.119987+0000","last_undegraded":"2024-03-12T04:12:56.119987+0000","last_fullsized":"2024-03-12T04:12:56.119987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:40:49.284778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119530+0000","last_change":"2024-03-12T04:12:54.790227+0000","last_active":"2024-03-12T04:12:56.119530+0000","last_peered":"2024-03-12T04:12:56.119530+0000","last_clean":"2024-03-12T04:12:56.119530+0000","last_became_active":"2024-03-12T04:12:54.789853+0000","last_became_peered":"2024-03-12T04:12:54.789853+0000","last_unstale":"2024-03-12T04:12:56.119530+0000","last_undegraded":"2024-03-12T04:12:56.119530+0000","last_fullsized":"2024-03-12T04:12:56.119530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:38:16.683661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119908+0000","last_change":"2024-03-12T04:12:54.796039+0000","last_active":"2024-03-12T04:12:56.119908+0000","last_peered":"2024-03-12T04:12:56.119908+0000","last_clean":"2024-03-12T04:12:56.119908+0000","last_became_active":"2024-03-12T04:12:54.795568+0000","last_became_peered":"2024-03-12T04:12:54.795568+0000","last_unstale":"2024-03-12T04:12:56.119908+0000","last_undegraded":"2024-03-12T04:12:56.119908+0000","last_fullsized":"2024-03-12T04:12:56.119908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:46.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119986+0000","last_change":"2024-03-12T04:12:54.785610+0000","last_active":"2024-03-12T04:12:56.119986+0000","last_peered":"2024-03-12T04:12:56.119986+0000","last_clean":"2024-03-12T04:12:56.119986+0000","last_became_active":"2024-03-12T04:12:54.785256+0000","last_became_peered":"2024-03-12T04:12:54.785256+0000","last_unstale":"2024-03-12T04:12:56.119986+0000","last_undegraded":"2024-03-12T04:12:56.119986+0000","last_fullsized":"2024-03-12T04:12:56.119986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:34.152361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120351+0000","last_change":"2024-03-12T04:12:54.787064+0000","last_active":"2024-03-12T04:12:56.120351+0000","last_peered":"2024-03-12T04:12:56.120351+0000","last_clean":"2024-03-12T04:12:56.120351+0000","last_became_active":"2024-03-12T04:12:54.786701+0000","last_became_peered":"2024-03-12T04:12:54.786701+0000","last_unstale":"2024-03-12T04:12:56.120351+0000","last_undegraded":"2024-03-12T04:12:56.120351+0000","last_fullsized":"2024-03-12T04:12:56.120351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:44:43.582171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250964,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7096,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":60010,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318285,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":441099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1042605,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743449,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":450487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005086,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":450273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":1032049,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":441099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":471146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":449759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":453253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":450077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":450335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1042666,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":39945,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T04:13:01.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T04:13:01.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 -- 172.21.15.42:0/994090322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 msgr2=0x7fef04101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 --2- 172.21.15.42:0/994090322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef04101790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feeec009900 tx=0x7feeec02bc30 comp rx=0 tx=0).stop 2024-03-12T04:13:01.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 -- 172.21.15.42:0/994090322 shutdown_connections 2024-03-12T04:13:01.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 --2- 172.21.15.42:0/994090322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef04101cd0 0x7fef04108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 --2- 172.21.15.42:0/994090322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef04101790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 --2- 172.21.15.42:0/994090322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef04100120 0x7fef04100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 -- 172.21.15.42:0/994090322 >> 172.21.15.42:0/994090322 conn(0x7fef040fb6b0 msgr2=0x7fef040fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:01.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 -- 172.21.15.42:0/994090322 shutdown_connections 2024-03-12T04:13:01.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.966+0000 7fef08ceb640 1 -- 172.21.15.42:0/994090322 wait complete. 2024-03-12T04:13:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 Processor -- start 2024-03-12T04:13:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 -- start start 2024-03-12T04:13:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef04100120 0x7fef04104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef041026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef04101cd0 0x7fef04102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef041033c0 con 0x7fef04101310 2024-03-12T04:13:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef04103530 con 0x7fef04100120 2024-03-12T04:13:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef08ceb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef041a1230 con 0x7fef04101cd0 2024-03-12T04:13:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef041026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef041026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35830/0 (socket says 172.21.15.42:35830) 2024-03-12T04:13:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 -- 172.21.15.42:0/3199151724 learned_addr learned my addr 172.21.15.42:0/3199151724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef0259c640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef04100120 0x7fef04104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef02d9d640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef04101cd0 0x7fef04102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 -- 172.21.15.42:0/3199151724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef04101cd0 msgr2=0x7fef04102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef04101cd0 0x7fef04102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 -- 172.21.15.42:0/3199151724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef04100120 msgr2=0x7fef04104060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef04100120 0x7fef04104060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7feefbfff640 1 -- 172.21.15.42:0/3199151724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009560 con 0x7fef04101310 2024-03-12T04:13:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.967+0000 7fef02d9d640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef04101cd0 0x7fef04102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.968+0000 7feefbfff640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef041026f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feeec009430 tx=0x7feeec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.968+0000 7feefb7fe640 1 -- 172.21.15.42:0/3199151724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec004510 con 0x7fef04101310 2024-03-12T04:13:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.968+0000 7feefb7fe640 1 -- 172.21.15.42:0/3199151724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feeec02de90 con 0x7fef04101310 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.968+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef041a1430 con 0x7fef04101310 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.968+0000 7feefb7fe640 1 -- 172.21.15.42:0/3199151724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec003cc0 con 0x7fef04101310 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.968+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef041a1940 con 0x7fef04101310 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.969+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0410b970 con 0x7fef04101310 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.970+0000 7feefb7fe640 1 -- 172.21.15.42:0/3199151724 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7feeec004fd0 con 0x7fef04101310 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.970+0000 7feefb7fe640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feef00a1ac0 0x7feef00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.970+0000 7fef0259c640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feef00a1ac0 0x7feef00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.970+0000 7feefb7fe640 1 -- 172.21.15.42:0/3199151724 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feeec0ef630 con 0x7fef04101310 2024-03-12T04:13:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.971+0000 7fef0259c640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feef00a1ac0 0x7feef00a3f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feef40058c0 tx=0x7feef400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:01.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:01.975+0000 7feefb7fe640 1 -- 172.21.15.42:0/3199151724 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feeec0b88a0 con 0x7fef04101310 2024-03-12T04:13:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.155+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef040fd8e0 con 0x7feef00a1ac0 2024-03-12T04:13:02.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.163+0000 7feefb7fe640 1 -- 172.21.15.42:0/3199151724 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258905 (secure 0 0 0) 0x7fef040fd8e0 con 0x7feef00a1ac0 2024-03-12T04:13:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:13:02.172 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-03-12T04:13:02.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feef00a1ac0 msgr2=0x7feef00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:02.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feef00a1ac0 0x7feef00a3f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feef40058c0 tx=0x7feef400a2d0 comp rx=0 tx=0).stop 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 msgr2=0x7fef041026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef041026f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feeec009430 tx=0x7feeec0042f0 comp rx=0 tx=0).stop 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 shutdown_connections 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef04101cd0 0x7fef04102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef04101310 0x7fef041026f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7feef00a1ac0 0x7feef00a3f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 --2- 172.21.15.42:0/3199151724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef04100120 0x7fef04104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 >> 172.21.15.42:0/3199151724 conn(0x7fef040fb6b0 msgr2=0x7fef040fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 shutdown_connections 2024-03-12T04:13:02.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:02.166+0000 7fef08ceb640 1 -- 172.21.15.42:0/3199151724 wait complete. 2024-03-12T04:13:02.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/630596108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:13:02.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/151819899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:02.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:02 smithi042 ceph-mon[32260]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:13:02.179 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-03-12T04:13:00.899408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597464,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":308,"num_write_kb":2979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":247,"ondisk_log_size":247,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":261,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373004,"kb_used_data":11328,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568820,"statfs":{"total":1151940427776,"available":1151558471680,"internally_reserved":0,"allocated":11599872,"data_stored":7174848,"data_compressed":108960,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7096,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.140985"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.121053+0000","last_change":"2024-03-12T04:12:54.788665+0000","last_active":"2024-03-12T04:12:56.121053+0000","last_peered":"2024-03-12T04:12:56.121053+0000","last_clean":"2024-03-12T04:12:56.121053+0000","last_became_active":"2024-03-12T04:12:54.788334+0000","last_became_peered":"2024-03-12T04:12:54.788334+0000","last_unstale":"2024-03-12T04:12:56.121053+0000","last_undegraded":"2024-03-12T04:12:56.121053+0000","last_fullsized":"2024-03-12T04:12:56.121053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:02.442543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775682+0000","last_change":"2024-03-12T04:12:54.791942+0000","last_active":"2024-03-12T04:12:55.775682+0000","last_peered":"2024-03-12T04:12:55.775682+0000","last_clean":"2024-03-12T04:12:55.775682+0000","last_became_active":"2024-03-12T04:12:54.791579+0000","last_became_peered":"2024-03-12T04:12:54.791579+0000","last_unstale":"2024-03-12T04:12:55.775682+0000","last_undegraded":"2024-03-12T04:12:55.775682+0000","last_fullsized":"2024-03-12T04:12:55.775682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:51:08.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775581+0000","last_change":"2024-03-12T04:12:54.793238+0000","last_active":"2024-03-12T04:12:55.775581+0000","last_peered":"2024-03-12T04:12:55.775581+0000","last_clean":"2024-03-12T04:12:55.775581+0000","last_became_active":"2024-03-12T04:12:54.792877+0000","last_became_peered":"2024-03-12T04:12:54.792877+0000","last_unstale":"2024-03-12T04:12:55.775581+0000","last_undegraded":"2024-03-12T04:12:55.775581+0000","last_fullsized":"2024-03-12T04:12:55.775581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:36:46.914204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120602+0000","last_change":"2024-03-12T04:12:54.797573+0000","last_active":"2024-03-12T04:12:56.120602+0000","last_peered":"2024-03-12T04:12:56.120602+0000","last_clean":"2024-03-12T04:12:56.120602+0000","last_became_active":"2024-03-12T04:12:54.796939+0000","last_became_peered":"2024-03-12T04:12:54.796939+0000","last_unstale":"2024-03-12T04:12:56.120602+0000","last_undegraded":"2024-03-12T04:12:56.120602+0000","last_fullsized":"2024-03-12T04:12:56.120602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:34:58.047492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775990+0000","last_change":"2024-03-12T04:12:54.780595+0000","last_active":"2024-03-12T04:12:55.775990+0000","last_peered":"2024-03-12T04:12:55.775990+0000","last_clean":"2024-03-12T04:12:55.775990+0000","last_became_active":"2024-03-12T04:12:54.780256+0000","last_became_peered":"2024-03-12T04:12:54.780256+0000","last_unstale":"2024-03-12T04:12:55.775990+0000","last_undegraded":"2024-03-12T04:12:55.775990+0000","last_fullsized":"2024-03-12T04:12:55.775990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:42.492006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776238+0000","last_change":"2024-03-12T04:12:54.781573+0000","last_active":"2024-03-12T04:12:55.776238+0000","last_peered":"2024-03-12T04:12:55.776238+0000","last_clean":"2024-03-12T04:12:55.776238+0000","last_became_active":"2024-03-12T04:12:54.781230+0000","last_became_peered":"2024-03-12T04:12:54.781230+0000","last_unstale":"2024-03-12T04:12:55.776238+0000","last_undegraded":"2024-03-12T04:12:55.776238+0000","last_fullsized":"2024-03-12T04:12:55.776238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:16.869952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393320+0000","last_change":"2024-03-12T04:12:54.781510+0000","last_active":"2024-03-12T04:12:56.393320+0000","last_peered":"2024-03-12T04:12:56.393320+0000","last_clean":"2024-03-12T04:12:56.393320+0000","last_became_active":"2024-03-12T04:12:54.781141+0000","last_became_peered":"2024-03-12T04:12:54.781141+0000","last_unstale":"2024-03-12T04:12:56.393320+0000","last_undegraded":"2024-03-12T04:12:56.393320+0000","last_fullsized":"2024-03-12T04:12:56.393320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:42:05.055308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120330+0000","last_change":"2024-03-12T04:12:54.782541+0000","last_active":"2024-03-12T04:12:56.120330+0000","last_peered":"2024-03-12T04:12:56.120330+0000","last_clean":"2024-03-12T04:12:56.120330+0000","last_became_active":"2024-03-12T04:12:54.781582+0000","last_became_peered":"2024-03-12T04:12:54.781582+0000","last_unstale":"2024-03-12T04:12:56.120330+0000","last_undegraded":"2024-03-12T04:12:56.120330+0000","last_fullsized":"2024-03-12T04:12:56.120330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:15.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120792+0000","last_change":"2024-03-12T04:12:54.788201+0000","last_active":"2024-03-12T04:12:56.120792+0000","last_peered":"2024-03-12T04:12:56.120792+0000","last_clean":"2024-03-12T04:12:56.120792+0000","last_became_active":"2024-03-12T04:12:54.787875+0000","last_became_peered":"2024-03-12T04:12:54.787875+0000","last_unstale":"2024-03-12T04:12:56.120792+0000","last_undegraded":"2024-03-12T04:12:56.120792+0000","last_fullsized":"2024-03-12T04:12:56.120792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:40:39.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776604+0000","last_change":"2024-03-12T04:12:54.789748+0000","last_active":"2024-03-12T04:12:55.776604+0000","last_peered":"2024-03-12T04:12:55.776604+0000","last_clean":"2024-03-12T04:12:55.776604+0000","last_became_active":"2024-03-12T04:12:54.788838+0000","last_became_peered":"2024-03-12T04:12:54.788838+0000","last_unstale":"2024-03-12T04:12:55.776604+0000","last_undegraded":"2024-03-12T04:12:55.776604+0000","last_fullsized":"2024-03-12T04:12:55.776604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:39:51.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393582+0000","last_change":"2024-03-12T04:12:54.793101+0000","last_active":"2024-03-12T04:12:56.393582+0000","last_peered":"2024-03-12T04:12:56.393582+0000","last_clean":"2024-03-12T04:12:56.393582+0000","last_became_active":"2024-03-12T04:12:54.792488+0000","last_became_peered":"2024-03-12T04:12:54.792488+0000","last_unstale":"2024-03-12T04:12:56.393582+0000","last_undegraded":"2024-03-12T04:12:56.393582+0000","last_fullsized":"2024-03-12T04:12:56.393582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:33.862011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120600+0000","last_change":"2024-03-12T04:12:54.797953+0000","last_active":"2024-03-12T04:12:56.120600+0000","last_peered":"2024-03-12T04:12:56.120600+0000","last_clean":"2024-03-12T04:12:56.120600+0000","last_became_active":"2024-03-12T04:12:54.797413+0000","last_became_peered":"2024-03-12T04:12:54.797413+0000","last_unstale":"2024-03-12T04:12:56.120600+0000","last_undegraded":"2024-03-12T04:12:56.120600+0000","last_fullsized":"2024-03-12T04:12:56.120600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:18:01.140872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120135+0000","last_change":"2024-03-12T04:12:54.775570+0000","last_active":"2024-03-12T04:12:56.120135+0000","last_peered":"2024-03-12T04:12:56.120135+0000","last_clean":"2024-03-12T04:12:56.120135+0000","last_became_active":"2024-03-12T04:12:54.775186+0000","last_became_peered":"2024-03-12T04:12:54.775186+0000","last_unstale":"2024-03-12T04:12:56.120135+0000","last_undegraded":"2024-03-12T04:12:56.120135+0000","last_fullsized":"2024-03-12T04:12:56.120135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:26.004458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775390+0000","last_change":"2024-03-12T04:12:54.780859+0000","last_active":"2024-03-12T04:12:55.775390+0000","last_peered":"2024-03-12T04:12:55.775390+0000","last_clean":"2024-03-12T04:12:55.775390+0000","last_became_active":"2024-03-12T04:12:54.780484+0000","last_became_peered":"2024-03-12T04:12:54.780484+0000","last_unstale":"2024-03-12T04:12:55.775390+0000","last_undegraded":"2024-03-12T04:12:55.775390+0000","last_fullsized":"2024-03-12T04:12:55.775390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:24:00.575324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775375+0000","last_change":"2024-03-12T04:12:54.796087+0000","last_active":"2024-03-12T04:12:55.775375+0000","last_peered":"2024-03-12T04:12:55.775375+0000","last_clean":"2024-03-12T04:12:55.775375+0000","last_became_active":"2024-03-12T04:12:54.793702+0000","last_became_peered":"2024-03-12T04:12:54.793702+0000","last_unstale":"2024-03-12T04:12:55.775375+0000","last_undegraded":"2024-03-12T04:12:55.775375+0000","last_fullsized":"2024-03-12T04:12:55.775375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:09:27.334791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120170+0000","last_change":"2024-03-12T04:12:54.783146+0000","last_active":"2024-03-12T04:12:56.120170+0000","last_peered":"2024-03-12T04:12:56.120170+0000","last_clean":"2024-03-12T04:12:56.120170+0000","last_became_active":"2024-03-12T04:12:54.782834+0000","last_became_peered":"2024-03-12T04:12:54.782834+0000","last_unstale":"2024-03-12T04:12:56.120170+0000","last_undegraded":"2024-03-12T04:12:56.120170+0000","last_fullsized":"2024-03-12T04:12:56.120170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:35:11.126629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.121227+0000","last_change":"2024-03-12T04:12:54.794879+0000","last_active":"2024-03-12T04:12:56.121227+0000","last_peered":"2024-03-12T04:12:56.121227+0000","last_clean":"2024-03-12T04:12:56.121227+0000","last_became_active":"2024-03-12T04:12:54.794251+0000","last_became_peered":"2024-03-12T04:12:54.794251+0000","last_unstale":"2024-03-12T04:12:56.121227+0000","last_undegraded":"2024-03-12T04:12:56.121227+0000","last_fullsized":"2024-03-12T04:12:56.121227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:33.754426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119871+0000","last_change":"2024-03-12T04:12:54.791301+0000","last_active":"2024-03-12T04:12:56.119871+0000","last_peered":"2024-03-12T04:12:56.119871+0000","last_clean":"2024-03-12T04:12:56.119871+0000","last_became_active":"2024-03-12T04:12:54.790850+0000","last_became_peered":"2024-03-12T04:12:54.790850+0000","last_unstale":"2024-03-12T04:12:56.119871+0000","last_undegraded":"2024-03-12T04:12:56.119871+0000","last_fullsized":"2024-03-12T04:12:56.119871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:26:04.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120305+0000","last_change":"2024-03-12T04:12:54.793419+0000","last_active":"2024-03-12T04:12:56.120305+0000","last_peered":"2024-03-12T04:12:56.120305+0000","last_clean":"2024-03-12T04:12:56.120305+0000","last_became_active":"2024-03-12T04:12:54.793080+0000","last_became_peered":"2024-03-12T04:12:54.793080+0000","last_unstale":"2024-03-12T04:12:56.120305+0000","last_undegraded":"2024-03-12T04:12:56.120305+0000","last_fullsized":"2024-03-12T04:12:56.120305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:46:26.506899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'11","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.058934+0000","last_change":"2024-03-12T04:12:53.774893+0000","last_active":"2024-03-12T04:12:59.058934+0000","last_peered":"2024-03-12T04:12:59.058934+0000","last_clean":"2024-03-12T04:12:59.058934+0000","last_became_active":"2024-03-12T04:12:53.773801+0000","last_became_peered":"2024-03-12T04:12:53.773801+0000","last_unstale":"2024-03-12T04:12:59.058934+0000","last_undegraded":"2024-03-12T04:12:59.058934+0000","last_fullsized":"2024-03-12T04:12:59.058934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:30:19.995765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120353+0000","last_change":"2024-03-12T04:12:54.786795+0000","last_active":"2024-03-12T04:12:56.120353+0000","last_peered":"2024-03-12T04:12:56.120353+0000","last_clean":"2024-03-12T04:12:56.120353+0000","last_became_active":"2024-03-12T04:12:54.786481+0000","last_became_peered":"2024-03-12T04:12:54.786481+0000","last_unstale":"2024-03-12T04:12:56.120353+0000","last_undegraded":"2024-03-12T04:12:56.120353+0000","last_fullsized":"2024-03-12T04:12:56.120353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:06.849644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120687+0000","last_change":"2024-03-12T04:12:53.779582+0000","last_active":"2024-03-12T04:12:56.120687+0000","last_peered":"2024-03-12T04:12:56.120687+0000","last_clean":"2024-03-12T04:12:56.120687+0000","last_became_active":"2024-03-12T04:12:53.779062+0000","last_became_peered":"2024-03-12T04:12:53.779062+0000","last_unstale":"2024-03-12T04:12:56.120687+0000","last_undegraded":"2024-03-12T04:12:56.120687+0000","last_fullsized":"2024-03-12T04:12:56.120687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:44:01.065183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775737+0000","last_change":"2024-03-12T04:12:54.789198+0000","last_active":"2024-03-12T04:12:55.775737+0000","last_peered":"2024-03-12T04:12:55.775737+0000","last_clean":"2024-03-12T04:12:55.775737+0000","last_became_active":"2024-03-12T04:12:54.788213+0000","last_became_peered":"2024-03-12T04:12:54.788213+0000","last_unstale":"2024-03-12T04:12:55.775737+0000","last_undegraded":"2024-03-12T04:12:55.775737+0000","last_fullsized":"2024-03-12T04:12:55.775737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:24:36.561553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776364+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:12:55.776364+0000","last_peered":"2024-03-12T04:12:55.776364+0000","last_clean":"2024-03-12T04:12:55.776364+0000","last_became_active":"2024-03-12T04:12:53.771178+0000","last_became_peered":"2024-03-12T04:12:53.771178+0000","last_unstale":"2024-03-12T04:12:55.776364+0000","last_undegraded":"2024-03-12T04:12:55.776364+0000","last_fullsized":"2024-03-12T04:12:55.776364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:12.882128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119957+0000","last_change":"2024-03-12T04:12:54.799758+0000","last_active":"2024-03-12T04:12:56.119957+0000","last_peered":"2024-03-12T04:12:56.119957+0000","last_clean":"2024-03-12T04:12:56.119957+0000","last_became_active":"2024-03-12T04:12:54.799338+0000","last_became_peered":"2024-03-12T04:12:54.799338+0000","last_unstale":"2024-03-12T04:12:56.119957+0000","last_undegraded":"2024-03-12T04:12:56.119957+0000","last_fullsized":"2024-03-12T04:12:56.119957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:32:54.087890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776196+0000","last_change":"2024-03-12T04:12:53.774390+0000","last_active":"2024-03-12T04:12:55.776196+0000","last_peered":"2024-03-12T04:12:55.776196+0000","last_clean":"2024-03-12T04:12:55.776196+0000","last_became_active":"2024-03-12T04:12:53.774042+0000","last_became_peered":"2024-03-12T04:12:53.774042+0000","last_unstale":"2024-03-12T04:12:55.776196+0000","last_undegraded":"2024-03-12T04:12:55.776196+0000","last_fullsized":"2024-03-12T04:12:55.776196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:37.361505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.121102+0000","last_change":"2024-03-12T04:12:54.783478+0000","last_active":"2024-03-12T04:12:56.121102+0000","last_peered":"2024-03-12T04:12:56.121102+0000","last_clean":"2024-03-12T04:12:56.121102+0000","last_became_active":"2024-03-12T04:12:54.783080+0000","last_became_peered":"2024-03-12T04:12:54.783080+0000","last_unstale":"2024-03-12T04:12:56.121102+0000","last_undegraded":"2024-03-12T04:12:56.121102+0000","last_fullsized":"2024-03-12T04:12:56.121102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:57:48.639090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.042994+0000","last_change":"2024-03-12T04:12:53.772182+0000","last_active":"2024-03-12T04:12:57.042994+0000","last_peered":"2024-03-12T04:12:57.042994+0000","last_clean":"2024-03-12T04:12:57.042994+0000","last_became_active":"2024-03-12T04:12:53.771858+0000","last_became_peered":"2024-03-12T04:12:53.771858+0000","last_unstale":"2024-03-12T04:12:57.042994+0000","last_undegraded":"2024-03-12T04:12:57.042994+0000","last_fullsized":"2024-03-12T04:12:57.042994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:42:41.466311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776120+0000","last_change":"2024-03-12T04:12:54.779869+0000","last_active":"2024-03-12T04:12:55.776120+0000","last_peered":"2024-03-12T04:12:55.776120+0000","last_clean":"2024-03-12T04:12:55.776120+0000","last_became_active":"2024-03-12T04:12:54.779500+0000","last_became_peered":"2024-03-12T04:12:54.779500+0000","last_unstale":"2024-03-12T04:12:55.776120+0000","last_undegraded":"2024-03-12T04:12:55.776120+0000","last_fullsized":"2024-03-12T04:12:55.776120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:29.538570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:58.047421+0000","last_change":"2024-03-12T04:12:53.774056+0000","last_active":"2024-03-12T04:12:58.047421+0000","last_peered":"2024-03-12T04:12:58.047421+0000","last_clean":"2024-03-12T04:12:58.047421+0000","last_became_active":"2024-03-12T04:12:53.773675+0000","last_became_peered":"2024-03-12T04:12:53.773675+0000","last_unstale":"2024-03-12T04:12:58.047421+0000","last_undegraded":"2024-03-12T04:12:58.047421+0000","last_fullsized":"2024-03-12T04:12:58.047421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:33.576947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120839+0000","last_change":"2024-03-12T04:12:54.790583+0000","last_active":"2024-03-12T04:12:56.120839+0000","last_peered":"2024-03-12T04:12:56.120839+0000","last_clean":"2024-03-12T04:12:56.120839+0000","last_became_active":"2024-03-12T04:12:54.788364+0000","last_became_peered":"2024-03-12T04:12:54.788364+0000","last_unstale":"2024-03-12T04:12:56.120839+0000","last_undegraded":"2024-03-12T04:12:56.120839+0000","last_fullsized":"2024-03-12T04:12:56.120839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:05:13.542279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119451+0000","last_change":"2024-03-12T04:12:53.769619+0000","last_active":"2024-03-12T04:12:56.119451+0000","last_peered":"2024-03-12T04:12:56.119451+0000","last_clean":"2024-03-12T04:12:56.119451+0000","last_became_active":"2024-03-12T04:12:53.769293+0000","last_became_peered":"2024-03-12T04:12:53.769293+0000","last_unstale":"2024-03-12T04:12:56.119451+0000","last_undegraded":"2024-03-12T04:12:56.119451+0000","last_fullsized":"2024-03-12T04:12:56.119451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:44.618499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120232+0000","last_change":"2024-03-12T04:12:54.776493+0000","last_active":"2024-03-12T04:12:56.120232+0000","last_peered":"2024-03-12T04:12:56.120232+0000","last_clean":"2024-03-12T04:12:56.120232+0000","last_became_active":"2024-03-12T04:12:54.775967+0000","last_became_peered":"2024-03-12T04:12:54.775967+0000","last_unstale":"2024-03-12T04:12:56.120232+0000","last_undegraded":"2024-03-12T04:12:56.120232+0000","last_fullsized":"2024-03-12T04:12:56.120232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:19:46.824145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.041412+0000","last_change":"2024-03-12T04:12:53.773523+0000","last_active":"2024-03-12T04:12:57.041412+0000","last_peered":"2024-03-12T04:12:57.041412+0000","last_clean":"2024-03-12T04:12:57.041412+0000","last_became_active":"2024-03-12T04:12:53.773161+0000","last_became_peered":"2024-03-12T04:12:53.773161+0000","last_unstale":"2024-03-12T04:12:57.041412+0000","last_undegraded":"2024-03-12T04:12:57.041412+0000","last_fullsized":"2024-03-12T04:12:57.041412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:41:40.810994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120661+0000","last_change":"2024-03-12T04:12:54.791688+0000","last_active":"2024-03-12T04:12:56.120661+0000","last_peered":"2024-03-12T04:12:56.120661+0000","last_clean":"2024-03-12T04:12:56.120661+0000","last_became_active":"2024-03-12T04:12:54.790854+0000","last_became_peered":"2024-03-12T04:12:54.790854+0000","last_unstale":"2024-03-12T04:12:56.120661+0000","last_undegraded":"2024-03-12T04:12:56.120661+0000","last_fullsized":"2024-03-12T04:12:56.120661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:42:50.343524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.039970+0000","last_change":"2024-03-12T04:12:53.777534+0000","last_active":"2024-03-12T04:12:57.039970+0000","last_peered":"2024-03-12T04:12:57.039970+0000","last_clean":"2024-03-12T04:12:57.039970+0000","last_became_active":"2024-03-12T04:12:53.777201+0000","last_became_peered":"2024-03-12T04:12:53.777201+0000","last_unstale":"2024-03-12T04:12:57.039970+0000","last_undegraded":"2024-03-12T04:12:57.039970+0000","last_fullsized":"2024-03-12T04:12:57.039970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:54.708892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120679+0000","last_change":"2024-03-12T04:12:54.798962+0000","last_active":"2024-03-12T04:12:56.120679+0000","last_peered":"2024-03-12T04:12:56.120679+0000","last_clean":"2024-03-12T04:12:56.120679+0000","last_became_active":"2024-03-12T04:12:54.798650+0000","last_became_peered":"2024-03-12T04:12:54.798650+0000","last_unstale":"2024-03-12T04:12:56.120679+0000","last_undegraded":"2024-03-12T04:12:56.120679+0000","last_fullsized":"2024-03-12T04:12:56.120679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:05:29.270242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393602+0000","last_change":"2024-03-12T04:12:53.777791+0000","last_active":"2024-03-12T04:12:56.393602+0000","last_peered":"2024-03-12T04:12:56.393602+0000","last_clean":"2024-03-12T04:12:56.393602+0000","last_became_active":"2024-03-12T04:12:53.777430+0000","last_became_peered":"2024-03-12T04:12:53.777430+0000","last_unstale":"2024-03-12T04:12:56.393602+0000","last_undegraded":"2024-03-12T04:12:56.393602+0000","last_fullsized":"2024-03-12T04:12:56.393602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:38:21.858857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775707+0000","last_change":"2024-03-12T04:12:54.784710+0000","last_active":"2024-03-12T04:12:55.775707+0000","last_peered":"2024-03-12T04:12:55.775707+0000","last_clean":"2024-03-12T04:12:55.775707+0000","last_became_active":"2024-03-12T04:12:54.784342+0000","last_became_peered":"2024-03-12T04:12:54.784342+0000","last_unstale":"2024-03-12T04:12:55.775707+0000","last_undegraded":"2024-03-12T04:12:55.775707+0000","last_fullsized":"2024-03-12T04:12:55.775707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:21.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.040427+0000","last_change":"2024-03-12T04:12:53.779289+0000","last_active":"2024-03-12T04:12:57.040427+0000","last_peered":"2024-03-12T04:12:57.040427+0000","last_clean":"2024-03-12T04:12:57.040427+0000","last_became_active":"2024-03-12T04:12:53.778894+0000","last_became_peered":"2024-03-12T04:12:53.778894+0000","last_unstale":"2024-03-12T04:12:57.040427+0000","last_undegraded":"2024-03-12T04:12:57.040427+0000","last_fullsized":"2024-03-12T04:12:57.040427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:13:33.328977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120160+0000","last_change":"2024-03-12T04:12:54.794102+0000","last_active":"2024-03-12T04:12:56.120160+0000","last_peered":"2024-03-12T04:12:56.120160+0000","last_clean":"2024-03-12T04:12:56.120160+0000","last_became_active":"2024-03-12T04:12:54.793789+0000","last_became_peered":"2024-03-12T04:12:54.793789+0000","last_unstale":"2024-03-12T04:12:56.120160+0000","last_undegraded":"2024-03-12T04:12:56.120160+0000","last_fullsized":"2024-03-12T04:12:56.120160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:55:33.632297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.039470+0000","last_change":"2024-03-12T04:12:53.774847+0000","last_active":"2024-03-12T04:12:57.039470+0000","last_peered":"2024-03-12T04:12:57.039470+0000","last_clean":"2024-03-12T04:12:57.039470+0000","last_became_active":"2024-03-12T04:12:53.774457+0000","last_became_peered":"2024-03-12T04:12:53.774457+0000","last_unstale":"2024-03-12T04:12:57.039470+0000","last_undegraded":"2024-03-12T04:12:57.039470+0000","last_fullsized":"2024-03-12T04:12:57.039470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:17.350636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776530+0000","last_change":"2024-03-12T04:12:54.783671+0000","last_active":"2024-03-12T04:12:55.776530+0000","last_peered":"2024-03-12T04:12:55.776530+0000","last_clean":"2024-03-12T04:12:55.776530+0000","last_became_active":"2024-03-12T04:12:54.782919+0000","last_became_peered":"2024-03-12T04:12:54.782919+0000","last_unstale":"2024-03-12T04:12:55.776530+0000","last_undegraded":"2024-03-12T04:12:55.776530+0000","last_fullsized":"2024-03-12T04:12:55.776530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:38:19.445096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:58.045694+0000","last_change":"2024-03-12T04:12:53.773779+0000","last_active":"2024-03-12T04:12:58.045694+0000","last_peered":"2024-03-12T04:12:58.045694+0000","last_clean":"2024-03-12T04:12:58.045694+0000","last_became_active":"2024-03-12T04:12:53.772507+0000","last_became_peered":"2024-03-12T04:12:53.772507+0000","last_unstale":"2024-03-12T04:12:58.045694+0000","last_undegraded":"2024-03-12T04:12:58.045694+0000","last_fullsized":"2024-03-12T04:12:58.045694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:36:39.265551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119602+0000","last_change":"2024-03-12T04:12:54.779319+0000","last_active":"2024-03-12T04:12:56.119602+0000","last_peered":"2024-03-12T04:12:56.119602+0000","last_clean":"2024-03-12T04:12:56.119602+0000","last_became_active":"2024-03-12T04:12:54.778927+0000","last_became_peered":"2024-03-12T04:12:54.778927+0000","last_unstale":"2024-03-12T04:12:56.119602+0000","last_undegraded":"2024-03-12T04:12:56.119602+0000","last_fullsized":"2024-03-12T04:12:56.119602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:20:53.947257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.041471+0000","last_change":"2024-03-12T04:12:53.774754+0000","last_active":"2024-03-12T04:12:57.041471+0000","last_peered":"2024-03-12T04:12:57.041471+0000","last_clean":"2024-03-12T04:12:57.041471+0000","last_became_active":"2024-03-12T04:12:53.774395+0000","last_became_peered":"2024-03-12T04:12:53.774395+0000","last_unstale":"2024-03-12T04:12:57.041471+0000","last_undegraded":"2024-03-12T04:12:57.041471+0000","last_fullsized":"2024-03-12T04:12:57.041471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:13.286204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393710+0000","last_change":"2024-03-12T04:12:54.786891+0000","last_active":"2024-03-12T04:12:56.393710+0000","last_peered":"2024-03-12T04:12:56.393710+0000","last_clean":"2024-03-12T04:12:56.393710+0000","last_became_active":"2024-03-12T04:12:54.786591+0000","last_became_peered":"2024-03-12T04:12:54.786591+0000","last_unstale":"2024-03-12T04:12:56.393710+0000","last_undegraded":"2024-03-12T04:12:56.393710+0000","last_fullsized":"2024-03-12T04:12:56.393710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:16:09.188557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.056764+0000","last_change":"2024-03-12T04:12:53.769839+0000","last_active":"2024-03-12T04:12:59.056764+0000","last_peered":"2024-03-12T04:12:59.056764+0000","last_clean":"2024-03-12T04:12:59.056764+0000","last_became_active":"2024-03-12T04:12:53.769486+0000","last_became_peered":"2024-03-12T04:12:53.769486+0000","last_unstale":"2024-03-12T04:12:59.056764+0000","last_undegraded":"2024-03-12T04:12:59.056764+0000","last_fullsized":"2024-03-12T04:12:59.056764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:46.050018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393344+0000","last_change":"2024-03-12T04:12:54.782447+0000","last_active":"2024-03-12T04:12:56.393344+0000","last_peered":"2024-03-12T04:12:56.393344+0000","last_clean":"2024-03-12T04:12:56.393344+0000","last_became_active":"2024-03-12T04:12:54.782091+0000","last_became_peered":"2024-03-12T04:12:54.782091+0000","last_unstale":"2024-03-12T04:12:56.393344+0000","last_undegraded":"2024-03-12T04:12:56.393344+0000","last_fullsized":"2024-03-12T04:12:56.393344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:22.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775825+0000","last_change":"2024-03-12T04:12:53.777303+0000","last_active":"2024-03-12T04:12:55.775825+0000","last_peered":"2024-03-12T04:12:55.775825+0000","last_clean":"2024-03-12T04:12:55.775825+0000","last_became_active":"2024-03-12T04:12:53.776987+0000","last_became_peered":"2024-03-12T04:12:53.776987+0000","last_unstale":"2024-03-12T04:12:55.775825+0000","last_undegraded":"2024-03-12T04:12:55.775825+0000","last_fullsized":"2024-03-12T04:12:55.775825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:24:33.590904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120868+0000","last_change":"2024-03-12T04:12:54.784282+0000","last_active":"2024-03-12T04:12:56.120868+0000","last_peered":"2024-03-12T04:12:56.120868+0000","last_clean":"2024-03-12T04:12:56.120868+0000","last_became_active":"2024-03-12T04:12:54.783953+0000","last_became_peered":"2024-03-12T04:12:54.783953+0000","last_unstale":"2024-03-12T04:12:56.120868+0000","last_undegraded":"2024-03-12T04:12:56.120868+0000","last_fullsized":"2024-03-12T04:12:56.120868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:14:02.182546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776348+0000","last_change":"2024-03-12T04:12:53.770230+0000","last_active":"2024-03-12T04:12:55.776348+0000","last_peered":"2024-03-12T04:12:55.776348+0000","last_clean":"2024-03-12T04:12:55.776348+0000","last_became_active":"2024-03-12T04:12:53.768817+0000","last_became_peered":"2024-03-12T04:12:53.768817+0000","last_unstale":"2024-03-12T04:12:55.776348+0000","last_undegraded":"2024-03-12T04:12:55.776348+0000","last_fullsized":"2024-03-12T04:12:55.776348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:09:12.656489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120172+0000","last_change":"2024-03-12T04:12:54.785654+0000","last_active":"2024-03-12T04:12:56.120172+0000","last_peered":"2024-03-12T04:12:56.120172+0000","last_clean":"2024-03-12T04:12:56.120172+0000","last_became_active":"2024-03-12T04:12:54.785244+0000","last_became_peered":"2024-03-12T04:12:54.785244+0000","last_unstale":"2024-03-12T04:12:56.120172+0000","last_undegraded":"2024-03-12T04:12:56.120172+0000","last_fullsized":"2024-03-12T04:12:56.120172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:00:41.961637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120284+0000","last_change":"2024-03-12T04:12:53.780962+0000","last_active":"2024-03-12T04:12:56.120284+0000","last_peered":"2024-03-12T04:12:56.120284+0000","last_clean":"2024-03-12T04:12:56.120284+0000","last_became_active":"2024-03-12T04:12:53.780623+0000","last_became_peered":"2024-03-12T04:12:53.780623+0000","last_unstale":"2024-03-12T04:12:56.120284+0000","last_undegraded":"2024-03-12T04:12:56.120284+0000","last_fullsized":"2024-03-12T04:12:56.120284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776520+0000","last_change":"2024-03-12T04:12:54.775065+0000","last_active":"2024-03-12T04:12:55.776520+0000","last_peered":"2024-03-12T04:12:55.776520+0000","last_clean":"2024-03-12T04:12:55.776520+0000","last_became_active":"2024-03-12T04:12:54.774686+0000","last_became_peered":"2024-03-12T04:12:54.774686+0000","last_unstale":"2024-03-12T04:12:55.776520+0000","last_undegraded":"2024-03-12T04:12:55.776520+0000","last_fullsized":"2024-03-12T04:12:55.776520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:56:25.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393624+0000","last_change":"2024-03-12T04:12:53.777702+0000","last_active":"2024-03-12T04:12:56.393624+0000","last_peered":"2024-03-12T04:12:56.393624+0000","last_clean":"2024-03-12T04:12:56.393624+0000","last_became_active":"2024-03-12T04:12:53.777346+0000","last_became_peered":"2024-03-12T04:12:53.777346+0000","last_unstale":"2024-03-12T04:12:56.393624+0000","last_undegraded":"2024-03-12T04:12:56.393624+0000","last_fullsized":"2024-03-12T04:12:56.393624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:09.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776500+0000","last_change":"2024-03-12T04:12:54.789087+0000","last_active":"2024-03-12T04:12:55.776500+0000","last_peered":"2024-03-12T04:12:55.776500+0000","last_clean":"2024-03-12T04:12:55.776500+0000","last_became_active":"2024-03-12T04:12:54.788735+0000","last_became_peered":"2024-03-12T04:12:54.788735+0000","last_unstale":"2024-03-12T04:12:55.776500+0000","last_undegraded":"2024-03-12T04:12:55.776500+0000","last_fullsized":"2024-03-12T04:12:55.776500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:14.947137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120170+0000","last_change":"2024-03-12T04:12:54.789974+0000","last_active":"2024-03-12T04:12:56.120170+0000","last_peered":"2024-03-12T04:12:56.120170+0000","last_clean":"2024-03-12T04:12:56.120170+0000","last_became_active":"2024-03-12T04:12:54.789578+0000","last_became_peered":"2024-03-12T04:12:54.789578+0000","last_unstale":"2024-03-12T04:12:56.120170+0000","last_undegraded":"2024-03-12T04:12:56.120170+0000","last_fullsized":"2024-03-12T04:12:56.120170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:22:25.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119350+0000","last_change":"2024-03-12T04:12:53.773601+0000","last_active":"2024-03-12T04:12:56.119350+0000","last_peered":"2024-03-12T04:12:56.119350+0000","last_clean":"2024-03-12T04:12:56.119350+0000","last_became_active":"2024-03-12T04:12:53.773264+0000","last_became_peered":"2024-03-12T04:12:53.773264+0000","last_unstale":"2024-03-12T04:12:56.119350+0000","last_undegraded":"2024-03-12T04:12:56.119350+0000","last_fullsized":"2024-03-12T04:12:56.119350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:07:03.580157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120285+0000","last_change":"2024-03-12T04:12:54.782496+0000","last_active":"2024-03-12T04:12:56.120285+0000","last_peered":"2024-03-12T04:12:56.120285+0000","last_clean":"2024-03-12T04:12:56.120285+0000","last_became_active":"2024-03-12T04:12:54.782014+0000","last_became_peered":"2024-03-12T04:12:54.782014+0000","last_unstale":"2024-03-12T04:12:56.120285+0000","last_undegraded":"2024-03-12T04:12:56.120285+0000","last_fullsized":"2024-03-12T04:12:56.120285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:08.580655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.057141+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:12:59.057141+0000","last_peered":"2024-03-12T04:12:59.057141+0000","last_clean":"2024-03-12T04:12:59.057141+0000","last_became_active":"2024-03-12T04:12:53.770855+0000","last_became_peered":"2024-03-12T04:12:53.770855+0000","last_unstale":"2024-03-12T04:12:59.057141+0000","last_undegraded":"2024-03-12T04:12:59.057141+0000","last_fullsized":"2024-03-12T04:12:59.057141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:32:33.969240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393415+0000","last_change":"2024-03-12T04:11:37.642780+0000","last_active":"2024-03-12T04:12:56.393415+0000","last_peered":"2024-03-12T04:12:56.393415+0000","last_clean":"2024-03-12T04:12:56.393415+0000","last_became_active":"2024-03-12T04:11:37.642145+0000","last_became_peered":"2024-03-12T04:11:37.642145+0000","last_unstale":"2024-03-12T04:12:56.393415+0000","last_undegraded":"2024-03-12T04:12:56.393415+0000","last_fullsized":"2024-03-12T04:12:56.393415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_clean_scrub_stamp":"2024-03-12T04:08:09.604393+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:46:36.666483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119402+0000","last_change":"2024-03-12T04:12:54.790464+0000","last_active":"2024-03-12T04:12:56.119402+0000","last_peered":"2024-03-12T04:12:56.119402+0000","last_clean":"2024-03-12T04:12:56.119402+0000","last_became_active":"2024-03-12T04:12:54.788440+0000","last_became_peered":"2024-03-12T04:12:54.788440+0000","last_unstale":"2024-03-12T04:12:56.119402+0000","last_undegraded":"2024-03-12T04:12:56.119402+0000","last_fullsized":"2024-03-12T04:12:56.119402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:54.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393427+0000","last_change":"2024-03-12T04:12:53.773334+0000","last_active":"2024-03-12T04:12:56.393427+0000","last_peered":"2024-03-12T04:12:56.393427+0000","last_clean":"2024-03-12T04:12:56.393427+0000","last_became_active":"2024-03-12T04:12:53.772759+0000","last_became_peered":"2024-03-12T04:12:53.772759+0000","last_unstale":"2024-03-12T04:12:56.393427+0000","last_undegraded":"2024-03-12T04:12:56.393427+0000","last_fullsized":"2024-03-12T04:12:56.393427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:03:17.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776143+0000","last_change":"2024-03-12T04:12:53.777035+0000","last_active":"2024-03-12T04:12:55.776143+0000","last_peered":"2024-03-12T04:12:55.776143+0000","last_clean":"2024-03-12T04:12:55.776143+0000","last_became_active":"2024-03-12T04:12:53.776587+0000","last_became_peered":"2024-03-12T04:12:53.776587+0000","last_unstale":"2024-03-12T04:12:55.776143+0000","last_undegraded":"2024-03-12T04:12:55.776143+0000","last_fullsized":"2024-03-12T04:12:55.776143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:21.390525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120113+0000","last_change":"2024-03-12T04:12:54.790835+0000","last_active":"2024-03-12T04:12:56.120113+0000","last_peered":"2024-03-12T04:12:56.120113+0000","last_clean":"2024-03-12T04:12:56.120113+0000","last_became_active":"2024-03-12T04:12:54.790512+0000","last_became_peered":"2024-03-12T04:12:54.790512+0000","last_unstale":"2024-03-12T04:12:56.120113+0000","last_undegraded":"2024-03-12T04:12:56.120113+0000","last_fullsized":"2024-03-12T04:12:56.120113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:37:58.029450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.059728+0000","last_change":"2024-03-12T04:12:53.774604+0000","last_active":"2024-03-12T04:12:59.059728+0000","last_peered":"2024-03-12T04:12:59.059728+0000","last_clean":"2024-03-12T04:12:59.059728+0000","last_became_active":"2024-03-12T04:12:53.774270+0000","last_became_peered":"2024-03-12T04:12:53.774270+0000","last_unstale":"2024-03-12T04:12:59.059728+0000","last_undegraded":"2024-03-12T04:12:59.059728+0000","last_fullsized":"2024-03-12T04:12:59.059728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:23.635179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120069+0000","last_change":"2024-03-12T04:12:54.786259+0000","last_active":"2024-03-12T04:12:56.120069+0000","last_peered":"2024-03-12T04:12:56.120069+0000","last_clean":"2024-03-12T04:12:56.120069+0000","last_became_active":"2024-03-12T04:12:54.783114+0000","last_became_peered":"2024-03-12T04:12:54.783114+0000","last_unstale":"2024-03-12T04:12:56.120069+0000","last_undegraded":"2024-03-12T04:12:56.120069+0000","last_fullsized":"2024-03-12T04:12:56.120069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:53.036505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.042079+0000","last_change":"2024-03-12T04:12:53.779618+0000","last_active":"2024-03-12T04:12:57.042079+0000","last_peered":"2024-03-12T04:12:57.042079+0000","last_clean":"2024-03-12T04:12:57.042079+0000","last_became_active":"2024-03-12T04:12:53.779264+0000","last_became_peered":"2024-03-12T04:12:53.779264+0000","last_unstale":"2024-03-12T04:12:57.042079+0000","last_undegraded":"2024-03-12T04:12:57.042079+0000","last_fullsized":"2024-03-12T04:12:57.042079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:57.633014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775649+0000","last_change":"2024-03-12T04:12:54.789012+0000","last_active":"2024-03-12T04:12:55.775649+0000","last_peered":"2024-03-12T04:12:55.775649+0000","last_clean":"2024-03-12T04:12:55.775649+0000","last_became_active":"2024-03-12T04:12:54.788441+0000","last_became_peered":"2024-03-12T04:12:54.788441+0000","last_unstale":"2024-03-12T04:12:55.775649+0000","last_undegraded":"2024-03-12T04:12:55.775649+0000","last_fullsized":"2024-03-12T04:12:55.775649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:55.858926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:57.040979+0000","last_change":"2024-03-12T04:12:53.776621+0000","last_active":"2024-03-12T04:12:57.040979+0000","last_peered":"2024-03-12T04:12:57.040979+0000","last_clean":"2024-03-12T04:12:57.040979+0000","last_became_active":"2024-03-12T04:12:53.776313+0000","last_became_peered":"2024-03-12T04:12:53.776313+0000","last_unstale":"2024-03-12T04:12:57.040979+0000","last_undegraded":"2024-03-12T04:12:57.040979+0000","last_fullsized":"2024-03-12T04:12:57.040979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:43.536159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775980+0000","last_change":"2024-03-12T04:12:54.786929+0000","last_active":"2024-03-12T04:12:55.775980+0000","last_peered":"2024-03-12T04:12:55.775980+0000","last_clean":"2024-03-12T04:12:55.775980+0000","last_became_active":"2024-03-12T04:12:54.786580+0000","last_became_peered":"2024-03-12T04:12:54.786580+0000","last_unstale":"2024-03-12T04:12:55.775980+0000","last_undegraded":"2024-03-12T04:12:55.775980+0000","last_fullsized":"2024-03-12T04:12:55.775980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:58.955293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:13:00.034428+0000","last_change":"2024-03-12T04:12:53.778577+0000","last_active":"2024-03-12T04:13:00.034428+0000","last_peered":"2024-03-12T04:13:00.034428+0000","last_clean":"2024-03-12T04:13:00.034428+0000","last_became_active":"2024-03-12T04:12:53.778033+0000","last_became_peered":"2024-03-12T04:12:53.778033+0000","last_unstale":"2024-03-12T04:13:00.034428+0000","last_undegraded":"2024-03-12T04:13:00.034428+0000","last_fullsized":"2024-03-12T04:13:00.034428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:02.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119728+0000","last_change":"2024-03-12T04:12:54.785861+0000","last_active":"2024-03-12T04:12:56.119728+0000","last_peered":"2024-03-12T04:12:56.119728+0000","last_clean":"2024-03-12T04:12:56.119728+0000","last_became_active":"2024-03-12T04:12:54.784682+0000","last_became_peered":"2024-03-12T04:12:54.784682+0000","last_unstale":"2024-03-12T04:12:56.119728+0000","last_undegraded":"2024-03-12T04:12:56.119728+0000","last_fullsized":"2024-03-12T04:12:56.119728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:35:12.288309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:58.036286+0000","last_change":"2024-03-12T04:12:53.772865+0000","last_active":"2024-03-12T04:12:58.036286+0000","last_peered":"2024-03-12T04:12:58.036286+0000","last_clean":"2024-03-12T04:12:58.036286+0000","last_became_active":"2024-03-12T04:12:53.772502+0000","last_became_peered":"2024-03-12T04:12:53.772502+0000","last_unstale":"2024-03-12T04:12:58.036286+0000","last_undegraded":"2024-03-12T04:12:58.036286+0000","last_fullsized":"2024-03-12T04:12:58.036286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:40:24.112197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120550+0000","last_change":"2024-03-12T04:12:54.780722+0000","last_active":"2024-03-12T04:12:56.120550+0000","last_peered":"2024-03-12T04:12:56.120550+0000","last_clean":"2024-03-12T04:12:56.120550+0000","last_became_active":"2024-03-12T04:12:54.780404+0000","last_became_peered":"2024-03-12T04:12:54.780404+0000","last_unstale":"2024-03-12T04:12:56.120550+0000","last_undegraded":"2024-03-12T04:12:56.120550+0000","last_fullsized":"2024-03-12T04:12:56.120550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.068094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.776032+0000","last_change":"2024-03-12T04:12:53.771471+0000","last_active":"2024-03-12T04:12:55.776032+0000","last_peered":"2024-03-12T04:12:55.776032+0000","last_clean":"2024-03-12T04:12:55.776032+0000","last_became_active":"2024-03-12T04:12:53.771147+0000","last_became_peered":"2024-03-12T04:12:53.771147+0000","last_unstale":"2024-03-12T04:12:55.776032+0000","last_undegraded":"2024-03-12T04:12:55.776032+0000","last_fullsized":"2024-03-12T04:12:55.776032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:46.159168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393503+0000","last_change":"2024-03-12T04:12:54.796253+0000","last_active":"2024-03-12T04:12:56.393503+0000","last_peered":"2024-03-12T04:12:56.393503+0000","last_clean":"2024-03-12T04:12:56.393503+0000","last_became_active":"2024-03-12T04:12:54.795656+0000","last_became_peered":"2024-03-12T04:12:54.795656+0000","last_unstale":"2024-03-12T04:12:56.393503+0000","last_undegraded":"2024-03-12T04:12:56.393503+0000","last_fullsized":"2024-03-12T04:12:56.393503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:39:32.788874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:59.059566+0000","last_change":"2024-03-12T04:12:53.780257+0000","last_active":"2024-03-12T04:12:59.059566+0000","last_peered":"2024-03-12T04:12:59.059566+0000","last_clean":"2024-03-12T04:12:59.059566+0000","last_became_active":"2024-03-12T04:12:53.779891+0000","last_became_peered":"2024-03-12T04:12:53.779891+0000","last_unstale":"2024-03-12T04:12:59.059566+0000","last_undegraded":"2024-03-12T04:12:59.059566+0000","last_fullsized":"2024-03-12T04:12:59.059566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:49:30.218372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393809+0000","last_change":"2024-03-12T04:12:54.792760+0000","last_active":"2024-03-12T04:12:56.393809+0000","last_peered":"2024-03-12T04:12:56.393809+0000","last_clean":"2024-03-12T04:12:56.393809+0000","last_became_active":"2024-03-12T04:12:54.792363+0000","last_became_peered":"2024-03-12T04:12:54.792363+0000","last_unstale":"2024-03-12T04:12:56.393809+0000","last_undegraded":"2024-03-12T04:12:56.393809+0000","last_fullsized":"2024-03-12T04:12:56.393809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:30:15.394318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120740+0000","last_change":"2024-03-12T04:12:53.775556+0000","last_active":"2024-03-12T04:12:56.120740+0000","last_peered":"2024-03-12T04:12:56.120740+0000","last_clean":"2024-03-12T04:12:56.120740+0000","last_became_active":"2024-03-12T04:12:53.775182+0000","last_became_peered":"2024-03-12T04:12:53.775182+0000","last_unstale":"2024-03-12T04:12:56.120740+0000","last_undegraded":"2024-03-12T04:12:56.120740+0000","last_fullsized":"2024-03-12T04:12:56.120740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:13:50.076672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393544+0000","last_change":"2024-03-12T04:12:54.795876+0000","last_active":"2024-03-12T04:12:56.393544+0000","last_peered":"2024-03-12T04:12:56.393544+0000","last_clean":"2024-03-12T04:12:56.393544+0000","last_became_active":"2024-03-12T04:12:54.793874+0000","last_became_peered":"2024-03-12T04:12:54.793874+0000","last_unstale":"2024-03-12T04:12:56.393544+0000","last_undegraded":"2024-03-12T04:12:56.393544+0000","last_fullsized":"2024-03-12T04:12:56.393544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:33:31.165974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120059+0000","last_change":"2024-03-12T04:12:53.774879+0000","last_active":"2024-03-12T04:12:56.120059+0000","last_peered":"2024-03-12T04:12:56.120059+0000","last_clean":"2024-03-12T04:12:56.120059+0000","last_became_active":"2024-03-12T04:12:53.774530+0000","last_became_peered":"2024-03-12T04:12:53.774530+0000","last_unstale":"2024-03-12T04:12:56.120059+0000","last_undegraded":"2024-03-12T04:12:56.120059+0000","last_fullsized":"2024-03-12T04:12:56.120059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:25.858165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.393659+0000","last_change":"2024-03-12T04:12:54.790140+0000","last_active":"2024-03-12T04:12:56.393659+0000","last_peered":"2024-03-12T04:12:56.393659+0000","last_clean":"2024-03-12T04:12:56.393659+0000","last_became_active":"2024-03-12T04:12:54.789798+0000","last_became_peered":"2024-03-12T04:12:54.789798+0000","last_unstale":"2024-03-12T04:12:56.393659+0000","last_undegraded":"2024-03-12T04:12:56.393659+0000","last_fullsized":"2024-03-12T04:12:56.393659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:12:38.722638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119825+0000","last_change":"2024-03-12T04:12:54.785029+0000","last_active":"2024-03-12T04:12:56.119825+0000","last_peered":"2024-03-12T04:12:56.119825+0000","last_clean":"2024-03-12T04:12:56.119825+0000","last_became_active":"2024-03-12T04:12:54.784197+0000","last_became_peered":"2024-03-12T04:12:54.784197+0000","last_unstale":"2024-03-12T04:12:56.119825+0000","last_undegraded":"2024-03-12T04:12:56.119825+0000","last_fullsized":"2024-03-12T04:12:56.119825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:22:41.352093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120083+0000","last_change":"2024-03-12T04:12:54.793132+0000","last_active":"2024-03-12T04:12:56.120083+0000","last_peered":"2024-03-12T04:12:56.120083+0000","last_clean":"2024-03-12T04:12:56.120083+0000","last_became_active":"2024-03-12T04:12:54.792838+0000","last_became_peered":"2024-03-12T04:12:54.792838+0000","last_unstale":"2024-03-12T04:12:56.120083+0000","last_undegraded":"2024-03-12T04:12:56.120083+0000","last_fullsized":"2024-03-12T04:12:56.120083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:25:22.641963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119945+0000","last_change":"2024-03-12T04:12:54.794892+0000","last_active":"2024-03-12T04:12:56.119945+0000","last_peered":"2024-03-12T04:12:56.119945+0000","last_clean":"2024-03-12T04:12:56.119945+0000","last_became_active":"2024-03-12T04:12:54.792656+0000","last_became_peered":"2024-03-12T04:12:54.792656+0000","last_unstale":"2024-03-12T04:12:56.119945+0000","last_undegraded":"2024-03-12T04:12:56.119945+0000","last_fullsized":"2024-03-12T04:12:56.119945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:14.820628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:55.775582+0000","last_change":"2024-03-12T04:12:54.785830+0000","last_active":"2024-03-12T04:12:55.775582+0000","last_peered":"2024-03-12T04:12:55.775582+0000","last_clean":"2024-03-12T04:12:55.775582+0000","last_became_active":"2024-03-12T04:12:54.785087+0000","last_became_peered":"2024-03-12T04:12:54.785087+0000","last_unstale":"2024-03-12T04:12:55.775582+0000","last_undegraded":"2024-03-12T04:12:55.775582+0000","last_fullsized":"2024-03-12T04:12:55.775582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:08:09.860311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119972+0000","last_change":"2024-03-12T04:12:54.787491+0000","last_active":"2024-03-12T04:12:56.119972+0000","last_peered":"2024-03-12T04:12:56.119972+0000","last_clean":"2024-03-12T04:12:56.119972+0000","last_became_active":"2024-03-12T04:12:54.787097+0000","last_became_peered":"2024-03-12T04:12:54.787097+0000","last_unstale":"2024-03-12T04:12:56.119972+0000","last_undegraded":"2024-03-12T04:12:56.119972+0000","last_fullsized":"2024-03-12T04:12:56.119972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:24:31.388053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120824+0000","last_change":"2024-03-12T04:12:54.781227+0000","last_active":"2024-03-12T04:12:56.120824+0000","last_peered":"2024-03-12T04:12:56.120824+0000","last_clean":"2024-03-12T04:12:56.120824+0000","last_became_active":"2024-03-12T04:12:54.780801+0000","last_became_peered":"2024-03-12T04:12:54.780801+0000","last_unstale":"2024-03-12T04:12:56.120824+0000","last_undegraded":"2024-03-12T04:12:56.120824+0000","last_fullsized":"2024-03-12T04:12:56.120824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:09:55.297944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120875+0000","last_change":"2024-03-12T04:12:54.796733+0000","last_active":"2024-03-12T04:12:56.120875+0000","last_peered":"2024-03-12T04:12:56.120875+0000","last_clean":"2024-03-12T04:12:56.120875+0000","last_became_active":"2024-03-12T04:12:54.796388+0000","last_became_peered":"2024-03-12T04:12:54.796388+0000","last_unstale":"2024-03-12T04:12:56.120875+0000","last_undegraded":"2024-03-12T04:12:56.120875+0000","last_fullsized":"2024-03-12T04:12:56.120875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:01.580286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120367+0000","last_change":"2024-03-12T04:12:54.787200+0000","last_active":"2024-03-12T04:12:56.120367+0000","last_peered":"2024-03-12T04:12:56.120367+0000","last_clean":"2024-03-12T04:12:56.120367+0000","last_became_active":"2024-03-12T04:12:54.786657+0000","last_became_peered":"2024-03-12T04:12:54.786657+0000","last_unstale":"2024-03-12T04:12:56.120367+0000","last_undegraded":"2024-03-12T04:12:56.120367+0000","last_fullsized":"2024-03-12T04:12:56.120367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:15:35.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119987+0000","last_change":"2024-03-12T04:12:54.785132+0000","last_active":"2024-03-12T04:12:56.119987+0000","last_peered":"2024-03-12T04:12:56.119987+0000","last_clean":"2024-03-12T04:12:56.119987+0000","last_became_active":"2024-03-12T04:12:54.782548+0000","last_became_peered":"2024-03-12T04:12:54.782548+0000","last_unstale":"2024-03-12T04:12:56.119987+0000","last_undegraded":"2024-03-12T04:12:56.119987+0000","last_fullsized":"2024-03-12T04:12:56.119987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:40:49.284778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119530+0000","last_change":"2024-03-12T04:12:54.790227+0000","last_active":"2024-03-12T04:12:56.119530+0000","last_peered":"2024-03-12T04:12:56.119530+0000","last_clean":"2024-03-12T04:12:56.119530+0000","last_became_active":"2024-03-12T04:12:54.789853+0000","last_became_peered":"2024-03-12T04:12:54.789853+0000","last_unstale":"2024-03-12T04:12:56.119530+0000","last_undegraded":"2024-03-12T04:12:56.119530+0000","last_fullsized":"2024-03-12T04:12:56.119530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:38:16.683661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119908+0000","last_change":"2024-03-12T04:12:54.796039+0000","last_active":"2024-03-12T04:12:56.119908+0000","last_peered":"2024-03-12T04:12:56.119908+0000","last_clean":"2024-03-12T04:12:56.119908+0000","last_became_active":"2024-03-12T04:12:54.795568+0000","last_became_peered":"2024-03-12T04:12:54.795568+0000","last_unstale":"2024-03-12T04:12:56.119908+0000","last_undegraded":"2024-03-12T04:12:56.119908+0000","last_fullsized":"2024-03-12T04:12:56.119908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:46.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.119986+0000","last_change":"2024-03-12T04:12:54.785610+0000","last_active":"2024-03-12T04:12:56.119986+0000","last_peered":"2024-03-12T04:12:56.119986+0000","last_clean":"2024-03-12T04:12:56.119986+0000","last_became_active":"2024-03-12T04:12:54.785256+0000","last_became_peered":"2024-03-12T04:12:54.785256+0000","last_unstale":"2024-03-12T04:12:56.119986+0000","last_undegraded":"2024-03-12T04:12:56.119986+0000","last_fullsized":"2024-03-12T04:12:56.119986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:34.152361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-12T04:12:56.120351+0000","last_change":"2024-03-12T04:12:54.787064+0000","last_active":"2024-03-12T04:12:56.120351+0000","last_peered":"2024-03-12T04:12:56.120351+0000","last_clean":"2024-03-12T04:12:56.120351+0000","last_became_active":"2024-03-12T04:12:54.786701+0000","last_became_peered":"2024-03-12T04:12:54.786701+0000","last_unstale":"2024-03-12T04:12:56.120351+0000","last_undegraded":"2024-03-12T04:12:56.120351+0000","last_fullsized":"2024-03-12T04:12:56.120351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:44:43.582171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250964,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7096,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":60010,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318285,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":441099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1042605,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743449,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":450487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005086,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":450273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":1032049,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":441099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":471146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":449759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":453253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":450077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":450335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1042666,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":39945,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T04:13:02.183 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-12T04:13:02.184 INFO:tasks.cephadm:Setting up client nodes... 2024-03-12T04:13:02.184 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-12T04:13:02.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/630596108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:13:02.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/151819899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:02.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:02 smithi119 ceph-mon[36119]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:13:02.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/630596108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:13:02.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/151819899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:02.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:02 smithi136 ceph-mon[35571]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:13:02.462 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:03.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:03 smithi042 ceph-mon[32260]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:13:03.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:03 smithi042 ceph-mon[32260]: pgmap v296: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 7.6 KiB/s wr, 24 op/s 2024-03-12T04:13:03.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:03 smithi119 ceph-mon[36119]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:13:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:03 smithi119 ceph-mon[36119]: pgmap v296: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 7.6 KiB/s wr, 24 op/s 2024-03-12T04:13:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:03 smithi136 ceph-mon[35571]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:13:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:03 smithi136 ceph-mon[35571]: pgmap v296: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 7.6 KiB/s wr, 24 op/s 2024-03-12T04:13:04.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.181+0000 7ff47474c700 1 -- 172.21.15.42:0/1079940127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1079e0 msgr2=0x7ff46c109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:04.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.181+0000 7ff47474c700 1 --2- 172.21.15.42:0/1079940127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1079e0 0x7ff46c109e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff46400b0a0 tx=0x7ff46400b3b0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.181+0000 7ff47474c700 1 -- 172.21.15.42:0/1079940127 shutdown_connections 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.181+0000 7ff47474c700 1 --2- 172.21.15.42:0/1079940127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1079e0 0x7ff46c109e50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.181+0000 7ff47474c700 1 --2- 172.21.15.42:0/1079940127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46c102780 0x7ff46c104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.181+0000 7ff47474c700 1 --2- 172.21.15.42:0/1079940127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff46c1050b0 0x7ff46c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.181+0000 7ff47474c700 1 -- 172.21.15.42:0/1079940127 >> 172.21.15.42:0/1079940127 conn(0x7ff46c0fc190 msgr2=0x7ff46c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 -- 172.21.15.42:0/1079940127 shutdown_connections 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 -- 172.21.15.42:0/1079940127 wait complete. 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 Processor -- start 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 -- start start 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff46c102780 0x7ff46c1020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1050b0 0x7ff46c100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46c1079e0 0x7ff46c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c1013f0 con 0x7ff46c1050b0 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c101560 con 0x7ff46c1079e0 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff47474c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c1a1b40 con 0x7ff46c102780 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff471ce7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1050b0 0x7ff46c100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff471ce7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1050b0 0x7ff46c100730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35856/0 (socket says 172.21.15.42:35856) 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.182+0000 7ff471ce7700 1 -- 172.21.15.42:0/626738009 learned_addr learned my addr 172.21.15.42:0/626738009 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff4724e8700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff46c102780 0x7ff46c1020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff472ce9700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46c1079e0 0x7ff46c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff471ce7700 1 -- 172.21.15.42:0/626738009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff46c102780 msgr2=0x7ff46c1020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff471ce7700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff46c102780 0x7ff46c1020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff471ce7700 1 -- 172.21.15.42:0/626738009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46c1079e0 msgr2=0x7ff46c100c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff471ce7700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46c1079e0 0x7ff46c100c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff471ce7700 1 -- 172.21.15.42:0/626738009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464009d00 con 0x7ff46c1050b0 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff471ce7700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1050b0 0x7ff46c100730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d950 tx=0x7ff45c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff4637fe700 1 -- 172.21.15.42:0/626738009 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c009940 con 0x7ff46c1050b0 2024-03-12T04:13:04.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff4637fe700 1 -- 172.21.15.42:0/626738009 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff45c010460 con 0x7ff46c1050b0 2024-03-12T04:13:04.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c1a1e20 con 0x7ff46c1050b0 2024-03-12T04:13:04.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff4637fe700 1 -- 172.21.15.42:0/626738009 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00f620 con 0x7ff46c1050b0 2024-03-12T04:13:04.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.183+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c1a22e0 con 0x7ff46c1050b0 2024-03-12T04:13:04.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.184+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c0fdd10 con 0x7ff46c1050b0 2024-03-12T04:13:04.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.185+0000 7ff4637fe700 1 -- 172.21.15.42:0/626738009 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff45c0105d0 con 0x7ff46c1050b0 2024-03-12T04:13:04.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.186+0000 7ff4637fe700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4580a1c80 0x7ff4580a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:04.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.186+0000 7ff4724e8700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4580a1c80 0x7ff4580a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:04.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.186+0000 7ff4637fe700 1 -- 172.21.15.42:0/626738009 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff45c0cd160 con 0x7ff46c1050b0 2024-03-12T04:13:04.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.186+0000 7ff4724e8700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4580a1c80 0x7ff4580a4140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff46800ac50 tx=0x7ff46800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:04.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.191+0000 7ff4637fe700 1 -- 172.21.15.42:0/626738009 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff45c0963c0 con 0x7ff46c1050b0 2024-03-12T04:13:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.479+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff46c10d570 con 0x7ff46c1050b0 2024-03-12T04:13:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.481+0000 7ff4637fe700 1 -- 172.21.15.42:0/626738009 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff45c099a80 con 0x7ff46c1050b0 2024-03-12T04:13:04.484 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-03-12T04:13:04.484 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBQ1u9l80+wHBAABG5QaYJ6AhyJknJ7NFSHIw== 2024-03-12T04:13:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.484+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4580a1c80 msgr2=0x7ff4580a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.484+0000 7ff47474c700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4580a1c80 0x7ff4580a4140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff46800ac50 tx=0x7ff46800a380 comp rx=0 tx=0).stop 2024-03-12T04:13:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.484+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1050b0 msgr2=0x7ff46c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.484+0000 7ff47474c700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1050b0 0x7ff46c100730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d950 tx=0x7ff45c00dc60 comp rx=0 tx=0).stop 2024-03-12T04:13:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 shutdown_connections 2024-03-12T04:13:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4580a1c80 0x7ff4580a4140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46c1079e0 0x7ff46c100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff46c102780 0x7ff46c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 --2- 172.21.15.42:0/626738009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46c1050b0 0x7ff46c100730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 >> 172.21.15.42:0/626738009 conn(0x7ff46c0fc190 msgr2=0x7ff46c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 shutdown_connections 2024-03-12T04:13:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:04.485+0000 7ff47474c700 1 -- 172.21.15.42:0/626738009 wait complete. 2024-03-12T04:13:04.807 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/626738009' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T04:13:04.807 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/626738009' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T04:13:04.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/626738009' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T04:13:04.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/626738009' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T04:13:04.946 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:13:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: 2024-03-12T04:13:04.752+0000 7f6f07212700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-12T04:13:04.946 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:13:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: 2024-03-12T04:13:04.752+0000 7f6f07212700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-12T04:13:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/626738009' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T04:13:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/626738009' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T04:13:05.144 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:13:05.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-12T04:13:05.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-12T04:13:05.188 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-12T04:13:05.189 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-12T04:13:05.189 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph mgr dump --format=json 2024-03-12T04:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:05.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:05 smithi042 ceph-mon[32260]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-03-12T04:13:05.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:05 smithi119 ceph-mon[36119]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-03-12T04:13:05.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:05 smithi136 ceph-mon[35571]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-03-12T04:13:07.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:06 smithi136 ceph-mon[35571]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-03-12T04:13:07.225 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:06 smithi042 ceph-mon[32260]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-03-12T04:13:07.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:06 smithi119 ceph-mon[36119]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-03-12T04:13:07.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.471+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1004408557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0102760 msgr2=0x7ffbe01067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:07.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.471+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1004408557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0102760 0x7ffbe01067b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009990 tx=0x7ffbd4009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.471+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1004408557 shutdown_connections 2024-03-12T04:13:07.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.471+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1004408557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbe0106e90 0x7ffbe010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.471+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1004408557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbe0101db0 0x7ffbe0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.471+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1004408557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0102760 0x7ffbe01067b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.471+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1004408557 >> 172.21.15.42:0/1004408557 conn(0x7ffbe00fd660 msgr2=0x7ffbe00ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:07.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.472+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1004408557 shutdown_connections 2024-03-12T04:13:07.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.472+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1004408557 wait complete. 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.472+0000 7ffbdf59e700 1 Processor -- start 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.472+0000 7ffbdf59e700 1 -- start start 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbdf59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbe0101db0 0x7ffbe0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbdf59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbe0102760 0x7ffbe01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbdf59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0106e90 0x7ffbe019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbdf59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe019ce60 con 0x7ffbe0106e90 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbdf59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe019cfd0 con 0x7ffbe0102760 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbdf59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe019d190 con 0x7ffbe0101db0 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0106e90 0x7ffbe019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0106e90 0x7ffbe019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35884/0 (socket says 172.21.15.42:35884) 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 -- 172.21.15.42:0/1608376973 learned_addr learned my addr 172.21.15.42:0/1608376973 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:07.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbddd9b700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbe0102760 0x7ffbe01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbde59c700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbe0101db0 0x7ffbe0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 -- 172.21.15.42:0/1608376973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbe0101db0 msgr2=0x7ffbe0197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbe0101db0 0x7ffbe0197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 -- 172.21.15.42:0/1608376973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbe0102760 msgr2=0x7ffbe01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbe0102760 0x7ffbe01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 -- 172.21.15.42:0/1608376973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40095f0 con 0x7ffbe0106e90 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbded9d700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0106e90 0x7ffbe019c850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800b970 tx=0x7ffbd800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbcf7fe700 1 -- 172.21.15.42:0/1608376973 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800c930 con 0x7ffbe0106e90 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.473+0000 7ffbcf7fe700 1 -- 172.21.15.42:0/1608376973 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffbd8014440 con 0x7ffbe0106e90 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.474+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe019d470 con 0x7ffbe0106e90 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.474+0000 7ffbcf7fe700 1 -- 172.21.15.42:0/1608376973 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80075d0 con 0x7ffbe0106e90 2024-03-12T04:13:07.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.474+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe01a4cd0 con 0x7ffbe0106e90 2024-03-12T04:13:07.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.475+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe010a1b0 con 0x7ffbe0106e90 2024-03-12T04:13:07.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.476+0000 7ffbcf7fe700 1 -- 172.21.15.42:0/1608376973 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ffbd800ca90 con 0x7ffbe0106e90 2024-03-12T04:13:07.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.476+0000 7ffbcf7fe700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc80a1a30 0x7ffbc80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:07.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.476+0000 7ffbde59c700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc80a1a30 0x7ffbc80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:07.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.476+0000 7ffbcf7fe700 1 -- 172.21.15.42:0/1608376973 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffbd8006e40 con 0x7ffbe0106e90 2024-03-12T04:13:07.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.477+0000 7ffbde59c700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc80a1a30 0x7ffbc80a3ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0005ea0 tx=0x7ffbd0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:07.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.481+0000 7ffbcf7fe700 1 -- 172.21.15.42:0/1608376973 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffbd8095de0 con 0x7ffbe0106e90 2024-03-12T04:13:07.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.741+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ffbe0103290 con 0x7ffbe0106e90 2024-03-12T04:13:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.746+0000 7ffbcf7fe700 1 -- 172.21.15.42:0/1608376973 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260663 (secure 0 0 0) 0x7ffbd8019090 con 0x7ffbe0106e90 2024-03-12T04:13:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.748+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc80a1a30 msgr2=0x7ffbc80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.748+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc80a1a30 0x7ffbc80a3ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0005ea0 tx=0x7ffbd0005e30 comp rx=0 tx=0).stop 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.748+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0106e90 msgr2=0x7ffbe019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.748+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0106e90 0x7ffbe019c850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800b970 tx=0x7ffbd800bc80 comp rx=0 tx=0).stop 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 shutdown_connections 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc80a1a30 0x7ffbc80a3ef0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe0106e90 0x7ffbe019c850 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbe0101db0 0x7ffbe0197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 --2- 172.21.15.42:0/1608376973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbe0102760 0x7ffbe01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:07.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 >> 172.21.15.42:0/1608376973 conn(0x7ffbe00fd660 msgr2=0x7ffbe00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:07.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 shutdown_connections 2024-03-12T04:13:07.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:07.749+0000 7ffbdf59e700 1 -- 172.21.15.42:0/1608376973 wait complete. 2024-03-12T04:13:08.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:07 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1608376973' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T04:13:08.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:07 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1608376973' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T04:13:08.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:07 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1608376973' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T04:13:08.504 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":2757555113},{"type":"v1","addr":"172.21.15.42:6801","nonce":2757555113}]},"active_addr":"172.21.15.42:6801/2757555113","active_change":"2024-03-12T04:05:44.816340+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.42:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2954927866}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":4126408073}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1061357309}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3686333836}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3354112666}]}]} 2024-03-12T04:13:08.511 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-12T04:13:08.511 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-12T04:13:08.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T04:13:08.759 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:09.220 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:08 smithi042 ceph-mon[32260]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-03-12T04:13:09.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:08 smithi119 ceph-mon[36119]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-03-12T04:13:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:08 smithi136 ceph-mon[35571]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-03-12T04:13:10.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 -- 172.21.15.42:0/3733743958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 msgr2=0x7f5350100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:10.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 --2- 172.21.15.42:0/3733743958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350100fd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5344009990 tx=0x7f5344009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 -- 172.21.15.42:0/3733743958 shutdown_connections 2024-03-12T04:13:10.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 --2- 172.21.15.42:0/3733743958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5350101510 0x7f535010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 --2- 172.21.15.42:0/3733743958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5350105ab0 0x7f5350105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 --2- 172.21.15.42:0/3733743958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350100fd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 -- 172.21.15.42:0/3733743958 >> 172.21.15.42:0/3733743958 conn(0x7f5350077b80 msgr2=0x7f5350077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:10.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 -- 172.21.15.42:0/3733743958 shutdown_connections 2024-03-12T04:13:10.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.404+0000 7f53574d6700 1 -- 172.21.15.42:0/3733743958 wait complete. 2024-03-12T04:13:10.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 Processor -- start 2024-03-12T04:13:10.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 -- start start 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5350101510 0x7f5350197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5350105ab0 0x7f53501977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53501a6450 con 0x7f5350100b50 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53501a6590 con 0x7f5350105ab0 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f53574d6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53501a6750 con 0x7f5350101510 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f5355272700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f5355272700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35904/0 (socket says 172.21.15.42:35904) 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f5355272700 1 -- 172.21.15.42:0/716198752 learned_addr learned my addr 172.21.15.42:0/716198752 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.405+0000 7f5355a73700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5350105ab0 0x7f53501977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5354a71700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5350101510 0x7f5350197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5355272700 1 -- 172.21.15.42:0/716198752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5350101510 msgr2=0x7f5350197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:10.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5355272700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5350101510 0x7f5350197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5355272700 1 -- 172.21.15.42:0/716198752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5350105ab0 msgr2=0x7f53501977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5355272700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5350105ab0 0x7f53501977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5355272700 1 -- 172.21.15.42:0/716198752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440095f0 con 0x7f5350100b50 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5355a73700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5350105ab0 0x7f53501977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f5355272700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350196d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5340007e20 tx=0x7f534000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f534e7fc700 1 -- 172.21.15.42:0/716198752 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534000ae00 con 0x7f5350100b50 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f534e7fc700 1 -- 172.21.15.42:0/716198752 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f53400128d0 con 0x7f5350100b50 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53501a6a30 con 0x7f5350100b50 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f534e7fc700 1 -- 172.21.15.42:0/716198752 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340011810 con 0x7f5350100b50 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.406+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53501a6f30 con 0x7f5350100b50 2024-03-12T04:13:10.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.407+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5350109090 con 0x7f5350100b50 2024-03-12T04:13:10.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.408+0000 7f534e7fc700 1 -- 172.21.15.42:0/716198752 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f5340012a40 con 0x7f5350100b50 2024-03-12T04:13:10.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.409+0000 7f534e7fc700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f53380a1b50 0x7f53380a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:10.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.409+0000 7f5354a71700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f53380a1b50 0x7f53380a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:10.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.409+0000 7f534e7fc700 1 -- 172.21.15.42:0/716198752 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f53400cd850 con 0x7f5350100b50 2024-03-12T04:13:10.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.410+0000 7f5354a71700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f53380a1b50 0x7f53380a4010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5344006010 tx=0x7f534400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:10.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.413+0000 7f534e7fc700 1 -- 172.21.15.42:0/716198752 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5340096b40 con 0x7f5350100b50 2024-03-12T04:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.614+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5350068930 con 0x7f5350100b50 2024-03-12T04:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.615+0000 7f534e7fc700 1 -- 172.21.15.42:0/716198752 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22219 (secure 0 0 0) 0x7f534001ae90 con 0x7f5350100b50 2024-03-12T04:13:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:13:10.618 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:55.769086+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f53380a1b50 msgr2=0x7f53380a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f53380a1b50 0x7f53380a4010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5344006010 tx=0x7f534400b540 comp rx=0 tx=0).stop 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 msgr2=0x7f5350196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350196d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5340007e20 tx=0x7f534000a340 comp rx=0 tx=0).stop 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 shutdown_connections 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f53380a1b50 0x7f53380a4010 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5350105ab0 0x7f53501977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5350100b50 0x7f5350196d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 --2- 172.21.15.42:0/716198752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5350101510 0x7f5350197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.617+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 >> 172.21.15.42:0/716198752 conn(0x7f5350077b80 msgr2=0x7f53500fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.618+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 shutdown_connections 2024-03-12T04:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:10.618+0000 7f53574d6700 1 -- 172.21.15.42:0/716198752 wait complete. 2024-03-12T04:13:10.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:10 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/716198752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:10 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/716198752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:10 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/716198752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:11.429 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-12T04:13:11.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T04:13:11.687 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:11.929 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:11 smithi042 ceph-mon[32260]: pgmap v300: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-03-12T04:13:11.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:11 smithi119 ceph-mon[36119]: pgmap v300: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-03-12T04:13:11.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:11 smithi136 ceph-mon[35571]: pgmap v300: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-03-12T04:13:13.210 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:12 smithi042 ceph-mon[32260]: pgmap v301: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 19 op/s 2024-03-12T04:13:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:12 smithi119 ceph-mon[36119]: pgmap v301: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 19 op/s 2024-03-12T04:13:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:12 smithi136 ceph-mon[35571]: pgmap v301: 97 pgs: 97 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 19 op/s 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1910734433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8101cc0 msgr2=0x7f0bd8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1910734433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8101cc0 0x7f0bd8102140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0099c0 tx=0x7f0bcc009cd0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1910734433 shutdown_connections 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1910734433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8102680 0x7f0bd810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1910734433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd8107ca0 0x7f0bd8108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1910734433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8101cc0 0x7f0bd8102140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1910734433 >> 172.21.15.42:0/1910734433 conn(0x7f0bd8078da0 msgr2=0x7f0bd80791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1910734433 shutdown_connections 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1910734433 wait complete. 2024-03-12T04:13:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 Processor -- start 2024-03-12T04:13:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.476+0000 7f0bde2b4700 1 -- start start 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bde2b4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd8101cc0 0x7f0bd8073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bde2b4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8102680 0x7f0bd80744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bde2b4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8107ca0 0x7f0bd80749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bde2b4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd81a6470 con 0x7f0bd8107ca0 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bde2b4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd81a65e0 con 0x7f0bd8101cc0 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bde2b4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd81a67a0 con 0x7f0bd8102680 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8107ca0 0x7f0bd80749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8107ca0 0x7f0bd80749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52966/0 (socket says 172.21.15.42:52966) 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 -- 172.21.15.42:0/1163999050 learned_addr learned my addr 172.21.15.42:0/1163999050 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bd7fff700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd8101cc0 0x7f0bd8073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bd77fe700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8102680 0x7f0bd80744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 -- 172.21.15.42:0/1163999050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8102680 msgr2=0x7f0bd80744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8102680 0x7f0bd80744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 -- 172.21.15.42:0/1163999050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd8101cc0 msgr2=0x7f0bd8073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd8101cc0 0x7f0bd8073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 -- 172.21.15.42:0/1163999050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc0095f0 con 0x7f0bd8107ca0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.477+0000 7f0bdc851700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8107ca0 0x7f0bd80749f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000ada0 tx=0x7f0bd000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.478+0000 7f0bd57fa700 1 -- 172.21.15.42:0/1163999050 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000f070 con 0x7f0bd8107ca0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.478+0000 7f0bd57fa700 1 -- 172.21.15.42:0/1163999050 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0bd0007b30 con 0x7f0bd8107ca0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.478+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd81a6a80 con 0x7f0bd8107ca0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.478+0000 7f0bd57fa700 1 -- 172.21.15.42:0/1163999050 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0005280 con 0x7f0bd8107ca0 2024-03-12T04:13:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.478+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd81a6f50 con 0x7f0bd8107ca0 2024-03-12T04:13:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.479+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd810a180 con 0x7f0bd8107ca0 2024-03-12T04:13:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.480+0000 7f0bd57fa700 1 -- 172.21.15.42:0/1163999050 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f0bd000c8d0 con 0x7f0bd8107ca0 2024-03-12T04:13:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.480+0000 7f0bd57fa700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bc00a1bd0 0x7f0bc00a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.480+0000 7f0bd7fff700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bc00a1bd0 0x7f0bc00a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:13.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.480+0000 7f0bd57fa700 1 -- 172.21.15.42:0/1163999050 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0bd00cc8f0 con 0x7f0bd8107ca0 2024-03-12T04:13:13.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.481+0000 7f0bd7fff700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bc00a1bd0 0x7f0bc00a4090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009be0 tx=0x7f0bc8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:13.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.485+0000 7f0bd57fa700 1 -- 172.21.15.42:0/1163999050 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0bd0095be0 con 0x7f0bd8107ca0 2024-03-12T04:13:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.686+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd8111b60 con 0x7f0bd8107ca0 2024-03-12T04:13:13.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.687+0000 7f0bd57fa700 1 -- 172.21.15.42:0/1163999050 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22219 (secure 0 0 0) 0x7f0bd00992a0 con 0x7f0bd8107ca0 2024-03-12T04:13:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:13:13.690 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:12:55.769086+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bc00a1bd0 msgr2=0x7f0bc00a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bc00a1bd0 0x7f0bc00a4090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009be0 tx=0x7f0bc8009260 comp rx=0 tx=0).stop 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8107ca0 msgr2=0x7f0bd80749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8107ca0 0x7f0bd80749f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000ada0 tx=0x7f0bd000c330 comp rx=0 tx=0).stop 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 shutdown_connections 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0bc00a1bd0 0x7f0bc00a4090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0bd8107ca0 0x7f0bd80749f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd8101cc0 0x7f0bd8073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 --2- 172.21.15.42:0/1163999050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8102680 0x7f0bd80744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:13.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 >> 172.21.15.42:0/1163999050 conn(0x7f0bd8078da0 msgr2=0x7f0bd8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:13.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 shutdown_connections 2024-03-12T04:13:13.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:13.689+0000 7f0bde2b4700 1 -- 172.21.15.42:0/1163999050 wait complete. 2024-03-12T04:13:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:13 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1163999050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:14.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-12T04:13:14.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-12T04:13:14.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-12T04:13:14.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-12T04:13:14.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-12T04:13:14.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-12T04:13:14.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-12T04:13:14.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-12T04:13:14.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-12T04:13:14.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-12T04:13:14.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-12T04:13:14.405 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-12T04:13:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:13 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1163999050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:13 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1163999050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:13:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:15.175 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:15.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:14 smithi042 ceph-mon[32260]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-12T04:13:15.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:14 smithi119 ceph-mon[36119]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-12T04:13:15.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:14 smithi136 ceph-mon[35571]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-12T04:13:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:16.628 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:16.629 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:16 smithi042 ceph-mon[32260]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-12T04:13:17.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:16 smithi119 ceph-mon[36119]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-12T04:13:17.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:16 smithi136 ceph-mon[35571]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-12T04:13:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:18 smithi042 ceph-mon[32260]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-12T04:13:19.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:18 smithi119 ceph-mon[36119]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-12T04:13:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:18 smithi136 ceph-mon[35571]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-12T04:13:21.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:20 smithi042 ceph-mon[32260]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-12T04:13:21.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:20 smithi119 ceph-mon[36119]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-12T04:13:21.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:20 smithi136 ceph-mon[35571]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-12T04:13:23.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:22 smithi042 ceph-mon[32260]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-12T04:13:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:22 smithi119 ceph-mon[36119]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-12T04:13:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:22 smithi136 ceph-mon[35571]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-12T04:13:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.710+0000 7f10bccc5700 1 -- 172.21.15.42:0/3647494837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 msgr2=0x7f10b81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.710+0000 7f10bccc5700 1 --2- 172.21.15.42:0/3647494837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 0x7f10b81012d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009990 tx=0x7f10a0009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.710+0000 7f10bccc5700 1 -- 172.21.15.42:0/3647494837 shutdown_connections 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.710+0000 7f10bccc5700 1 --2- 172.21.15.42:0/3647494837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10b8101810 0x7f10b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.710+0000 7f10bccc5700 1 --2- 172.21.15.42:0/3647494837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b8106d60 0x7f10b8107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.710+0000 7f10bccc5700 1 --2- 172.21.15.42:0/3647494837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 0x7f10b81012d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.710+0000 7f10bccc5700 1 -- 172.21.15.42:0/3647494837 >> 172.21.15.42:0/3647494837 conn(0x7f10b80779d0 msgr2=0x7f10b8077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.711+0000 7f10bccc5700 1 -- 172.21.15.42:0/3647494837 shutdown_connections 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.711+0000 7f10bccc5700 1 -- 172.21.15.42:0/3647494837 wait complete. 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.711+0000 7f10bccc5700 1 Processor -- start 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10bccc5700 1 -- start start 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10bccc5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 0x7f10b81970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10bccc5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b8101810 0x7f10b81975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10bccc5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10b8106d60 0x7f10b819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10bccc5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b819bf40 con 0x7f10b8100e50 2024-03-12T04:13:23.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10bccc5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b819c0b0 con 0x7f10b8101810 2024-03-12T04:13:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10bccc5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b819c270 con 0x7f10b8106d60 2024-03-12T04:13:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 0x7f10b81970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 0x7f10b81970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42382/0 (socket says 172.21.15.42:42382) 2024-03-12T04:13:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 -- 172.21.15.42:0/2264027186 learned_addr learned my addr 172.21.15.42:0/2264027186 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b6d9d700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10b8106d60 0x7f10b819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b5d9b700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b8101810 0x7f10b81975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 -- 172.21.15.42:0/2264027186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10b8106d60 msgr2=0x7f10b819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10b8106d60 0x7f10b819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 -- 172.21.15.42:0/2264027186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b8101810 msgr2=0x7f10b81975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b8101810 0x7f10b81975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 -- 172.21.15.42:0/2264027186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00095f0 con 0x7f10b8100e50 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10b659c700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 0x7f10b81970b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c920 tx=0x7f10b000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.712+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0007b00 con 0x7f10b8100e50 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.713+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b819c550 con 0x7f10b8100e50 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.713+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f10b0004ce0 con 0x7f10b8100e50 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.713+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00054e0 con 0x7f10b8100e50 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.713+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b81a3db0 con 0x7f10b8100e50 2024-03-12T04:13:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.713+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f10b8110d90 con 0x7f10b8100e50 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.715+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f10b0004750 con 0x7f10b8100e50 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.715+0000 7f10af7fe700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f10a40a1b50 0x7f10a40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.715+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10b00cd3c0 con 0x7f10b8100e50 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.715+0000 7f10b5d9b700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f10a40a1b50 0x7f10a40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.715+0000 7f10af7fe700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] conn(0x7f10a40ab330 0x7f10a40ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.715+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 --> [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10a40ade00 con 0x7f10a40ab330 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.715+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b00cd860 con 0x7f10b8100e50 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.716+0000 7f10b6d9d700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] conn(0x7f10a40ab330 0x7f10a40ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.716+0000 7f10b5d9b700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f10a40a1b50 0x7f10a40a4010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10a000b5c0 tx=0x7f10a0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.716+0000 7f10b6d9d700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] conn(0x7f10a40ab330 0x7f10a40ad750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.717+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== osd.3 v2:172.21.15.42:6810/3878628201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f10a40ade00 con 0x7f10a40ab330 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.733+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 --> [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f10b8062380 con 0x7f10a40ab330 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10af7fe700 1 -- 172.21.15.42:0/2264027186 <== osd.3 v2:172.21.15.42:6810/3878628201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f10b8062380 con 0x7f10a40ab330 2024-03-12T04:13:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] conn(0x7f10a40ab330 msgr2=0x7f10a40ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] conn(0x7f10a40ab330 0x7f10a40ad750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f10a40a1b50 msgr2=0x7f10a40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f10a40a1b50 0x7f10a40a4010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10a000b5c0 tx=0x7f10a0005fd0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 msgr2=0x7f10b81970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 --2- 172.21.15.42:0/2264027186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10b8100e50 0x7f10b81970b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c920 tx=0x7f10b000cce0 comp rx=0 tx=0).stop 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10b6d9d700 1 -- 172.21.15.42:0/2264027186 reap_dead start 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 shutdown_connections 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 >> 172.21.15.42:0/2264027186 conn(0x7f10b80779d0 msgr2=0x7f10b80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 shutdown_connections 2024-03-12T04:13:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:23.734+0000 7f10bccc5700 1 -- 172.21.15.42:0/2264027186 wait complete. 2024-03-12T04:13:24.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 -- 172.21.15.42:0/3098395770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24101220 msgr2=0x7f3d24106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 --2- 172.21.15.42:0/3098395770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24101220 0x7f3d24106960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009990 tx=0x7f3d18009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 -- 172.21.15.42:0/3098395770 shutdown_connections 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 --2- 172.21.15.42:0/3098395770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d24107040 0x7f3d2410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 --2- 172.21.15.42:0/3098395770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d24100870 0x7f3d24100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 --2- 172.21.15.42:0/3098395770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24101220 0x7f3d24106960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 -- 172.21.15.42:0/3098395770 >> 172.21.15.42:0/3098395770 conn(0x7f3d24075ef0 msgr2=0x7f3d24076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 -- 172.21.15.42:0/3098395770 shutdown_connections 2024-03-12T04:13:24.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.862+0000 7f3d2c123700 1 -- 172.21.15.42:0/3098395770 wait complete. 2024-03-12T04:13:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 Processor -- start 2024-03-12T04:13:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 -- start start 2024-03-12T04:13:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24100870 0x7f3d24198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d24101220 0x7f3d241986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d24107040 0x7f3d2419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419d050 con 0x7f3d24100870 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419d1c0 con 0x7f3d24107040 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2c123700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419d380 con 0x7f3d24101220 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24100870 0x7f3d24198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24100870 0x7f3d24198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42400/0 (socket says 172.21.15.42:42400) 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 -- 172.21.15.42:0/2268079414 learned_addr learned my addr 172.21.15.42:0/2268079414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d296be700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d24101220 0x7f3d241986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 -- 172.21.15.42:0/2268079414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d24101220 msgr2=0x7f3d241986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d24101220 0x7f3d241986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 -- 172.21.15.42:0/2268079414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d24107040 msgr2=0x7f3d2419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d2a6c0700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d24107040 0x7f3d2419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d24107040 0x7f3d2419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d29ebf700 1 -- 172.21.15.42:0/2268079414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180095f0 con 0x7f3d24100870 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.863+0000 7f3d296be700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d24101220 0x7f3d241986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.864+0000 7f3d29ebf700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24100870 0x7f3d24198170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400ef10 tx=0x7f3d1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.864+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1400cad0 con 0x7f3d24100870 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.864+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2419d600 con 0x7f3d24100870 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.864+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d1400f040 con 0x7f3d24100870 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.864+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d140146e0 con 0x7f3d24100870 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.864+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241a4ec0 con 0x7f3d24100870 2024-03-12T04:13:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.864+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d24111ef0 con 0x7f3d24100870 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.866+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3d14007500 con 0x7f3d24100870 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.866+0000 7f3d22ffd700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3d0c0a1b50 0x7f3d0c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.866+0000 7f3d296be700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3d0c0a1b50 0x7f3d0c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.867+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3d140ce3a0 con 0x7f3d24100870 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.867+0000 7f3d22ffd700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] conn(0x7f3d0c0ab330 0x7f3d0c0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.867+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 --> [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d0c0ade00 con 0x7f3d0c0ab330 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.867+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d140ce710 con 0x7f3d24100870 2024-03-12T04:13:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.867+0000 7f3d2a6c0700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] conn(0x7f3d0c0ab330 0x7f3d0c0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.867+0000 7f3d296be700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3d0c0a1b50 0x7f3d0c0a4010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b2e0 tx=0x7f3d18005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.867+0000 7f3d2a6c0700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] conn(0x7f3d0c0ab330 0x7f3d0c0ad750 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.868+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== osd.6 v2:172.21.15.42:6818/515757893 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3d0c0ade00 con 0x7f3d0c0ab330 2024-03-12T04:13:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.884+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 --> [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d24062330 con 0x7f3d0c0ab330 2024-03-12T04:13:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.884+0000 7f3d22ffd700 1 -- 172.21.15.42:0/2268079414 <== osd.6 v2:172.21.15.42:6818/515757893 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3d24062330 con 0x7f3d0c0ab330 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] conn(0x7f3d0c0ab330 msgr2=0x7f3d0c0ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] conn(0x7f3d0c0ab330 0x7f3d0c0ad750 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3d0c0a1b50 msgr2=0x7f3d0c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3d0c0a1b50 0x7f3d0c0a4010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b2e0 tx=0x7f3d18005e80 comp rx=0 tx=0).stop 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24100870 msgr2=0x7f3d24198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 --2- 172.21.15.42:0/2268079414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d24100870 0x7f3d24198170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400ef10 tx=0x7f3d1400c3a0 comp rx=0 tx=0).stop 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2a6c0700 1 -- 172.21.15.42:0/2268079414 reap_dead start 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 shutdown_connections 2024-03-12T04:13:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 >> 172.21.15.42:0/2268079414 conn(0x7f3d24075ef0 msgr2=0x7f3d240fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:24.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 shutdown_connections 2024-03-12T04:13:24.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:24.885+0000 7f3d2c123700 1 -- 172.21.15.42:0/2268079414 wait complete. 2024-03-12T04:13:25.232 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:24 smithi042 ceph-mon[32260]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-12T04:13:25.258 INFO:teuthology.orchestra.run.smithi042.stdout:107374182461 2024-03-12T04:13:25.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-12T04:13:25.259 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:24 smithi136 ceph-mon[35571]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-12T04:13:25.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:24 smithi119 ceph-mon[36119]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-12T04:13:25.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.752+0000 7f829b4b7700 1 -- 172.21.15.42:0/3563562592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 msgr2=0x7f8294102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.752+0000 7f829b4b7700 1 --2- 172.21.15.42:0/3563562592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 0x7f8294102170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8284009990 tx=0x7f8284009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.752+0000 7f829b4b7700 1 -- 172.21.15.42:0/3563562592 shutdown_connections 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.752+0000 7f829b4b7700 1 --2- 172.21.15.42:0/3563562592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82941026b0 0x7f829410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.752+0000 7f829b4b7700 1 --2- 172.21.15.42:0/3563562592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8294107d40 0x7f8294108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.752+0000 7f829b4b7700 1 --2- 172.21.15.42:0/3563562592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 0x7f8294102170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.752+0000 7f829b4b7700 1 -- 172.21.15.42:0/3563562592 >> 172.21.15.42:0/3563562592 conn(0x7f8294078dc0 msgr2=0x7f82940791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 -- 172.21.15.42:0/3563562592 shutdown_connections 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 -- 172.21.15.42:0/3563562592 wait complete. 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 Processor -- start 2024-03-12T04:13:25.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 -- start start 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 0x7f8294198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82941026b0 0x7f8294198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8294107d40 0x7f829419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829419cfd0 con 0x7f8294101cf0 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829419d140 con 0x7f82941026b0 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f829b4b7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829419d300 con 0x7f8294107d40 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.753+0000 7f8299253700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 0x7f8294198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 0x7f8294198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42414/0 (socket says 172.21.15.42:42414) 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 -- 172.21.15.42:0/2006339244 learned_addr learned my addr 172.21.15.42:0/2006339244 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8298a52700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82941026b0 0x7f8294198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299a54700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8294107d40 0x7f829419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 -- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8294107d40 msgr2=0x7f829419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8294107d40 0x7f829419c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 -- 172.21.15.42:0/2006339244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82941026b0 msgr2=0x7f8294198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82941026b0 0x7f8294198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 -- 172.21.15.42:0/2006339244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82840095f0 con 0x7f8294101cf0 2024-03-12T04:13:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f8299253700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 0x7f8294198140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f829000d950 tx=0x7f829000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290009940 con 0x7f8294101cf0 2024-03-12T04:13:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829419d5e0 con 0x7f8294101cf0 2024-03-12T04:13:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8290010460 con 0x7f8294101cf0 2024-03-12T04:13:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000f620 con 0x7f8294101cf0 2024-03-12T04:13:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.754+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82941a4e40 con 0x7f8294101cf0 2024-03-12T04:13:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.755+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8294111e80 con 0x7f8294101cf0 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.756+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f82900105d0 con 0x7f8294101cf0 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f828a7fc700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82800a1b50 0x7f82800a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f8298a52700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82800a1b50 0x7f82800a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f82900cd790 con 0x7f8294101cf0 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f828a7fc700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] conn(0x7f82800ab330 0x7f82800ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 --> [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82800ade00 con 0x7f82800ab330 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82900cdb00 con 0x7f8294101cf0 2024-03-12T04:13:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f8298a52700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82800a1b50 0x7f82800a4010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f828400b5c0 tx=0x7f8284005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:25.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.757+0000 7f8299a54700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] conn(0x7f82800ab330 0x7f82800ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:25.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.758+0000 7f8299a54700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] conn(0x7f82800ab330 0x7f82800ad750 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:25.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.759+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== osd.5 v2:172.21.15.136:6808/2791487312 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f82800ade00 con 0x7f82800ab330 2024-03-12T04:13:25.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.777+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 --> [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8294062380 con 0x7f82800ab330 2024-03-12T04:13:25.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.777+0000 7f828a7fc700 1 -- 172.21.15.42:0/2006339244 <== osd.5 v2:172.21.15.136:6808/2791487312 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8294062380 con 0x7f82800ab330 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] conn(0x7f82800ab330 msgr2=0x7f82800ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] conn(0x7f82800ab330 0x7f82800ad750 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82800a1b50 msgr2=0x7f82800a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82800a1b50 0x7f82800a4010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f828400b5c0 tx=0x7f8284005960 comp rx=0 tx=0).stop 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 msgr2=0x7f8294198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 --2- 172.21.15.42:0/2006339244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8294101cf0 0x7f8294198140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f829000d950 tx=0x7f829000dc60 comp rx=0 tx=0).stop 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f8299a54700 1 -- 172.21.15.42:0/2006339244 reap_dead start 2024-03-12T04:13:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 shutdown_connections 2024-03-12T04:13:25.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 >> 172.21.15.42:0/2006339244 conn(0x7f8294078dc0 msgr2=0x7f8294100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:25.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 shutdown_connections 2024-03-12T04:13:25.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:25.778+0000 7f829b4b7700 1 -- 172.21.15.42:0/2006339244 wait complete. 2024-03-12T04:13:27.248 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:26 smithi119 ceph-mon[36119]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-12T04:13:27.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:26 smithi042 ceph-mon[32260]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-12T04:13:27.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:26 smithi136 ceph-mon[35571]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-12T04:13:27.909 INFO:teuthology.orchestra.run.smithi042.stdout:197568495662 2024-03-12T04:13:27.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-12T04:13:29.229 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:28 smithi042 ceph-mon[32260]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:13:29.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:28 smithi119 ceph-mon[36119]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:13:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:28 smithi136 ceph-mon[35571]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 -- 172.21.15.42:0/2107847966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 msgr2=0x7fc23c10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 --2- 172.21.15.42:0/2107847966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 0x7fc23c10eb80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00b0a0 tx=0x7fc22c00b3b0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 -- 172.21.15.42:0/2107847966 shutdown_connections 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 --2- 172.21.15.42:0/2107847966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 0x7fc23c10eb80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 --2- 172.21.15.42:0/2107847966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c107d40 0x7fc23c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 --2- 172.21.15.42:0/2107847966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc23c06a3b0 0x7fc23c06a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 -- 172.21.15.42:0/2107847966 >> 172.21.15.42:0/2107847966 conn(0x7fc23c0779d0 msgr2=0x7fc23c077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 -- 172.21.15.42:0/2107847966 shutdown_connections 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.203+0000 7fc243a14700 1 -- 172.21.15.42:0/2107847966 wait complete. 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 Processor -- start 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 -- start start 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc23c06a3b0 0x7fc23c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 0x7fc23c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c107d40 0x7fc23c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c19d020 con 0x7fc23c06ad50 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c19d190 con 0x7fc23c06a3b0 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc243a14700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c19d350 con 0x7fc23c107d40 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 0x7fc23c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 0x7fc23c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42438/0 (socket says 172.21.15.42:42438) 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 -- 172.21.15.42:0/425928029 learned_addr learned my addr 172.21.15.42:0/425928029 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc241fb1700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c107d40 0x7fc23c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc2417b0700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc23c06a3b0 0x7fc23c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 -- 172.21.15.42:0/425928029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c107d40 msgr2=0x7fc23c19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c107d40 0x7fc23c19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 -- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc23c06a3b0 msgr2=0x7fc23c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc23c06a3b0 0x7fc23c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc240faf700 1 -- 172.21.15.42:0/425928029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c009d00 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.204+0000 7fc2417b0700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc23c06a3b0 0x7fc23c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.205+0000 7fc240faf700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 0x7fc23c1986d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c9b0 tx=0x7fc23000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.205+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230007b00 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.205+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc230007c60 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.205+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c19d630 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.205+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230018660 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.205+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c1a4e90 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.205+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc23c111e50 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.207+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fc23000f450 con 0x7fc23c06ad50 2024-03-12T04:13:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.207+0000 7fc23a7fc700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2240a1b50 0x7fc2240a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.207+0000 7fc2417b0700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2240a1b50 0x7fc2240a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.207+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc2300cd860 con 0x7fc23c06ad50 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.208+0000 7fc23a7fc700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] conn(0x7fc2240ab330 0x7fc2240ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.208+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 --> [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2240ade00 con 0x7fc2240ab330 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.208+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2300cdd00 con 0x7fc23c06ad50 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.208+0000 7fc2417b0700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2240a1b50 0x7fc2240a4010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc23400baa0 tx=0x7fc234005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.208+0000 7fc241fb1700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] conn(0x7fc2240ab330 0x7fc2240ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.208+0000 7fc241fb1700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] conn(0x7fc2240ab330 0x7fc2240ad750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.209+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== osd.7 v2:172.21.15.119:6816/1789109628 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc2240ade00 con 0x7fc2240ab330 2024-03-12T04:13:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.226+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 --> [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc23c062380 con 0x7fc2240ab330 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc23a7fc700 1 -- 172.21.15.42:0/425928029 <== osd.7 v2:172.21.15.119:6816/1789109628 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc23c062380 con 0x7fc2240ab330 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] conn(0x7fc2240ab330 msgr2=0x7fc2240ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] conn(0x7fc2240ab330 0x7fc2240ad750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2240a1b50 msgr2=0x7fc2240a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2240a1b50 0x7fc2240a4010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc23400baa0 tx=0x7fc234005db0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 msgr2=0x7fc23c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 --2- 172.21.15.42:0/425928029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23c06ad50 0x7fc23c1986d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c9b0 tx=0x7fc23000ccc0 comp rx=0 tx=0).stop 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc241fb1700 1 -- 172.21.15.42:0/425928029 reap_dead start 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 shutdown_connections 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 >> 172.21.15.42:0/425928029 conn(0x7fc23c0779d0 msgr2=0x7fc23c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 shutdown_connections 2024-03-12T04:13:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.227+0000 7fc243a14700 1 -- 172.21.15.42:0/425928029 wait complete. 2024-03-12T04:13:31.064 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:30 smithi042 ceph-mon[32260]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:13:31.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:30 smithi119 ceph-mon[36119]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:13:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:30 smithi136 ceph-mon[35571]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:13:32.735 INFO:teuthology.orchestra.run.smithi042.stdout:223338299434 2024-03-12T04:13:32.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-12T04:13:32.990 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:32 smithi042 ceph-mon[32260]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-12T04:13:33.398 INFO:teuthology.orchestra.run.smithi042.stdout:163208757300 2024-03-12T04:13:33.398 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-12T04:13:33.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:32 smithi119 ceph-mon[36119]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-12T04:13:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:32 smithi136 ceph-mon[35571]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 -- 172.21.15.42:0/1413243053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c101cf0 msgr2=0x7f7c3c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 --2- 172.21.15.42:0/1413243053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c101cf0 0x7f7c3c102170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009990 tx=0x7f7c24009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 -- 172.21.15.42:0/1413243053 shutdown_connections 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 --2- 172.21.15.42:0/1413243053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c3c1026b0 0x7f7c3c10eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 --2- 172.21.15.42:0/1413243053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c3c107dd0 0x7f7c3c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 --2- 172.21.15.42:0/1413243053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c101cf0 0x7f7c3c102170 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 -- 172.21.15.42:0/1413243053 >> 172.21.15.42:0/1413243053 conn(0x7f7c3c078d30 msgr2=0x7f7c3c079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:33.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 -- 172.21.15.42:0/1413243053 shutdown_connections 2024-03-12T04:13:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.593+0000 7f7c4120c700 1 -- 172.21.15.42:0/1413243053 wait complete. 2024-03-12T04:13:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 Processor -- start 2024-03-12T04:13:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 -- start start 2024-03-12T04:13:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c3c101cf0 0x7f7c3c073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c3c1026b0 0x7f7c3c0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c107dd0 0x7f7c3c0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c1a6550 con 0x7f7c3c107dd0 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c1a66c0 con 0x7f7c3c1026b0 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c4120c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c1a6880 con 0x7f7c3c101cf0 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c3b59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c107dd0 0x7f7c3c0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c3b59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c107dd0 0x7f7c3c0749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56718/0 (socket says 172.21.15.42:56718) 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c3b59e700 1 -- 172.21.15.42:0/2293249664 learned_addr learned my addr 172.21.15.42:0/2293249664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c3ad9d700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c3c101cf0 0x7f7c3c073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.594+0000 7f7c3a59c700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c3c1026b0 0x7f7c3c0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c3b59e700 1 -- 172.21.15.42:0/2293249664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c3c101cf0 msgr2=0x7f7c3c073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c3b59e700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c3c101cf0 0x7f7c3c073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c3b59e700 1 -- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c3c1026b0 msgr2=0x7f7c3c0744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c3b59e700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c3c1026b0 0x7f7c3c0744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c3b59e700 1 -- 172.21.15.42:0/2293249664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c240095f0 con 0x7f7c3c107dd0 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c3b59e700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c107dd0 0x7f7c3c0749f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400b550 tx=0x7f7c3400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34011830 con 0x7f7c3c107dd0 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c1a6b60 con 0x7f7c3c107dd0 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c34011e40 con 0x7f7c3c107dd0 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34010db0 con 0x7f7c3c107dd0 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.595+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c1a6fe0 con 0x7f7c3c107dd0 2024-03-12T04:13:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.596+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c3c111e40 con 0x7f7c3c107dd0 2024-03-12T04:13:33.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.597+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f7c34011990 con 0x7f7c3c107dd0 2024-03-12T04:13:33.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.598+0000 7f7c33fff700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7c280a1b50 0x7f7c280a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:33.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.598+0000 7f7c3ad9d700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7c280a1b50 0x7f7c280a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:33.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.598+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7c340cd290 con 0x7f7c3c107dd0 2024-03-12T04:13:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.598+0000 7f7c33fff700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] conn(0x7f7c280ab330 0x7f7c280ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.598+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 --> [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c280ade00 con 0x7f7c280ab330 2024-03-12T04:13:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.598+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c340cd730 con 0x7f7c3c107dd0 2024-03-12T04:13:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.599+0000 7f7c3ad9d700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7c280a1b50 0x7f7c280a4010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009550 tx=0x7f7c2c0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.599+0000 7f7c3a59c700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] conn(0x7f7c280ab330 0x7f7c280ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.599+0000 7f7c3a59c700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] conn(0x7f7c280ab330 0x7f7c280ad750 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.600+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== osd.10 v2:172.21.15.119:6824/3868326698 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7c280ade00 con 0x7f7c280ab330 2024-03-12T04:13:33.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.622+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 --> [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c3c062300 con 0x7f7c280ab330 2024-03-12T04:13:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.622+0000 7f7c33fff700 1 -- 172.21.15.42:0/2293249664 <== osd.10 v2:172.21.15.119:6824/3868326698 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7c3c062300 con 0x7f7c280ab330 2024-03-12T04:13:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] conn(0x7f7c280ab330 msgr2=0x7f7c280ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] conn(0x7f7c280ab330 0x7f7c280ad750 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7c280a1b50 msgr2=0x7f7c280a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7c280a1b50 0x7f7c280a4010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009550 tx=0x7f7c2c0091b0 comp rx=0 tx=0).stop 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c107dd0 msgr2=0x7f7c3c0749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 --2- 172.21.15.42:0/2293249664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3c107dd0 0x7f7c3c0749f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400b550 tx=0x7f7c3400b860 comp rx=0 tx=0).stop 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c3b59e700 1 -- 172.21.15.42:0/2293249664 reap_dead start 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 shutdown_connections 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 >> 172.21.15.42:0/2293249664 conn(0x7f7c3c078d30 msgr2=0x7f7c3c100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 shutdown_connections 2024-03-12T04:13:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:33.623+0000 7f7c4120c700 1 -- 172.21.15.42:0/2293249664 wait complete. 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 -- 172.21.15.42:0/3796872085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 msgr2=0x7f2134102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 --2- 172.21.15.42:0/3796872085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 0x7f2134102170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2124009990 tx=0x7f2124009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 -- 172.21.15.42:0/3796872085 shutdown_connections 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 --2- 172.21.15.42:0/3796872085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21341026b0 0x7f213410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 --2- 172.21.15.42:0/3796872085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2134107dd0 0x7f21341081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 --2- 172.21.15.42:0/3796872085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 0x7f2134102170 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 -- 172.21.15.42:0/3796872085 >> 172.21.15.42:0/3796872085 conn(0x7f2134078d30 msgr2=0x7f2134079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 -- 172.21.15.42:0/3796872085 shutdown_connections 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.034+0000 7f213bba5700 1 -- 172.21.15.42:0/3796872085 wait complete. 2024-03-12T04:13:34.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 Processor -- start 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 -- start start 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 0x7f2134073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21341026b0 0x7f21340744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2134107dd0 0x7f21340749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21341a64b0 con 0x7f2134101cf0 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21341a6620 con 0x7f21341026b0 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213bba5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21341a67e0 con 0x7f2134107dd0 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 0x7f2134073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 0x7f2134073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56730/0 (socket says 172.21.15.42:56730) 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 -- 172.21.15.42:0/1735368755 learned_addr learned my addr 172.21.15.42:0/1735368755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139140700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21341026b0 0x7f21340744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 -- 172.21.15.42:0/1735368755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2134107dd0 msgr2=0x7f21340749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213a142700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2134107dd0 0x7f21340749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2134107dd0 0x7f21340749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 -- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21341026b0 msgr2=0x7f21340744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21341026b0 0x7f21340744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139941700 1 -- 172.21.15.42:0/1735368755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240095f0 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f213a142700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2134107dd0 0x7f21340749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.035+0000 7f2139140700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21341026b0 0x7f21340744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.036+0000 7f2139941700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 0x7f2134073f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f213000d980 tx=0x7f213000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.036+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000bb60 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.036+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2130007640 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.036+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130004e10 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.036+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21341a6ac0 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.036+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21341a6f70 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.037+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2134111dc0 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.038+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f2130004020 con 0x7f2134101cf0 2024-03-12T04:13:34.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.038+0000 7f212affd700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21200a1bd0 0x7f21200a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.038+0000 7f2139140700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21200a1bd0 0x7f21200a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.038+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f21300cd900 con 0x7f2134101cf0 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.039+0000 7f212affd700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] conn(0x7f21200ab3b0 0x7f21200ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.039+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 --> [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21200ade80 con 0x7f21200ab3b0 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.039+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21300cdc70 con 0x7f2134101cf0 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.039+0000 7f2139140700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21200a1bd0 0x7f21200a4090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2124006010 tx=0x7f212400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.039+0000 7f213a142700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] conn(0x7f21200ab3b0 0x7f21200ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.039+0000 7f213a142700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] conn(0x7f21200ab3b0 0x7f21200ad7d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.040+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== osd.1 v2:172.21.15.119:6800/3462838566 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f21200ade80 con 0x7f21200ab3b0 2024-03-12T04:13:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.057+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 --> [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2134062380 con 0x7f21200ab3b0 2024-03-12T04:13:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f212affd700 1 -- 172.21.15.42:0/1735368755 <== osd.1 v2:172.21.15.119:6800/3462838566 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2134062380 con 0x7f21200ab3b0 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] conn(0x7f21200ab3b0 msgr2=0x7f21200ad7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] conn(0x7f21200ab3b0 0x7f21200ad7d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21200a1bd0 msgr2=0x7f21200a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f21200a1bd0 0x7f21200a4090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2124006010 tx=0x7f212400b540 comp rx=0 tx=0).stop 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 msgr2=0x7f2134073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 --2- 172.21.15.42:0/1735368755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2134101cf0 0x7f2134073f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f213000d980 tx=0x7f213000dd40 comp rx=0 tx=0).stop 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213a142700 1 -- 172.21.15.42:0/1735368755 reap_dead start 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 shutdown_connections 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 >> 172.21.15.42:0/1735368755 conn(0x7f2134078d30 msgr2=0x7f2134100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 shutdown_connections 2024-03-12T04:13:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:34.058+0000 7f213bba5700 1 -- 172.21.15.42:0/1735368755 wait complete. 2024-03-12T04:13:34.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 -- 172.21.15.42:0/1237544130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c107030 msgr2=0x7f0b8c10ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 --2- 172.21.15.42:0/1237544130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c107030 0x7f0b8c10ebb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400b0a0 tx=0x7f0b8400b3b0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 -- 172.21.15.42:0/1237544130 shutdown_connections 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 --2- 172.21.15.42:0/1237544130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c107030 0x7f0b8c10ebb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 --2- 172.21.15.42:0/1237544130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b8c100810 0x7f0b8c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 --2- 172.21.15.42:0/1237544130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b8c1011c0 0x7f0b8c106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 -- 172.21.15.42:0/1237544130 >> 172.21.15.42:0/1237544130 conn(0x7f0b8c075eb0 msgr2=0x7f0b8c0762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 -- 172.21.15.42:0/1237544130 shutdown_connections 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.312+0000 7f0b9230e700 1 -- 172.21.15.42:0/1237544130 wait complete. 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 Processor -- start 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 -- start start 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b8c100810 0x7f0b8c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c1011c0 0x7f0b8c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b8c107030 0x7f0b8c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c19d0b0 con 0x7f0b8c1011c0 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c19d220 con 0x7f0b8c107030 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b9230e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c19d3e0 con 0x7f0b8c100810 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.313+0000 7f0b8b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c1011c0 0x7f0b8c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c1011c0 0x7f0b8c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42460/0 (socket says 172.21.15.42:42460) 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 -- 172.21.15.42:0/1696479510 learned_addr learned my addr 172.21.15.42:0/1696479510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8bfff700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b8c100810 0x7f0b8c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b908ab700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b8c107030 0x7f0b8c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 -- 172.21.15.42:0/1696479510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b8c100810 msgr2=0x7f0b8c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b8c100810 0x7f0b8c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 -- 172.21.15.42:0/1696479510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b8c107030 msgr2=0x7f0b8c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b8c107030 0x7f0b8c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 -- 172.21.15.42:0/1696479510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b84009d00 con 0x7f0b8c1011c0 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b908ab700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b8c107030 0x7f0b8c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b8b7fe700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c1011c0 0x7f0b8c198710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d950 tx=0x7f0b8000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80009930 con 0x7f0b8c1011c0 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b80010460 con 0x7f0b8c1011c0 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c19d6c0 con 0x7f0b8c1011c0 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000f620 con 0x7f0b8c1011c0 2024-03-12T04:13:34.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.314+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c1a4f20 con 0x7f0b8c1011c0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.315+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b8c111ef0 con 0x7f0b8c1011c0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.316+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f0b800105d0 con 0x7f0b8c1011c0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b897fa700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b740a1b50 0x7f0b740a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b8bfff700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b740a1b50 0x7f0b740a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0b800cd3b0 con 0x7f0b8c1011c0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b897fa700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] conn(0x7f0b740ab330 0x7f0b740ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 --> [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b740ade00 con 0x7f0b740ab330 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b800cd720 con 0x7f0b8c1011c0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b908ab700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] conn(0x7f0b740ab330 0x7f0b740ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b8bfff700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b740a1b50 0x7f0b740a4010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009c60 tx=0x7f0b7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.317+0000 7f0b908ab700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] conn(0x7f0b740ab330 0x7f0b740ad750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.318+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== osd.9 v2:172.21.15.42:6826/1422185869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0b740ade00 con 0x7f0b740ab330 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.334+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 --> [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b8c062380 con 0x7f0b740ab330 2024-03-12T04:13:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b897fa700 1 -- 172.21.15.42:0/1696479510 <== osd.9 v2:172.21.15.42:6826/1422185869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0b8c062380 con 0x7f0b740ab330 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] conn(0x7f0b740ab330 msgr2=0x7f0b740ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] conn(0x7f0b740ab330 0x7f0b740ad750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b740a1b50 msgr2=0x7f0b740a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b740a1b50 0x7f0b740a4010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009c60 tx=0x7f0b7c009260 comp rx=0 tx=0).stop 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c1011c0 msgr2=0x7f0b8c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 --2- 172.21.15.42:0/1696479510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b8c1011c0 0x7f0b8c198710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d950 tx=0x7f0b8000dc60 comp rx=0 tx=0).stop 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b908ab700 1 -- 172.21.15.42:0/1696479510 reap_dead start 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 shutdown_connections 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 >> 172.21.15.42:0/1696479510 conn(0x7f0b8c075eb0 msgr2=0x7f0b8c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 shutdown_connections 2024-03-12T04:13:34.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:29.335+0000 7f0b9230e700 1 -- 172.21.15.42:0/1696479510 wait complete. 2024-03-12T04:13:35.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:34 smithi042 ceph-mon[32260]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:13:35.388 INFO:teuthology.orchestra.run.smithi042.stdout:296352743456 2024-03-12T04:13:35.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-12T04:13:35.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:34 smithi119 ceph-mon[36119]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:13:35.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:34 smithi136 ceph-mon[35571]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 -- 172.21.15.42:0/1363853020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc1011b0 msgr2=0x7efbfc104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 --2- 172.21.15.42:0/1363853020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc1011b0 0x7efbfc104fa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efbec0099c0 tx=0x7efbec009cd0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 -- 172.21.15.42:0/1363853020 shutdown_connections 2024-03-12T04:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 --2- 172.21.15.42:0/1363853020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbfc1056d0 0x7efbfc10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 --2- 172.21.15.42:0/1363853020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc100800 0x7efbfc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 --2- 172.21.15.42:0/1363853020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc1011b0 0x7efbfc104fa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 -- 172.21.15.42:0/1363853020 >> 172.21.15.42:0/1363853020 conn(0x7efbfc075ea0 msgr2=0x7efbfc0762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 -- 172.21.15.42:0/1363853020 shutdown_connections 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 -- 172.21.15.42:0/1363853020 wait complete. 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 Processor -- start 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.125+0000 7efc03a16700 1 -- start start 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc03a16700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc100800 0x7efbfc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc03a16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc1011b0 0x7efbfc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc03a16700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbfc1056d0 0x7efbfc19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc03a16700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc19cf40 con 0x7efbfc100800 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc03a16700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc19d0b0 con 0x7efbfc1056d0 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc03a16700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc19d270 con 0x7efbfc1011b0 2024-03-12T04:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc100800 0x7efbfc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc100800 0x7efbfc1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56746/0 (socket says 172.21.15.42:56746) 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 -- 172.21.15.42:0/3920752145 learned_addr learned my addr 172.21.15.42:0/3920752145 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc01fb3700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbfc1056d0 0x7efbfc19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc00fb1700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc1011b0 0x7efbfc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 -- 172.21.15.42:0/3920752145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc1011b0 msgr2=0x7efbfc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc1011b0 0x7efbfc1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 -- 172.21.15.42:0/3920752145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbfc1056d0 msgr2=0x7efbfc19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbfc1056d0 0x7efbfc19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 -- 172.21.15.42:0/3920752145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec0095f0 con 0x7efbfc100800 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc00fb1700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc1011b0 0x7efbfc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc017b2700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc100800 0x7efbfc1980b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efbf800d950 tx=0x7efbf800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8009940 con 0x7efbfc100800 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efbf8010460 con 0x7efbfc100800 2024-03-12T04:13:36.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.126+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc19d550 con 0x7efbfc100800 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.127+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf800f620 con 0x7efbfc100800 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.127+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc1a4db0 con 0x7efbfc100800 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.127+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efbfc101a10 con 0x7efbfc100800 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.129+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7efbf80105d0 con 0x7efbfc100800 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.129+0000 7efbf27fc700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efbe80a1b50 0x7efbe80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.129+0000 7efc00fb1700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efbe80a1b50 0x7efbe80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.129+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efbf80cd730 con 0x7efbfc100800 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.129+0000 7efbf27fc700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] conn(0x7efbe80ab330 0x7efbe80ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.129+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 --> [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efbe80ade00 con 0x7efbe80ab330 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.129+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbf80cdbd0 con 0x7efbfc100800 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.130+0000 7efc01fb3700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] conn(0x7efbe80ab330 0x7efbe80ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.130+0000 7efc00fb1700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efbe80a1b50 0x7efbe80a4010 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efbec0058e0 tx=0x7efbec005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:36.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.130+0000 7efc01fb3700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] conn(0x7efbe80ab330 0x7efbe80ad750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:36.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.131+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== osd.0 v2:172.21.15.42:6802/1137485788 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efbe80ade00 con 0x7efbe80ab330 2024-03-12T04:13:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.147+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 --> [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efbfc110e00 con 0x7efbe80ab330 2024-03-12T04:13:36.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.147+0000 7efbf27fc700 1 -- 172.21.15.42:0/3920752145 <== osd.0 v2:172.21.15.42:6802/1137485788 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efbfc110e00 con 0x7efbe80ab330 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.147+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] conn(0x7efbe80ab330 msgr2=0x7efbe80ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.147+0000 7efc03a16700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] conn(0x7efbe80ab330 0x7efbe80ad750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efbe80a1b50 msgr2=0x7efbe80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efbe80a1b50 0x7efbe80a4010 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efbec0058e0 tx=0x7efbec005940 comp rx=0 tx=0).stop 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc100800 msgr2=0x7efbfc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 --2- 172.21.15.42:0/3920752145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbfc100800 0x7efbfc1980b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efbf800d950 tx=0x7efbf800dc60 comp rx=0 tx=0).stop 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc01fb3700 1 -- 172.21.15.42:0/3920752145 reap_dead start 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 shutdown_connections 2024-03-12T04:13:36.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 >> 172.21.15.42:0/3920752145 conn(0x7efbfc075ea0 msgr2=0x7efbfc10dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:36.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 shutdown_connections 2024-03-12T04:13:36.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.148+0000 7efc03a16700 1 -- 172.21.15.42:0/3920752145 wait complete. 2024-03-12T04:13:36.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 -- 172.21.15.42:0/561504454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c101cc0 msgr2=0x7f053c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 --2- 172.21.15.42:0/561504454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c101cc0 0x7f053c102140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0530009990 tx=0x7f0530009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 -- 172.21.15.42:0/561504454 shutdown_connections 2024-03-12T04:13:36.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 --2- 172.21.15.42:0/561504454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c102680 0x7f053c10eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 --2- 172.21.15.42:0/561504454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053c107d70 0x7f053c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 --2- 172.21.15.42:0/561504454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c101cc0 0x7f053c102140 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 -- 172.21.15.42:0/561504454 >> 172.21.15.42:0/561504454 conn(0x7f053c078da0 msgr2=0x7f053c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:36.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 -- 172.21.15.42:0/561504454 shutdown_connections 2024-03-12T04:13:36.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.325+0000 7f0541bb0700 1 -- 172.21.15.42:0/561504454 wait complete. 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 Processor -- start 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 -- start start 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053c101cc0 0x7f053c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c102680 0x7f053c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c107d70 0x7f053c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c19cf10 con 0x7f053c107d70 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c19d080 con 0x7f053c101cc0 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.326+0000 7f0541bb0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c19d240 con 0x7f053c102680 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c107d70 0x7f053c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c107d70 0x7f053c19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56764/0 (socket says 172.21.15.42:56764) 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 -- 172.21.15.42:0/2936481102 learned_addr learned my addr 172.21.15.42:0/2936481102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053b7fe700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053c101cc0 0x7f053c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053affd700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c102680 0x7f053c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 -- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c102680 msgr2=0x7f053c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c102680 0x7f053c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 -- 172.21.15.42:0/2936481102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053c101cc0 msgr2=0x7f053c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053c101cc0 0x7f053c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 -- 172.21.15.42:0/2936481102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300095f0 con 0x7f053c107d70 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f053bfff700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c107d70 0x7f053c19c900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f053400ada0 tx=0x7f053400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400f070 con 0x7f053c107d70 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c19d520 con 0x7f053c107d70 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0534007b30 con 0x7f053c107d70 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340053e0 con 0x7f053c107d70 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.327+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c1a4d80 con 0x7f053c107d70 2024-03-12T04:13:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.328+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f053c111e50 con 0x7f053c107d70 2024-03-12T04:13:36.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.329+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f053400c980 con 0x7f053c107d70 2024-03-12T04:13:36.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.330+0000 7f0538ff9700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f05240a1b50 0x7f05240a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.330+0000 7f053b7fe700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f05240a1b50 0x7f05240a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.330+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05340cd370 con 0x7f053c107d70 2024-03-12T04:13:36.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.330+0000 7f0538ff9700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] conn(0x7f05240ab330 0x7f05240ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:36.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.330+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 --> [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05240ade00 con 0x7f05240ab330 2024-03-12T04:13:36.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.330+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05340cd810 con 0x7f053c107d70 2024-03-12T04:13:36.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.330+0000 7f053b7fe700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f05240a1b50 0x7f05240a4010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f052c00ac50 tx=0x7f052c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:36.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.331+0000 7f053affd700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] conn(0x7f05240ab330 0x7f05240ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:36.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.331+0000 7f053affd700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] conn(0x7f05240ab330 0x7f05240ad750 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:36.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.332+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== osd.2 v2:172.21.15.136:6800/2837276479 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f05240ade00 con 0x7f05240ab330 2024-03-12T04:13:36.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.348+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 --> [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f053c062300 con 0x7f05240ab330 2024-03-12T04:13:36.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0538ff9700 1 -- 172.21.15.42:0/2936481102 <== osd.2 v2:172.21.15.136:6800/2837276479 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f053c062300 con 0x7f05240ab330 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] conn(0x7f05240ab330 msgr2=0x7f05240ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] conn(0x7f05240ab330 0x7f05240ad750 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f05240a1b50 msgr2=0x7f05240a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f05240a1b50 0x7f05240a4010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f052c00ac50 tx=0x7f052c00a380 comp rx=0 tx=0).stop 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c107d70 msgr2=0x7f053c19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 --2- 172.21.15.42:0/2936481102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f053c107d70 0x7f053c19c900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f053400ada0 tx=0x7f053400c330 comp rx=0 tx=0).stop 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f053bfff700 1 -- 172.21.15.42:0/2936481102 reap_dead start 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 shutdown_connections 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.349+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 >> 172.21.15.42:0/2936481102 conn(0x7f053c078da0 msgr2=0x7f053c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.350+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 shutdown_connections 2024-03-12T04:13:36.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:36.350+0000 7f0541bb0700 1 -- 172.21.15.42:0/2936481102 wait complete. 2024-03-12T04:13:37.157 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:36 smithi042 ceph-mon[32260]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-12T04:13:37.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:36 smithi119 ceph-mon[36119]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-12T04:13:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:36 smithi136 ceph-mon[35571]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-12T04:13:37.629 INFO:teuthology.orchestra.run.smithi042.stdout:55834574920 2024-03-12T04:13:37.629 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-12T04:13:38.560 INFO:teuthology.orchestra.run.smithi042.stdout:317827579930 2024-03-12T04:13:38.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-12T04:13:38.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/2605392529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406a410 msgr2=0x7fe1a406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 --2- 172.21.15.42:0/2605392529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406a410 0x7fe1a406a7f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009990 tx=0x7fe1a0009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/2605392529 shutdown_connections 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 --2- 172.21.15.42:0/2605392529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a4106ff0 0x7fe1a410eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 --2- 172.21.15.42:0/2605392529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406a410 0x7fe1a406a7f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 --2- 172.21.15.42:0/2605392529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a406adc0 0x7fe1a41068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/2605392529 >> 172.21.15.42:0/2605392529 conn(0x7fe1a4076bc0 msgr2=0x7fe1a4076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/2605392529 shutdown_connections 2024-03-12T04:13:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.757+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/2605392529 wait complete. 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 Processor -- start 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 -- start start 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a406a410 0x7fe1a4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406adc0 0x7fe1a41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a4106ff0 0x7fe1a419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419cf40 con 0x7fe1a406adc0 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1a9770700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a406a410 0x7fe1a4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1a8f6f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406adc0 0x7fe1a41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419d0b0 con 0x7fe1a406a410 2024-03-12T04:13:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1ab9d4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419d270 con 0x7fe1a4106ff0 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1a9770700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a406a410 0x7fe1a4198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:52126/0 (socket says 172.21.15.42:52126) 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1a9770700 1 -- 172.21.15.42:0/1831871333 learned_addr learned my addr 172.21.15.42:0/1831871333 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.758+0000 7fe1a9f71700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a4106ff0 0x7fe1a419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1a8f6f700 1 -- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a4106ff0 msgr2=0x7fe1a419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1a8f6f700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a4106ff0 0x7fe1a419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1a8f6f700 1 -- 172.21.15.42:0/1831871333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a406a410 msgr2=0x7fe1a4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1a8f6f700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a406a410 0x7fe1a4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1a8f6f700 1 -- 172.21.15.42:0/1831871333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00095f0 con 0x7fe1a406adc0 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1a9770700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a406a410 0x7fe1a4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1a8f6f700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406adc0 0x7fe1a41985a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1a41030f0 tx=0x7fe19400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194004230 con 0x7fe1a406adc0 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a419d550 con 0x7fe1a406adc0 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe194004390 con 0x7fe1a406adc0 2024-03-12T04:13:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194003c10 con 0x7fe1a406adc0 2024-03-12T04:13:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.759+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a4100ba0 con 0x7fe1a406adc0 2024-03-12T04:13:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.760+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1a41033a0 con 0x7fe1a406adc0 2024-03-12T04:13:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.762+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fe194007690 con 0x7fe1a406adc0 2024-03-12T04:13:38.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe19a7fc700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe1900a1b50 0x7fe1900a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:38.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe1a9770700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe1900a1b50 0x7fe1900a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:38.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe1940cd9a0 con 0x7fe1a406adc0 2024-03-12T04:13:38.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe19a7fc700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] conn(0x7fe1900ab330 0x7fe1900ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 --> [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1900ade00 con 0x7fe1900ab330 2024-03-12T04:13:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1940cdd10 con 0x7fe1a406adc0 2024-03-12T04:13:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe1a9770700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe1900a1b50 0x7fe1900a4010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006010 tx=0x7fe1a0003700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.763+0000 7fe1a9f71700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] conn(0x7fe1900ab330 0x7fe1900ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.764+0000 7fe1a9f71700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] conn(0x7fe1900ab330 0x7fe1900ad750 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:38.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.765+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== osd.8 v2:172.21.15.136:6816/509882015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe1900ade00 con 0x7fe1900ab330 2024-03-12T04:13:38.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.782+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 --> [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1a4111ec0 con 0x7fe1900ab330 2024-03-12T04:13:38.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe19a7fc700 1 -- 172.21.15.42:0/1831871333 <== osd.8 v2:172.21.15.136:6816/509882015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe1a4111ec0 con 0x7fe1900ab330 2024-03-12T04:13:38.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] conn(0x7fe1900ab330 msgr2=0x7fe1900ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:38.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] conn(0x7fe1900ab330 0x7fe1900ad750 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:38.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe1900a1b50 msgr2=0x7fe1900a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe1900a1b50 0x7fe1900a4010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006010 tx=0x7fe1a0003700 comp rx=0 tx=0).stop 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406adc0 msgr2=0x7fe1a41985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 --2- 172.21.15.42:0/1831871333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1a406adc0 0x7fe1a41985a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1a41030f0 tx=0x7fe19400cf90 comp rx=0 tx=0).stop 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1a9f71700 1 -- 172.21.15.42:0/1831871333 reap_dead start 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 shutdown_connections 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.783+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 >> 172.21.15.42:0/1831871333 conn(0x7fe1a4076bc0 msgr2=0x7fe1a40fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.784+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 shutdown_connections 2024-03-12T04:13:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:38.784+0000 7fe1ab9d4700 1 -- 172.21.15.42:0/1831871333 wait complete. 2024-03-12T04:13:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:38 smithi042 ceph-mon[32260]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:13:39.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:38 smithi119 ceph-mon[36119]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:13:39.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:38 smithi136 ceph-mon[35571]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 -- 172.21.15.42:0/2829542276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 msgr2=0x7f23441068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 --2- 172.21.15.42:0/2829542276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 0x7f23441068c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2334009990 tx=0x7f2334009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 -- 172.21.15.42:0/2829542276 shutdown_connections 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 --2- 172.21.15.42:0/2829542276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2344106ff0 0x7f234410eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 --2- 172.21.15.42:0/2829542276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f234406a410 0x7f234406a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 --2- 172.21.15.42:0/2829542276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 0x7f23441068c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 -- 172.21.15.42:0/2829542276 >> 172.21.15.42:0/2829542276 conn(0x7f2344076bc0 msgr2=0x7f2344076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 -- 172.21.15.42:0/2829542276 shutdown_connections 2024-03-12T04:13:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.607+0000 7f234c16d700 1 -- 172.21.15.42:0/2829542276 wait complete. 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 Processor -- start 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 -- start start 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f234406a410 0x7f2344198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 0x7f23441986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2344106ff0 0x7f234419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234419d020 con 0x7f234406adc0 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234419d190 con 0x7f234406a410 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f234c16d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234419d350 con 0x7f2344106ff0 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f2349708700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 0x7f23441986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f2349708700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 0x7f23441986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56792/0 (socket says 172.21.15.42:56792) 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f2349708700 1 -- 172.21.15.42:0/1214175254 learned_addr learned my addr 172.21.15.42:0/1214175254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.608+0000 7f2349f09700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f234406a410 0x7f2344198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f234a70a700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2344106ff0 0x7f234419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f2349708700 1 -- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2344106ff0 msgr2=0x7f234419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f2349708700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2344106ff0 0x7f234419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f2349708700 1 -- 172.21.15.42:0/1214175254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f234406a410 msgr2=0x7f2344198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f2349708700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f234406a410 0x7f2344198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f2349708700 1 -- 172.21.15.42:0/1214175254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340095f0 con 0x7f234406adc0 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f2349f09700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f234406a410 0x7f2344198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f234a70a700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2344106ff0 0x7f234419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f2349708700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 0x7f23441986d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2334006010 tx=0x7f2334009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233401d070 con 0x7f234406adc0 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f233400fd50 con 0x7f234406adc0 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23340037c0 con 0x7f234406adc0 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234419d5d0 con 0x7f234406adc0 2024-03-12T04:13:39.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.609+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23441a4e90 con 0x7f234406adc0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.610+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2344111f10 con 0x7f234406adc0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.611+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f233400f8a0 con 0x7f234406adc0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f233affd700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f23300a1bd0 0x7f23300a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f2349f09700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f23300a1bd0 0x7f23300a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f23340ced10 con 0x7f234406adc0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f233affd700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] conn(0x7f23300ab3b0 0x7f23300ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 --> [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23300ade80 con 0x7f23300ab3b0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23340cf1b0 con 0x7f234406adc0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f2349f09700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f23300a1bd0 0x7f23300a4090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2340005950 tx=0x7f2340005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.612+0000 7f234a70a700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] conn(0x7f23300ab3b0 0x7f23300ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:39.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.613+0000 7f234a70a700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] conn(0x7f23300ab3b0 0x7f23300ad7d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:39.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.614+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== osd.11 v2:172.21.15.136:6824/2421468523 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f23300ade80 con 0x7f23300ab3b0 2024-03-12T04:13:39.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.630+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 --> [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2344062300 con 0x7f23300ab3b0 2024-03-12T04:13:39.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.630+0000 7f233affd700 1 -- 172.21.15.42:0/1214175254 <== osd.11 v2:172.21.15.136:6824/2421468523 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2344062300 con 0x7f23300ab3b0 2024-03-12T04:13:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.630+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] conn(0x7f23300ab3b0 msgr2=0x7f23300ad7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.630+0000 7f234c16d700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] conn(0x7f23300ab3b0 0x7f23300ad7d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f23300a1bd0 msgr2=0x7f23300a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f23300a1bd0 0x7f23300a4090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2340005950 tx=0x7f2340005f50 comp rx=0 tx=0).stop 2024-03-12T04:13:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 msgr2=0x7f23441986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 --2- 172.21.15.42:0/1214175254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234406adc0 0x7f23441986d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2334006010 tx=0x7f2334009f70 comp rx=0 tx=0).stop 2024-03-12T04:13:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234a70a700 1 -- 172.21.15.42:0/1214175254 reap_dead start 2024-03-12T04:13:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 shutdown_connections 2024-03-12T04:13:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 >> 172.21.15.42:0/1214175254 conn(0x7f2344076bc0 msgr2=0x7f23440ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 shutdown_connections 2024-03-12T04:13:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:39.631+0000 7f234c16d700 1 -- 172.21.15.42:0/1214175254 wait complete. 2024-03-12T04:13:40.301 INFO:teuthology.orchestra.run.smithi042.stdout:77309411397 2024-03-12T04:13:40.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-12T04:13:40.745 INFO:teuthology.orchestra.run.smithi042.stdout:34359738446 2024-03-12T04:13:40.746 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-12T04:13:40.791 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:40.791 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:40.793 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:41.192 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:41.226 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:40 smithi042 ceph-mon[32260]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:13:41.227 INFO:teuthology.orchestra.run.smithi042.stdout:261993005095 2024-03-12T04:13:41.227 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-12T04:13:41.348 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:41.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:40 smithi119 ceph-mon[36119]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:13:41.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:40 smithi136 ceph-mon[35571]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:13:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:42 smithi042 ceph-mon[32260]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-12T04:13:43.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 -- 172.21.15.42:0/3640289552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3684069920 msgr2=0x7f3684069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 --2- 172.21.15.42:0/3640289552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3684069920 0x7f3684069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f36780099c0 tx=0x7f3678009cd0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 -- 172.21.15.42:0/3640289552 shutdown_connections 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 --2- 172.21.15.42:0/3640289552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368406a2c0 0x7f368410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 --2- 172.21.15.42:0/3640289552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3684107cc0 0x7f36841080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 --2- 172.21.15.42:0/3640289552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3684069920 0x7f3684069d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 -- 172.21.15.42:0/3640289552 >> 172.21.15.42:0/3640289552 conn(0x7f3684077aa0 msgr2=0x7f3684077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 -- 172.21.15.42:0/3640289552 shutdown_connections 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.389+0000 7f368884d700 1 -- 172.21.15.42:0/3640289552 wait complete. 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 Processor -- start 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 -- start start 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3684069920 0x7f36841980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f368406a2c0 0x7f36841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3684107cc0 0x7f368419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368419cf80 con 0x7f368406a2c0 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368419d0f0 con 0x7f3684069920 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368884d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368419d2b0 con 0x7f3684107cc0 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f368406a2c0 0x7f36841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f368406a2c0 0x7f36841985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38108/0 (socket says 172.21.15.42:38108) 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 -- 172.21.15.42:0/1573006029 learned_addr learned my addr 172.21.15.42:0/1573006029 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f368259c700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3684069920 0x7f36841980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3682d9d700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3684107cc0 0x7f368419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 -- 172.21.15.42:0/1573006029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3684107cc0 msgr2=0x7f368419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3684107cc0 0x7f368419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 -- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3684069920 msgr2=0x7f36841980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3684069920 0x7f36841980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.390+0000 7f3681d9b700 1 -- 172.21.15.42:0/1573006029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36780095f0 con 0x7f368406a2c0 2024-03-12T04:13:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.391+0000 7f3681d9b700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f368406a2c0 0x7f36841985e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f36780098e0 tx=0x7f36780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:43.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.391+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367801d070 con 0x7f368406a2c0 2024-03-12T04:13:43.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.391+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f367800fd90 con 0x7f368406a2c0 2024-03-12T04:13:43.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.391+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368419d530 con 0x7f368406a2c0 2024-03-12T04:13:43.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.391+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678003790 con 0x7f368406a2c0 2024-03-12T04:13:43.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.391+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36841a4df0 con 0x7f368406a2c0 2024-03-12T04:13:43.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.392+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3684103370 con 0x7f368406a2c0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.393+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3678022700 con 0x7f368406a2c0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.394+0000 7f36737fe700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f366c0a1bd0 0x7f366c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.394+0000 7f368259c700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f366c0a1bd0 0x7f366c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.394+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36780cf350 con 0x7f368406a2c0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.394+0000 7f36737fe700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] conn(0x7f366c0ab3b0 0x7f366c0ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.394+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 --> [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f366c0ade80 con 0x7f366c0ab3b0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.394+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36780cf6c0 con 0x7f368406a2c0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.395+0000 7f368259c700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f366c0a1bd0 0x7f366c0a4090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3674006fd0 tx=0x7f3674008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.395+0000 7f3682d9d700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] conn(0x7f366c0ab3b0 0x7f366c0ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.395+0000 7f3682d9d700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] conn(0x7f366c0ab3b0 0x7f366c0ad7d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.396+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== osd.4 v2:172.21.15.119:6808/239664950 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f366c0ade80 con 0x7f366c0ab3b0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.420+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 --> [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3684111ec0 con 0x7f366c0ab3b0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.421+0000 7f36737fe700 1 -- 172.21.15.42:0/1573006029 <== osd.4 v2:172.21.15.119:6808/239664950 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3684111ec0 con 0x7f366c0ab3b0 2024-03-12T04:13:43.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] conn(0x7f366c0ab3b0 msgr2=0x7f366c0ad7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] conn(0x7f366c0ab3b0 0x7f366c0ad7d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f366c0a1bd0 msgr2=0x7f366c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f366c0a1bd0 0x7f366c0a4090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3674006fd0 tx=0x7f3674008040 comp rx=0 tx=0).stop 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f368406a2c0 msgr2=0x7f36841985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 --2- 172.21.15.42:0/1573006029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f368406a2c0 0x7f36841985e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f36780098e0 tx=0x7f36780057f0 comp rx=0 tx=0).stop 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f3682d9d700 1 -- 172.21.15.42:0/1573006029 reap_dead start 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 shutdown_connections 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 >> 172.21.15.42:0/1573006029 conn(0x7f3684077aa0 msgr2=0x7f36840ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 shutdown_connections 2024-03-12T04:13:43.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:42.422+0000 7f368884d700 1 -- 172.21.15.42:0/1573006029 wait complete. 2024-03-12T04:13:43.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:42 smithi119 ceph-mon[36119]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-12T04:13:43.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:42 smithi136 ceph-mon[35571]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-12T04:13:45.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-12T04:13:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-12T04:13:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-03-12T04:13:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-03-12T04:13:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-12T04:13:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-12T04:13:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:44 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-03-12T04:13:45.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-12T04:13:45.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-12T04:13:45.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-03-12T04:13:45.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-03-12T04:13:45.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-12T04:13:45.201 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-12T04:13:45.201 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:44 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-03-12T04:13:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-12T04:13:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-12T04:13:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-03-12T04:13:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-03-12T04:13:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-12T04:13:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-12T04:13:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:44 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-03-12T04:13:46.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:45 smithi119 ceph-mon[36119]: osdmap e88: 12 total, 12 up, 12 in 2024-03-12T04:13:46.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-12T04:13:46.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-12T04:13:46.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-12T04:13:46.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-03-12T04:13:46.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-03-12T04:13:46.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-12T04:13:46.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-12T04:13:46.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-03-12T04:13:46.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:45 smithi136 ceph-mon[35571]: osdmap e88: 12 total, 12 up, 12 in 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-03-12T04:13:46.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:45 smithi042 ceph-mon[32260]: osdmap e88: 12 total, 12 up, 12 in 2024-03-12T04:13:46.455 INFO:teuthology.orchestra.run.smithi042.stdout:141733920827 2024-03-12T04:13:46.456 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-12T04:13:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:46 smithi042 ceph-mon[32260]: osdmap e89: 12 total, 12 up, 12 in 2024-03-12T04:13:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:46 smithi042 ceph-mon[32260]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:46 smithi119 ceph-mon[36119]: osdmap e89: 12 total, 12 up, 12 in 2024-03-12T04:13:47.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:46 smithi119 ceph-mon[36119]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:47.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:46 smithi136 ceph-mon[35571]: osdmap e89: 12 total, 12 up, 12 in 2024-03-12T04:13:47.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:46 smithi136 ceph-mon[35571]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:48.717 INFO:teuthology.orchestra.run.smithi042.stdout:352187318294 2024-03-12T04:13:48.718 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-12T04:13:49.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:48 smithi042 ceph-mon[32260]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:49.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:48 smithi119 ceph-mon[36119]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:48 smithi136 ceph-mon[35571]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/3505245088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0106b80 msgr2=0x7f3ee0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/3505245088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0106b80 0x7f3ee0106f60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009990 tx=0x7f3ed0009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/3505245088 shutdown_connections 2024-03-12T04:13:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/3505245088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ee0101530 0x7f3ee010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/3505245088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0106b80 0x7f3ee0106f60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/3505245088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee0100b70 0x7f3ee0100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/3505245088 >> 172.21.15.42:0/3505245088 conn(0x7f3ee0077aa0 msgr2=0x7f3ee0077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/3505245088 shutdown_connections 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.078+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/3505245088 wait complete. 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 Processor -- start 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 -- start start 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0100b70 0x7f3ee0196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ee0101530 0x7f3ee0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee0106b80 0x7f3ee0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01a6420 con 0x7f3ee0100b70 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01a6560 con 0x7f3ee0106b80 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3ee5d3c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01a6720 con 0x7f3ee0101530 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3edf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0100b70 0x7f3ee0196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3edf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0100b70 0x7f3ee0196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38116/0 (socket says 172.21.15.42:38116) 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3edf7fe700 1 -- 172.21.15.42:0/1673188386 learned_addr learned my addr 172.21.15.42:0/1673188386 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.079+0000 7f3edffff700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee0106b80 0x7f3ee0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edeffd700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ee0101530 0x7f3ee0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edf7fe700 1 -- 172.21.15.42:0/1673188386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ee0101530 msgr2=0x7f3ee0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edf7fe700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ee0101530 0x7f3ee0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edf7fe700 1 -- 172.21.15.42:0/1673188386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee0106b80 msgr2=0x7f3ee0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edf7fe700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee0106b80 0x7f3ee0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edf7fe700 1 -- 172.21.15.42:0/1673188386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00095f0 con 0x7f3ee0100b70 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edf7fe700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0100b70 0x7f3ee0196d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0000c00 tx=0x7f3ed00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edcff9700 1 -- 172.21.15.42:0/1673188386 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed001d070 con 0x7f3ee0100b70 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edcff9700 1 -- 172.21.15.42:0/1673188386 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ed000fdc0 con 0x7f3ee0100b70 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee01a69a0 con 0x7f3ee0100b70 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3edcff9700 1 -- 172.21.15.42:0/1673188386 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed00037c0 con 0x7f3ee0100b70 2024-03-12T04:13:50.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.080+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee01a6e30 con 0x7f3ee0100b70 2024-03-12T04:13:50.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.081+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee0109020 con 0x7f3ee0100b70 2024-03-12T04:13:50.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.083+0000 7f3edcff9700 1 -- 172.21.15.42:0/1673188386 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3ed000f910 con 0x7f3ee0100b70 2024-03-12T04:13:50.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.083+0000 7f3edcff9700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3ec80a1b50 0x7f3ec80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.084+0000 7f3edeffd700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3ec80a1b50 0x7f3ec80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.084+0000 7f3edcff9700 1 -- 172.21.15.42:0/1673188386 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3ed00cf100 con 0x7f3ee0100b70 2024-03-12T04:13:50.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.084+0000 7f3edeffd700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3ec80a1b50 0x7f3ec80a4010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4006fd0 tx=0x7f3ed4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:50.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.087+0000 7f3edcff9700 1 -- 172.21.15.42:0/1673188386 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ed0098210 con 0x7f3ee0100b70 2024-03-12T04:13:50.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.290+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3ee0110b20 con 0x7f3ee0100b70 2024-03-12T04:13:50.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.291+0000 7f3edcff9700 1 -- 172.21.15.42:0/1673188386 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ed009b8d0 con 0x7f3ee0100b70 2024-03-12T04:13:50.293 INFO:teuthology.orchestra.run.smithi042.stdout:223338299438 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3ec80a1b50 msgr2=0x7f3ec80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3ec80a1b50 0x7f3ec80a4010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4006fd0 tx=0x7f3ed4008040 comp rx=0 tx=0).stop 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0100b70 msgr2=0x7f3ee0196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0100b70 0x7f3ee0196d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0000c00 tx=0x7f3ed00047a0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 shutdown_connections 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3ec80a1b50 0x7f3ec80a4010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee0106b80 0x7f3ee0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ee0100b70 0x7f3ee0196d10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 --2- 172.21.15.42:0/1673188386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ee0101530 0x7f3ee0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 >> 172.21.15.42:0/1673188386 conn(0x7f3ee0077aa0 msgr2=0x7f3ee00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 shutdown_connections 2024-03-12T04:13:50.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.293+0000 7f3ee5d3c700 1 -- 172.21.15.42:0/1673188386 wait complete. 2024-03-12T04:13:50.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 -- 172.21.15.42:0/3785921543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c106ff0 msgr2=0x7f327c10eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 --2- 172.21.15.42:0/3785921543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c106ff0 0x7f327c10eb70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f327400b0a0 tx=0x7f327400b3b0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 -- 172.21.15.42:0/3785921543 shutdown_connections 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 --2- 172.21.15.42:0/3785921543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c106ff0 0x7f327c10eb70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 --2- 172.21.15.42:0/3785921543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c06a410 0x7f327c06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 --2- 172.21.15.42:0/3785921543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c06adc0 0x7f327c1068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 -- 172.21.15.42:0/3785921543 >> 172.21.15.42:0/3785921543 conn(0x7f327c076bc0 msgr2=0x7f327c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 -- 172.21.15.42:0/3785921543 shutdown_connections 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.305+0000 7f3281aa4700 1 -- 172.21.15.42:0/3785921543 wait complete. 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 Processor -- start 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 -- start start 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c06a410 0x7f327c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c06adc0 0x7f327c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c106ff0 0x7f327c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c19d090 con 0x7f327c06a410 2024-03-12T04:13:50.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c19d200 con 0x7f327c06adc0 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.306+0000 7f3281aa4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c19d3c0 con 0x7f327c106ff0 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c06a410 0x7f327c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c06a410 0x7f327c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38146/0 (socket says 172.21.15.42:38146) 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 -- 172.21.15.42:0/138696737 learned_addr learned my addr 172.21.15.42:0/138696737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327bfff700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c106ff0 0x7f327c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327affd700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c06adc0 0x7f327c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 -- 172.21.15.42:0/138696737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c106ff0 msgr2=0x7f327c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c106ff0 0x7f327c19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 -- 172.21.15.42:0/138696737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c06adc0 msgr2=0x7f327c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c06adc0 0x7f327c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 -- 172.21.15.42:0/138696737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3274009d00 con 0x7f327c06a410 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f327b7fe700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c06a410 0x7f327c1981b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c920 tx=0x7f326c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f3278ff9700 1 -- 172.21.15.42:0/138696737 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c007b00 con 0x7f327c06a410 2024-03-12T04:13:50.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f3278ff9700 1 -- 172.21.15.42:0/138696737 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f326c00ce70 con 0x7f327c06a410 2024-03-12T04:13:50.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c19d6a0 con 0x7f327c06a410 2024-03-12T04:13:50.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f3278ff9700 1 -- 172.21.15.42:0/138696737 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0054e0 con 0x7f327c06a410 2024-03-12T04:13:50.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.307+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c1a4f00 con 0x7f327c06a410 2024-03-12T04:13:50.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.308+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c10a1e0 con 0x7f327c06a410 2024-03-12T04:13:50.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.310+0000 7f3278ff9700 1 -- 172.21.15.42:0/138696737 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f326c004750 con 0x7f327c06a410 2024-03-12T04:13:50.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.311+0000 7f3278ff9700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32680a1b50 0x7f32680a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:50.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.311+0000 7f3278ff9700 1 -- 172.21.15.42:0/138696737 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f326c0cd4d0 con 0x7f327c06a410 2024-03-12T04:13:50.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.311+0000 7f327affd700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32680a1b50 0x7f32680a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:50.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.312+0000 7f327affd700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32680a1b50 0x7f32680a4010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3264005fd0 tx=0x7f3264005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.315+0000 7f3278ff9700 1 -- 172.21.15.42:0/138696737 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f326c0965b0 con 0x7f327c06a410 2024-03-12T04:13:50.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.521+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f327c068930 con 0x7f327c06a410 2024-03-12T04:13:50.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.521+0000 7f3278ff9700 1 -- 172.21.15.42:0/138696737 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f326c01d020 con 0x7f327c06a410 2024-03-12T04:13:50.523 INFO:teuthology.orchestra.run.smithi042.stdout:163208757304 2024-03-12T04:13:50.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32680a1b50 msgr2=0x7f32680a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32680a1b50 0x7f32680a4010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3264005fd0 tx=0x7f3264005db0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c06a410 msgr2=0x7f327c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:50.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c06a410 0x7f327c1981b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c920 tx=0x7f326c00cc30 comp rx=0 tx=0).stop 2024-03-12T04:13:50.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 shutdown_connections 2024-03-12T04:13:50.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32680a1b50 0x7f32680a4010 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c106ff0 0x7f327c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327c06a410 0x7f327c1981b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 --2- 172.21.15.42:0/138696737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c06adc0 0x7f327c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:50.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.523+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 >> 172.21.15.42:0/138696737 conn(0x7f327c076bc0 msgr2=0x7f327c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:50.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.524+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 shutdown_connections 2024-03-12T04:13:50.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:50.524+0000 7f3281aa4700 1 -- 172.21.15.42:0/138696737 wait complete. 2024-03-12T04:13:50.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:50 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1673188386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T04:13:50.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:50 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1673188386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T04:13:50.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:50 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1673188386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T04:13:51.512 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:51 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/138696737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T04:13:51.512 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:51 smithi042 ceph-mon[32260]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:51.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:51 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/138696737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T04:13:51.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:51 smithi119 ceph-mon[36119]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:51.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:51 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/138696737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T04:13:51.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:51 smithi136 ceph-mon[35571]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:51.753 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299438 for osd.7 2024-03-12T04:13:51.754 DEBUG:teuthology.parallel:result is None 2024-03-12T04:13:52.396 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757304 for osd.5 2024-03-12T04:13:52.396 DEBUG:teuthology.parallel:result is None 2024-03-12T04:13:52.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.566+0000 7f3245975700 1 -- 172.21.15.42:0/2926937588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 msgr2=0x7f324010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:52.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.566+0000 7f3245975700 1 --2- 172.21.15.42:0/2926937588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f324010eb40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f323800b0a0 tx=0x7f323800b3b0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 -- 172.21.15.42:0/2926937588 shutdown_connections 2024-03-12T04:13:52.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 --2- 172.21.15.42:0/2926937588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f324010eb40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 --2- 172.21.15.42:0/2926937588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240107d40 0x7f3240108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 --2- 172.21.15.42:0/2926937588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3240101cf0 0x7f3240102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 -- 172.21.15.42:0/2926937588 >> 172.21.15.42:0/2926937588 conn(0x7f3240078dc0 msgr2=0x7f32400791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:52.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 -- 172.21.15.42:0/2926937588 shutdown_connections 2024-03-12T04:13:52.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 -- 172.21.15.42:0/2926937588 wait complete. 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 Processor -- start 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.567+0000 7f3245975700 1 -- start start 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3245975700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3240101cf0 0x7f3240073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3245975700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f32400744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3245975700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240107d40 0x7f32400749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3245975700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32401a6640 con 0x7f32401026b0 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3245975700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32401a67b0 con 0x7f3240107d40 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3245975700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32401a6970 con 0x7f3240101cf0 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f32400744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f32400744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47438/0 (socket says 172.21.15.42:47438) 2024-03-12T04:13:52.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 -- 172.21.15.42:0/2096284050 learned_addr learned my addr 172.21.15.42:0/2096284050 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323f7fe700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240107d40 0x7f32400749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323effd700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3240101cf0 0x7f3240073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 -- 172.21.15.42:0/2096284050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3240101cf0 msgr2=0x7f3240073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3240101cf0 0x7f3240073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 -- 172.21.15.42:0/2096284050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240107d40 msgr2=0x7f32400749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240107d40 0x7f32400749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 -- 172.21.15.42:0/2096284050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238009d00 con 0x7f32401026b0 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323f7fe700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240107d40 0x7f32400749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f323e7fc700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f32400744b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f323400c920 tx=0x7f323400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3244973700 1 -- 172.21.15.42:0/2096284050 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234007b00 con 0x7f32401026b0 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3244973700 1 -- 172.21.15.42:0/2096284050 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3234004ce0 con 0x7f32401026b0 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32401a6c50 con 0x7f32401026b0 2024-03-12T04:13:52.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.568+0000 7f3244973700 1 -- 172.21.15.42:0/2096284050 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340055e0 con 0x7f32401026b0 2024-03-12T04:13:52.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.569+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32401a7150 con 0x7f32401026b0 2024-03-12T04:13:52.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.569+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324010a220 con 0x7f32401026b0 2024-03-12T04:13:52.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.571+0000 7f3244973700 1 -- 172.21.15.42:0/2096284050 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3234007c60 con 0x7f32401026b0 2024-03-12T04:13:52.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.571+0000 7f3244973700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32280a1b50 0x7f32280a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:52.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.571+0000 7f323effd700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32280a1b50 0x7f32280a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:52.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.571+0000 7f3244973700 1 -- 172.21.15.42:0/2096284050 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f32340cd4f0 con 0x7f32401026b0 2024-03-12T04:13:52.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.572+0000 7f323effd700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32280a1b50 0x7f32280a4010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3230005fd0 tx=0x7f3230005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:52.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.575+0000 7f3244973700 1 -- 172.21.15.42:0/2096284050 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32340965d0 con 0x7f32401026b0 2024-03-12T04:13:52.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.785+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f3240111bd0 con 0x7f32401026b0 2024-03-12T04:13:52.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.785+0000 7f3244973700 1 -- 172.21.15.42:0/2096284050 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f323401d090 con 0x7f32401026b0 2024-03-12T04:13:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:296352743460 2024-03-12T04:13:52.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32280a1b50 msgr2=0x7f32280a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:52.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32280a1b50 0x7f32280a4010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3230005fd0 tx=0x7f3230005ee0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 msgr2=0x7f32400744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:52.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f32400744b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f323400c920 tx=0x7f323400cce0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 shutdown_connections 2024-03-12T04:13:52.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f32280a1b50 0x7f32280a4010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240107d40 0x7f32400749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3240101cf0 0x7f3240073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 --2- 172.21.15.42:0/2096284050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32401026b0 0x7f32400744b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:52.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.787+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 >> 172.21.15.42:0/2096284050 conn(0x7f3240078dc0 msgr2=0x7f3240100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:52.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.788+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 shutdown_connections 2024-03-12T04:13:52.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:52.788+0000 7f3245975700 1 -- 172.21.15.42:0/2096284050 wait complete. 2024-03-12T04:13:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:53.029 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:52 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2096284050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-12T04:13:53.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:52 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2096284050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-12T04:13:53.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:52 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2096284050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-12T04:13:53.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743456 got 296352743460 for osd.9 2024-03-12T04:13:53.672 DEBUG:teuthology.parallel:result is None 2024-03-12T04:13:53.929 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:53 smithi042 ceph-mon[32260]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:53.929 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:53 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:13:54.187 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:53 smithi136 ceph-mon[35571]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:54.187 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:53 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:13:54.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:53 smithi119 ceph-mon[36119]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:54.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:53 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:13:54.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.203+0000 7faaf46f1700 1 -- 172.21.15.42:0/2389514070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec10e670 msgr2=0x7faaec110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.203+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2389514070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec10e670 0x7faaec110a60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faae400b0a0 tx=0x7faae400b3b0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 -- 172.21.15.42:0/2389514070 shutdown_connections 2024-03-12T04:13:54.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2389514070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec10e670 0x7faaec110a60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2389514070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec073ea0 0x7faaec074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2389514070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaec0747c0 0x7faaec10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 -- 172.21.15.42:0/2389514070 >> 172.21.15.42:0/2389514070 conn(0x7faaec078da0 msgr2=0x7faaec0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 -- 172.21.15.42:0/2389514070 shutdown_connections 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 -- 172.21.15.42:0/2389514070 wait complete. 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 Processor -- start 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 -- start start 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.204+0000 7faaf46f1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec073ea0 0x7faaec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf46f1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec0747c0 0x7faaec19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf46f1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaec10e670 0x7faaec19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf46f1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec19f3d0 con 0x7faaec073ea0 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf46f1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec19f540 con 0x7faaec0747c0 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf46f1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec19f700 con 0x7faaec10e670 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec073ea0 0x7faaec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec073ea0 0x7faaec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47458/0 (socket says 172.21.15.42:47458) 2024-03-12T04:13:54.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 -- 172.21.15.42:0/2568094015 learned_addr learned my addr 172.21.15.42:0/2568094015 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf1c8c700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec0747c0 0x7faaec19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf2c8e700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaec10e670 0x7faaec19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 -- 172.21.15.42:0/2568094015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaec10e670 msgr2=0x7faaec19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaec10e670 0x7faaec19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 -- 172.21.15.42:0/2568094015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec0747c0 msgr2=0x7faaec19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec0747c0 0x7faaec19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 -- 172.21.15.42:0/2568094015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae4009d00 con 0x7faaec073ea0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf2c8e700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaec10e670 0x7faaec19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf248d700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec073ea0 0x7faaec19a4f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faae800b530 tx=0x7faae800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faae37fe700 1 -- 172.21.15.42:0/2568094015 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8011830 con 0x7faaec073ea0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec19f9e0 con 0x7faaec073ea0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faae37fe700 1 -- 172.21.15.42:0/2568094015 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faae8011e40 con 0x7faaec073ea0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faae37fe700 1 -- 172.21.15.42:0/2568094015 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae800f2e0 con 0x7faaec073ea0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.205+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec1a7240 con 0x7faaec073ea0 2024-03-12T04:13:54.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.206+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec109880 con 0x7faaec073ea0 2024-03-12T04:13:54.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.207+0000 7faae37fe700 1 -- 172.21.15.42:0/2568094015 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7faae8010380 con 0x7faaec073ea0 2024-03-12T04:13:54.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.208+0000 7faae37fe700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7faad80a1b50 0x7faad80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.208+0000 7faaf1c8c700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7faad80a1b50 0x7faad80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.208+0000 7faae37fe700 1 -- 172.21.15.42:0/2568094015 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faae80ccfb0 con 0x7faaec073ea0 2024-03-12T04:13:54.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.208+0000 7faaf1c8c700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7faad80a1b50 0x7faad80a4010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faaec1103a0 tx=0x7faadc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:54.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.212+0000 7faae37fe700 1 -- 172.21.15.42:0/2568094015 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faae8100d20 con 0x7faaec073ea0 2024-03-12T04:13:54.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.416+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faaec19b850 con 0x7faaec073ea0 2024-03-12T04:13:54.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.416+0000 7faae37fe700 1 -- 172.21.15.42:0/2568094015 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faae80961a0 con 0x7faaec073ea0 2024-03-12T04:13:54.419 INFO:teuthology.orchestra.run.smithi042.stdout:197568495668 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7faad80a1b50 msgr2=0x7faad80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7faad80a1b50 0x7faad80a4010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faaec1103a0 tx=0x7faadc009190 comp rx=0 tx=0).stop 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec073ea0 msgr2=0x7faaec19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec073ea0 0x7faaec19a4f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faae800b530 tx=0x7faae800b840 comp rx=0 tx=0).stop 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 shutdown_connections 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7faad80a1b50 0x7faad80a4010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaec10e670 0x7faaec19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec073ea0 0x7faaec19a4f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 --2- 172.21.15.42:0/2568094015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec0747c0 0x7faaec19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.418+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 >> 172.21.15.42:0/2568094015 conn(0x7faaec078da0 msgr2=0x7faaec0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:54.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.419+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 shutdown_connections 2024-03-12T04:13:54.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.419+0000 7faaf46f1700 1 -- 172.21.15.42:0/2568094015 wait complete. 2024-03-12T04:13:54.454 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:54.455 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:54.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 -- 172.21.15.42:0/3478981967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 msgr2=0x7f177c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 --2- 172.21.15.42:0/3478981967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c102330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1778009990 tx=0x7f1778009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 -- 172.21.15.42:0/3478981967 shutdown_connections 2024-03-12T04:13:54.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 --2- 172.21.15.42:0/3478981967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f177c106f80 0x7f177c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 --2- 172.21.15.42:0/3478981967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c102330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 --2- 172.21.15.42:0/3478981967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c102900 0x7f177c1068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 -- 172.21.15.42:0/3478981967 >> 172.21.15.42:0/3478981967 conn(0x7f177c078d00 msgr2=0x7f177c079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:54.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 -- 172.21.15.42:0/3478981967 shutdown_connections 2024-03-12T04:13:54.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.455+0000 7f1783729700 1 -- 172.21.15.42:0/3478981967 wait complete. 2024-03-12T04:13:54.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.457+0000 7f1783729700 1 Processor -- start 2024-03-12T04:13:54.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.457+0000 7f1783729700 1 -- start start 2024-03-12T04:13:54.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.457+0000 7f1783729700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f17814c5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f17814c5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47482/0 (socket says 172.21.15.42:47482) 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f17814c5700 1 -- 172.21.15.42:0/3236839718 learned_addr learned my addr 172.21.15.42:0/3236839718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f177c102900 0x7f177c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c106f80 0x7f177c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c19ce00 con 0x7f177c101f50 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c19cf70 con 0x7f177c106f80 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c19d130 con 0x7f177c102900 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f1781cc6700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c106f80 0x7f177c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.458+0000 7f1780cc4700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f177c102900 0x7f177c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f1781cc6700 1 -- 172.21.15.42:0/3236839718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f177c102900 msgr2=0x7f177c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f1781cc6700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f177c102900 0x7f177c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f1781cc6700 1 -- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 msgr2=0x7f177c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f1781cc6700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f1781cc6700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17780095f0 con 0x7f177c106f80 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f17814c5700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f1781cc6700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c106f80 0x7f177c19c7f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f177400b330 tx=0x7f177400b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f17727fc700 1 -- 172.21.15.42:0/3236839718 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774015070 con 0x7f177c106f80 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f17727fc700 1 -- 172.21.15.42:0/3236839718 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f177400bdd0 con 0x7f177c106f80 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f17727fc700 1 -- 172.21.15.42:0/3236839718 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774007c10 con 0x7f177c106f80 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.459+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177c19d3b0 con 0x7f177c106f80 2024-03-12T04:13:54.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.460+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177c073ea0 con 0x7f177c106f80 2024-03-12T04:13:54.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.461+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177c068930 con 0x7f177c106f80 2024-03-12T04:13:54.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.462+0000 7f17727fc700 1 -- 172.21.15.42:0/3236839718 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f1774004ad0 con 0x7f177c106f80 2024-03-12T04:13:54.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.462+0000 7f17727fc700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17680a1bd0 0x7f17680a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:54.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.462+0000 7f17727fc700 1 -- 172.21.15.42:0/3236839718 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f17740d0cf0 con 0x7f177c106f80 2024-03-12T04:13:54.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.462+0000 7f17814c5700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17680a1bd0 0x7f17680a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:54.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.463+0000 7f17814c5700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17680a1bd0 0x7f17680a4090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1778006010 tx=0x7f177800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:54.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.467+0000 7f17727fc700 1 -- 172.21.15.42:0/3236839718 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1774099d70 con 0x7f177c106f80 2024-03-12T04:13:54.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.674+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f177c111b70 con 0x7f177c106f80 2024-03-12T04:13:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.674+0000 7f17727fc700 1 -- 172.21.15.42:0/3236839718 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f177409d430 con 0x7f177c106f80 2024-03-12T04:13:54.677 INFO:teuthology.orchestra.run.smithi042.stdout:55834574924 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17680a1bd0 msgr2=0x7f17680a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17680a1bd0 0x7f17680a4090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1778006010 tx=0x7f177800b260 comp rx=0 tx=0).stop 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c106f80 msgr2=0x7f177c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c106f80 0x7f177c19c7f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f177400b330 tx=0x7f177400b640 comp rx=0 tx=0).stop 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 shutdown_connections 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f17680a1bd0 0x7f17680a4090 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c106f80 0x7f177c19c7f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f177c101f50 0x7f177c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 --2- 172.21.15.42:0/3236839718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f177c102900 0x7f177c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 >> 172.21.15.42:0/3236839718 conn(0x7f177c078d00 msgr2=0x7f177c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 shutdown_connections 2024-03-12T04:13:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:54.676+0000 7f1783729700 1 -- 172.21.15.42:0/3236839718 wait complete. 2024-03-12T04:13:55.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:54 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2568094015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T04:13:55.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:54 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3236839718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T04:13:55.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:54 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2568094015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T04:13:55.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:54 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3236839718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T04:13:55.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:54 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2568094015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T04:13:55.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:54 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3236839718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T04:13:55.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495668 for osd.6 2024-03-12T04:13:55.355 DEBUG:teuthology.parallel:result is None 2024-03-12T04:13:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 -- 172.21.15.42:0/4183236152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470100b70 msgr2=0x7fa470100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 --2- 172.21.15.42:0/4183236152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470100b70 0x7fa470100ff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa460009990 tx=0x7fa460009ca0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 -- 172.21.15.42:0/4183236152 shutdown_connections 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 --2- 172.21.15.42:0/4183236152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa470101530 0x7fa47010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 --2- 172.21.15.42:0/4183236152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa470106b80 0x7fa470106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 --2- 172.21.15.42:0/4183236152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470100b70 0x7fa470100ff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 -- 172.21.15.42:0/4183236152 >> 172.21.15.42:0/4183236152 conn(0x7fa470077aa0 msgr2=0x7fa470077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 -- 172.21.15.42:0/4183236152 shutdown_connections 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 -- 172.21.15.42:0/4183236152 wait complete. 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 Processor -- start 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.704+0000 7fa47739d700 1 -- start start 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47739d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa470100b70 0x7fa470197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47739d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa470101530 0x7fa470198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47739d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470106b80 0x7fa47019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47739d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47019ced0 con 0x7fa470106b80 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47739d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47019d040 con 0x7fa470100b70 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47739d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47019d200 con 0x7fa470101530 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47593a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470106b80 0x7fa47019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa475139700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa470100b70 0x7fa470197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47593a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470106b80 0x7fa47019c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47500/0 (socket says 172.21.15.42:47500) 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa474938700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa470101530 0x7fa470198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47593a700 1 -- 172.21.15.42:0/2474426549 learned_addr learned my addr 172.21.15.42:0/2474426549 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa475139700 1 -- 172.21.15.42:0/2474426549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa470101530 msgr2=0x7fa470198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa475139700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa470101530 0x7fa470198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa475139700 1 -- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470106b80 msgr2=0x7fa47019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa475139700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470106b80 0x7fa47019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa475139700 1 -- 172.21.15.42:0/2474426549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600095f0 con 0x7fa470100b70 2024-03-12T04:13:55.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa475139700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa470100b70 0x7fa470197ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e9f0 tx=0x7fa46c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa4667fc700 1 -- 172.21.15.42:0/2474426549 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00c9c0 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa4667fc700 1 -- 172.21.15.42:0/2474426549 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa46c010430 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47019d4e0 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.705+0000 7fa4667fc700 1 -- 172.21.15.42:0/2474426549 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c018680 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.706+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4701028b0 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.706+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa470068930 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.708+0000 7fa4667fc700 1 -- 172.21.15.42:0/2474426549 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fa46c0105a0 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.708+0000 7fa4667fc700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa45c0a1bd0 0x7fa45c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.708+0000 7fa4667fc700 1 -- 172.21.15.42:0/2474426549 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa46c014070 con 0x7fa470100b70 2024-03-12T04:13:55.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.708+0000 7fa474938700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa45c0a1bd0 0x7fa45c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:13:55.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.709+0000 7fa474938700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa45c0a1bd0 0x7fa45c0a4090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa4600098e0 tx=0x7fa460005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:13:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.712+0000 7fa4667fc700 1 -- 172.21.15.42:0/2474426549 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa46c0966f0 con 0x7fa470100b70 2024-03-12T04:13:55.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.921+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa470110a30 con 0x7fa470100b70 2024-03-12T04:13:55.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.922+0000 7fa4667fc700 1 -- 172.21.15.42:0/2474426549 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa46c099db0 con 0x7fa470100b70 2024-03-12T04:13:55.925 INFO:teuthology.orchestra.run.smithi042.stdout:77309411401 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa45c0a1bd0 msgr2=0x7fa45c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa45c0a1bd0 0x7fa45c0a4090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa4600098e0 tx=0x7fa460005960 comp rx=0 tx=0).stop 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa470100b70 msgr2=0x7fa470197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa470100b70 0x7fa470197ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e9f0 tx=0x7fa46c00ed00 comp rx=0 tx=0).stop 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 shutdown_connections 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa45c0a1bd0 0x7fa45c0a4090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa470106b80 0x7fa47019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa470100b70 0x7fa470197ff0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 --2- 172.21.15.42:0/2474426549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa470101530 0x7fa470198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 >> 172.21.15.42:0/2474426549 conn(0x7fa470077aa0 msgr2=0x7fa4700ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 shutdown_connections 2024-03-12T04:13:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:13:55.924+0000 7fa47739d700 1 -- 172.21.15.42:0/2474426549 wait complete. 2024-03-12T04:13:56.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:55 smithi119 ceph-mon[36119]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:55 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:55 smithi136 ceph-mon[35571]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:55 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:55 smithi042 ceph-mon[32260]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-03-12T04:13:56.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:55 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:13:56.647 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411401 for osd.2 2024-03-12T04:13:56.648 DEBUG:teuthology.parallel:result is None 2024-03-12T04:13:57.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:56 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2474426549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T04:13:57.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:56 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2474426549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T04:13:57.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:56 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2474426549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T04:13:58.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:57 smithi119 ceph-mon[36119]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-03-12T04:13:58.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:57 smithi136 ceph-mon[35571]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-03-12T04:13:58.221 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:57 smithi042 ceph-mon[32260]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-03-12T04:13:58.858 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574924 for osd.1 2024-03-12T04:13:58.858 DEBUG:teuthology.parallel:result is None 2024-03-12T04:13:59.269 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:59.269 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:13:59.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:13:58 smithi042 ceph-mon[32260]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:13:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:13:58 smithi119 ceph-mon[36119]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:13:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:13:58 smithi136 ceph-mon[35571]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:14:01.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:00 smithi042 ceph-mon[32260]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:00 smithi119 ceph-mon[36119]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:00 smithi136 ceph-mon[35571]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:03.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:02 smithi042 ceph-mon[32260]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:03.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:02 smithi119 ceph-mon[36119]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:02 smithi136 ceph-mon[35571]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:03.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 -- 172.21.15.42:0/2928492309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf4105ef0 msgr2=0x7f9cf410dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:03.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 --2- 172.21.15.42:0/2928492309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf4105ef0 0x7f9cf410dac0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b0a0 tx=0x7f9cec00b3b0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 -- 172.21.15.42:0/2928492309 shutdown_connections 2024-03-12T04:14:03.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 --2- 172.21.15.42:0/2928492309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf4105ef0 0x7f9cf410dac0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 --2- 172.21.15.42:0/2928492309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf41015d0 0x7f9cf41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 --2- 172.21.15.42:0/2928492309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf4101f80 0x7f9cf4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 -- 172.21.15.42:0/2928492309 >> 172.21.15.42:0/2928492309 conn(0x7f9cf4077170 msgr2=0x7f9cf40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 -- 172.21.15.42:0/2928492309 shutdown_connections 2024-03-12T04:14:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.569+0000 7f9cfa462700 1 -- 172.21.15.42:0/2928492309 wait complete. 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 Processor -- start 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 -- start start 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf41015d0 0x7f9cf41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf4101f80 0x7f9cf4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4105ef0 0x7f9cf419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf419d1b0 con 0x7f9cf41015d0 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf419d320 con 0x7f9cf4105ef0 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cfa462700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf419d4e0 con 0x7f9cf4101f80 2024-03-12T04:14:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf41015d0 0x7f9cf41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf41015d0 0x7f9cf41982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58554/0 (socket says 172.21.15.42:58554) 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.570+0000 7f9cf3fff700 1 -- 172.21.15.42:0/759225602 learned_addr learned my addr 172.21.15.42:0/759225602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf89ff700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4105ef0 0x7f9cf419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf37fe700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf4101f80 0x7f9cf4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf3fff700 1 -- 172.21.15.42:0/759225602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf4101f80 msgr2=0x7f9cf4198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf3fff700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf4101f80 0x7f9cf4198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf3fff700 1 -- 172.21.15.42:0/759225602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4105ef0 msgr2=0x7f9cf419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf3fff700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4105ef0 0x7f9cf419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf3fff700 1 -- 172.21.15.42:0/759225602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec009d00 con 0x7f9cf41015d0 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf37fe700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf4101f80 0x7f9cf4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf3fff700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf41015d0 0x7f9cf41982d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400d950 tx=0x7f9ce400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf17fa700 1 -- 172.21.15.42:0/759225602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4009940 con 0x7f9cf41015d0 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf17fa700 1 -- 172.21.15.42:0/759225602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ce4010460 con 0x7f9cf41015d0 2024-03-12T04:14:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf419d7c0 con 0x7f9cf41015d0 2024-03-12T04:14:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cf17fa700 1 -- 172.21.15.42:0/759225602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400f620 con 0x7f9cf41015d0 2024-03-12T04:14:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.571+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf40fc750 con 0x7f9cf41015d0 2024-03-12T04:14:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.572+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf4102380 con 0x7f9cf41015d0 2024-03-12T04:14:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.574+0000 7f9cf17fa700 1 -- 172.21.15.42:0/759225602 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f9ce40105d0 con 0x7f9cf41015d0 2024-03-12T04:14:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.575+0000 7f9cf17fa700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ce00a1b50 0x7f9ce00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.575+0000 7f9cf37fe700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ce00a1b50 0x7f9ce00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.575+0000 7f9cf17fa700 1 -- 172.21.15.42:0/759225602 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9ce40cd410 con 0x7f9cf41015d0 2024-03-12T04:14:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.575+0000 7f9cf37fe700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ce00a1b50 0x7f9ce00a4010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4107540 tx=0x7f9cdc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:03.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.578+0000 7f9cf17fa700 1 -- 172.21.15.42:0/759225602 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ce4096490 con 0x7f9cf41015d0 2024-03-12T04:14:03.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.785+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9cf4199660 con 0x7f9cf41015d0 2024-03-12T04:14:03.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.785+0000 7f9cf17fa700 1 -- 172.21.15.42:0/759225602 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ce4020020 con 0x7f9cf41015d0 2024-03-12T04:14:03.788 INFO:teuthology.orchestra.run.smithi042.stdout:107374182469 2024-03-12T04:14:03.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ce00a1b50 msgr2=0x7f9ce00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:03.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ce00a1b50 0x7f9ce00a4010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4107540 tx=0x7f9cdc008040 comp rx=0 tx=0).stop 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf41015d0 msgr2=0x7f9cf41982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf41015d0 0x7f9cf41982d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400d950 tx=0x7f9ce400dc60 comp rx=0 tx=0).stop 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 shutdown_connections 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ce00a1b50 0x7f9ce00a4010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4105ef0 0x7f9cf419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cf41015d0 0x7f9cf41982d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 --2- 172.21.15.42:0/759225602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf4101f80 0x7f9cf4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.787+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 >> 172.21.15.42:0/759225602 conn(0x7f9cf4077170 msgr2=0x7f9cf4100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.788+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 shutdown_connections 2024-03-12T04:14:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:03.788+0000 7f9cfa462700 1 -- 172.21.15.42:0/759225602 wait complete. 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 -- 172.21.15.42:0/1766331846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0100b50 msgr2=0x7f8fe0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/1766331846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0100b50 0x7f8fe0100fd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009990 tx=0x7f8fd4009ca0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 -- 172.21.15.42:0/1766331846 shutdown_connections 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/1766331846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe0101510 0x7f8fe010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/1766331846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fe0105ab0 0x7f8fe0105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/1766331846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0100b50 0x7f8fe0100fd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 -- 172.21.15.42:0/1766331846 >> 172.21.15.42:0/1766331846 conn(0x7f8fe0077b80 msgr2=0x7f8fe0077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 -- 172.21.15.42:0/1766331846 shutdown_connections 2024-03-12T04:14:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 -- 172.21.15.42:0/1766331846 wait complete. 2024-03-12T04:14:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 Processor -- start 2024-03-12T04:14:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.119+0000 7f8fe5d52700 1 -- start start 2024-03-12T04:14:04.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fe5d52700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fe0100b50 0x7f8fe0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fe5d52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0101510 0x7f8fe01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fe5d52700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe0105ab0 0x7f8fe019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fe5d52700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe019cf40 con 0x7f8fe0101510 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fe5d52700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe019d0b0 con 0x7f8fe0105ab0 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fe5d52700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe019d270 con 0x7f8fe0100b50 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0101510 0x7f8fe01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0101510 0x7f8fe01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58570/0 (socket says 172.21.15.42:58570) 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 -- 172.21.15.42:0/2290389840 learned_addr learned my addr 172.21.15.42:0/2290389840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdffff700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe0105ab0 0x7f8fe019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdf7fe700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fe0100b50 0x7f8fe0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 -- 172.21.15.42:0/2290389840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fe0100b50 msgr2=0x7f8fe0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fe0100b50 0x7f8fe0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 -- 172.21.15.42:0/2290389840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe0105ab0 msgr2=0x7f8fe019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe0105ab0 0x7f8fe019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 -- 172.21.15.42:0/2290389840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd40095f0 con 0x7f8fe0101510 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdf7fe700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fe0100b50 0x7f8fe0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.120+0000 7f8fdeffd700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0101510 0x7f8fe01985a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400f6c0 tx=0x7f8fd400f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.121+0000 7f8fdcff9700 1 -- 172.21.15.42:0/2290389840 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd401d070 con 0x7f8fe0101510 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.121+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe019d4f0 con 0x7f8fe0101510 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.121+0000 7f8fdcff9700 1 -- 172.21.15.42:0/2290389840 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8fd400bdc0 con 0x7f8fe0101510 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.121+0000 7f8fdcff9700 1 -- 172.21.15.42:0/2290389840 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4022ca0 con 0x7f8fe0101510 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.121+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe01a4db0 con 0x7f8fe0101510 2024-03-12T04:14:04.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.122+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe010a0a0 con 0x7f8fe0101510 2024-03-12T04:14:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.123+0000 7f8fdcff9700 1 -- 172.21.15.42:0/2290389840 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f8fd4022470 con 0x7f8fe0101510 2024-03-12T04:14:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.123+0000 7f8fdcff9700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8fc80a1b50 0x7f8fc80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.124+0000 7f8fdf7fe700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8fc80a1b50 0x7f8fc80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.124+0000 7f8fdcff9700 1 -- 172.21.15.42:0/2290389840 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8fd40ce890 con 0x7f8fe0101510 2024-03-12T04:14:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.124+0000 7f8fdf7fe700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8fc80a1b50 0x7f8fc80a4010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000baa0 tx=0x7f8fd0005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:04.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.128+0000 7f8fdcff9700 1 -- 172.21.15.42:0/2290389840 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fd4097a00 con 0x7f8fe0101510 2024-03-12T04:14:04.230 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:03 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/759225602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T04:14:04.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.331+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f8fe01993c0 con 0x7f8fe0101510 2024-03-12T04:14:04.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.332+0000 7f8fdcff9700 1 -- 172.21.15.42:0/2290389840 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fd4027090 con 0x7f8fe0101510 2024-03-12T04:14:04.335 INFO:teuthology.orchestra.run.smithi042.stdout:261993005100 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8fc80a1b50 msgr2=0x7f8fc80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8fc80a1b50 0x7f8fc80a4010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000baa0 tx=0x7f8fd0005db0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0101510 msgr2=0x7f8fe01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0101510 0x7f8fe01985a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400f6c0 tx=0x7f8fd400f6f0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 shutdown_connections 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f8fc80a1b50 0x7f8fc80a4010 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe0105ab0 0x7f8fe019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fe0100b50 0x7f8fe0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 --2- 172.21.15.42:0/2290389840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0101510 0x7f8fe01985a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 >> 172.21.15.42:0/2290389840 conn(0x7f8fe0077b80 msgr2=0x7f8fe006a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 shutdown_connections 2024-03-12T04:14:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f8fe5d52700 1 -- 172.21.15.42:0/2290389840 wait complete. 2024-03-12T04:14:04.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:03 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/759225602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T04:14:04.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:03 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/759225602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T04:14:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2290389840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-12T04:14:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1584715618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T04:14:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:04 smithi042 ceph-mon[32260]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:05.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182469 for osd.3 2024-03-12T04:14:05.139 DEBUG:teuthology.parallel:result is None 2024-03-12T04:14:05.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2290389840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-12T04:14:05.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1584715618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T04:14:05.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:04 smithi119 ceph-mon[36119]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:05.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2290389840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-12T04:14:05.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1584715618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T04:14:05.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:04 smithi136 ceph-mon[35571]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:05.549 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005095 got 261993005100 for osd.8 2024-03-12T04:14:05.549 DEBUG:teuthology.parallel:result is None 2024-03-12T04:14:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/1706654334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 msgr2=0x7ff49c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/1706654334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c1067c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009990 tx=0x7ff48c009ca0 comp rx=0 tx=0).stop 2024-03-12T04:14:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/1706654334 shutdown_connections 2024-03-12T04:14:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/1706654334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c106ea0 0x7ff49c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/1706654334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c101dc0 0x7ff49c1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/1706654334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c1067c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/1706654334 >> 172.21.15.42:0/1706654334 conn(0x7ff49c0fd670 msgr2=0x7ff49c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/1706654334 shutdown_connections 2024-03-12T04:14:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.967+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/1706654334 wait complete. 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 Processor -- start 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 -- start start 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c101dc0 0x7ff49c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106ea0 0x7ff49c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19cfc0 con 0x7ff49c102770 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19d130 con 0x7ff49c101dc0 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a2b9d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19d2f0 con 0x7ff49c106ea0 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a139a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a139a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58618/0 (socket says 172.21.15.42:58618) 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a139a700 1 -- 172.21.15.42:0/2972091423 learned_addr learned my addr 172.21.15.42:0/2972091423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a1b9b700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c101dc0 0x7ff49c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a239c700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106ea0 0x7ff49c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a139a700 1 -- 172.21.15.42:0/2972091423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106ea0 msgr2=0x7ff49c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.968+0000 7ff4a139a700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106ea0 0x7ff49c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff4a139a700 1 -- 172.21.15.42:0/2972091423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c101dc0 msgr2=0x7ff49c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff4a139a700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c101dc0 0x7ff49c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff4a139a700 1 -- 172.21.15.42:0/2972091423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0095f0 con 0x7ff49c102770 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff4a1b9b700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c101dc0 0x7ff49c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff4a139a700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c198670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff492ffd700 1 -- 172.21.15.42:0/2972091423 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c01d070 con 0x7ff49c102770 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff492ffd700 1 -- 172.21.15.42:0/2972091423 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff48c00fd80 con 0x7ff49c102770 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c19d570 con 0x7ff49c102770 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff492ffd700 1 -- 172.21.15.42:0/2972091423 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c022df0 con 0x7ff49c102770 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.969+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c1a4e30 con 0x7ff49c102770 2024-03-12T04:14:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.970+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c10a1c0 con 0x7ff49c102770 2024-03-12T04:14:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.972+0000 7ff492ffd700 1 -- 172.21.15.42:0/2972091423 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff48c022590 con 0x7ff49c102770 2024-03-12T04:14:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.972+0000 7ff492ffd700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4880a1b50 0x7ff4880a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.972+0000 7ff4a1b9b700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4880a1b50 0x7ff4880a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.972+0000 7ff492ffd700 1 -- 172.21.15.42:0/2972091423 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff48c0cef70 con 0x7ff49c102770 2024-03-12T04:14:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.973+0000 7ff4a1b9b700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4880a1b50 0x7ff4880a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff4980099d0 tx=0x7ff498008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:05.976+0000 7ff492ffd700 1 -- 172.21.15.42:0/2972091423 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff48c0980e0 con 0x7ff49c102770 2024-03-12T04:14:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 -- 172.21.15.42:0/3962580077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406adc0 msgr2=0x7f0e041068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 --2- 172.21.15.42:0/3962580077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406adc0 0x7f0e041068c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009990 tx=0x7f0e00009ca0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 -- 172.21.15.42:0/3962580077 shutdown_connections 2024-03-12T04:14:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 --2- 172.21.15.42:0/3962580077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e04106ff0 0x7f0e0410eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 --2- 172.21.15.42:0/3962580077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e0406a410 0x7f0e0406a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 --2- 172.21.15.42:0/3962580077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406adc0 0x7f0e041068c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 -- 172.21.15.42:0/3962580077 >> 172.21.15.42:0/3962580077 conn(0x7f0e04076bc0 msgr2=0x7f0e04076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 -- 172.21.15.42:0/3962580077 shutdown_connections 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.329+0000 7f0e0c423700 1 -- 172.21.15.42:0/3962580077 wait complete. 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 Processor -- start 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 -- start start 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406a410 0x7f0e04100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e0406adc0 0x7f0e041011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e04106ff0 0x7f0e041016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041a6500 con 0x7f0e0406a410 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041a6670 con 0x7f0e0406adc0 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0c423700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041a6830 con 0x7f0e04106ff0 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406a410 0x7f0e04100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406a410 0x7f0e04100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58592/0 (socket says 172.21.15.42:58592) 2024-03-12T04:14:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 -- 172.21.15.42:0/1584715618 learned_addr learned my addr 172.21.15.42:0/1584715618 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e099be700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e0406adc0 0x7f0e041011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:06.061 INFO:teuthology.orchestra.run.smithi042.stdout:34359738452 2024-03-12T04:14:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 -- 172.21.15.42:0/1584715618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e04106ff0 msgr2=0x7f0e041016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a9c0700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e04106ff0 0x7f0e041016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e04106ff0 0x7f0e041016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 -- 172.21.15.42:0/1584715618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e0406adc0 msgr2=0x7f0e041011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e0406adc0 0x7f0e041011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.330+0000 7f0e0a1bf700 1 -- 172.21.15.42:0/1584715618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000095f0 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.331+0000 7f0e0a1bf700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406a410 0x7f0e04100c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0df800b530 tx=0x7f0df800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.331+0000 7f0df77fe700 1 -- 172.21.15.42:0/1584715618 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8011830 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.331+0000 7f0df77fe700 1 -- 172.21.15.42:0/1584715618 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0df8011e40 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.331+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e041a6b10 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.331+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e041a7030 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.331+0000 7f0df77fe700 1 -- 172.21.15.42:0/1584715618 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800f2e0 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.332+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0410a1a0 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.333+0000 7f0df77fe700 1 -- 172.21.15.42:0/1584715618 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f0df8010380 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f0df77fe700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0df00a1bd0 0x7f0df00a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f0e099be700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0df00a1bd0 0x7f0df00a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f0df77fe700 1 -- 172.21.15.42:0/1584715618 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0df80cd4a0 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.334+0000 7f0e099be700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0df00a1bd0 0x7f0df00a4090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e00006010 tx=0x7f0e00005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.338+0000 7f0df77fe700 1 -- 172.21.15.42:0/1584715618 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0df8096690 con 0x7f0e0406a410 2024-03-12T04:14:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.546+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0e04111bf0 con 0x7f0e0406a410 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.546+0000 7f0df77fe700 1 -- 172.21.15.42:0/1584715618 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0df8099d50 con 0x7f0e0406a410 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0df00a1bd0 msgr2=0x7f0df00a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0df00a1bd0 0x7f0df00a4090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e00006010 tx=0x7f0e00005960 comp rx=0 tx=0).stop 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406a410 msgr2=0x7f0e04100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406a410 0x7f0e04100c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0df800b530 tx=0x7f0df800b840 comp rx=0 tx=0).stop 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 shutdown_connections 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0df00a1bd0 0x7f0df00a4090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e04106ff0 0x7f0e041016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0406a410 0x7f0e04100c70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 --2- 172.21.15.42:0/1584715618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e0406adc0 0x7f0e041011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.548+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 >> 172.21.15.42:0/1584715618 conn(0x7f0e04076bc0 msgr2=0x7f0e040ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.549+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 shutdown_connections 2024-03-12T04:14:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:04.549+0000 7f0e0c423700 1 -- 172.21.15.42:0/1584715618 wait complete. 2024-03-12T04:14:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.179+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7ff49c199440 con 0x7ff49c102770 2024-03-12T04:14:06.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.179+0000 7ff492ffd700 1 -- 172.21.15.42:0/2972091423 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff48c09b7a0 con 0x7ff49c102770 2024-03-12T04:14:06.182 INFO:teuthology.orchestra.run.smithi042.stdout:317827579937 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4880a1b50 msgr2=0x7ff4880a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4880a1b50 0x7ff4880a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff4980099d0 tx=0x7ff498008040 comp rx=0 tx=0).stop 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 msgr2=0x7ff49c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c198670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c0057c0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 shutdown_connections 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4880a1b50 0x7ff4880a4010 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106ea0 0x7ff49c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c101dc0 0x7ff49c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 --2- 172.21.15.42:0/2972091423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff49c102770 0x7ff49c198670 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 >> 172.21.15.42:0/2972091423 conn(0x7ff49c0fd670 msgr2=0x7ff49c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 shutdown_connections 2024-03-12T04:14:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:06.181+0000 7ff4a2b9d700 1 -- 172.21.15.42:0/2972091423 wait complete. 2024-03-12T04:14:06.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:06 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2972091423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-12T04:14:06.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:06 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2972091423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-12T04:14:06.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:06 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2972091423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-12T04:14:06.772 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738452 for osd.0 2024-03-12T04:14:06.773 DEBUG:teuthology.parallel:result is None 2024-03-12T04:14:07.499 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:07 smithi042 ceph-mon[32260]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:07.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:07 smithi119 ceph-mon[36119]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 -- 172.21.15.42:0/306546942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc06a3c0 msgr2=0x7fc6dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/306546942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc06a3c0 0x7fc6dc06a840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00099c0 tx=0x7fc6d0009cd0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 -- 172.21.15.42:0/306546942 shutdown_connections 2024-03-12T04:14:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/306546942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6dc06ad80 0x7fc6dc10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/306546942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6dc107ca0 0x7fc6dc108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/306546942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc06a3c0 0x7fc6dc06a840 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 -- 172.21.15.42:0/306546942 >> 172.21.15.42:0/306546942 conn(0x7fc6dc077b80 msgr2=0x7fc6dc077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 -- 172.21.15.42:0/306546942 shutdown_connections 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 -- 172.21.15.42:0/306546942 wait complete. 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 Processor -- start 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.600+0000 7fc6e0f56700 1 -- start start 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6e0f56700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6dc06a3c0 0x7fc6dc198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6e0f56700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6dc06ad80 0x7fc6dc198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6e0f56700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc107ca0 0x7fc6dc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6e0f56700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc19d000 con 0x7fc6dc107ca0 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6e0f56700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc19d170 con 0x7fc6dc06a3c0 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6e0f56700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc19d330 con 0x7fc6dc06ad80 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc107ca0 0x7fc6dc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc107ca0 0x7fc6dc19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58638/0 (socket says 172.21.15.42:58638) 2024-03-12T04:14:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 -- 172.21.15.42:0/2790306375 learned_addr learned my addr 172.21.15.42:0/2790306375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6da59c700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6dc06a3c0 0x7fc6dc198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 -- 172.21.15.42:0/2790306375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6dc06ad80 msgr2=0x7fc6dc198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6dc06ad80 0x7fc6dc198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 -- 172.21.15.42:0/2790306375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6dc06a3c0 msgr2=0x7fc6dc198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6dc06a3c0 0x7fc6dc198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 -- 172.21.15.42:0/2790306375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d00095f0 con 0x7fc6dc107ca0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6dad9d700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc107ca0 0x7fc6dc19c9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400b970 tx=0x7fc6d400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6cb7fe700 1 -- 172.21.15.42:0/2790306375 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400c9b0 con 0x7fc6dc107ca0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc19d610 con 0x7fc6dc107ca0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6cb7fe700 1 -- 172.21.15.42:0/2790306375 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6d4014440 con 0x7fc6dc107ca0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.601+0000 7fc6cb7fe700 1 -- 172.21.15.42:0/2790306375 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40075d0 con 0x7fc6dc107ca0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.602+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc1a4e70 con 0x7fc6dc107ca0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.602+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc10a180 con 0x7fc6dc107ca0 2024-03-12T04:14:07.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.603+0000 7fc6cb7fe700 1 -- 172.21.15.42:0/2790306375 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fc6d40145b0 con 0x7fc6dc107ca0 2024-03-12T04:14:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.604+0000 7fc6cb7fe700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc6c40a1b50 0x7fc6c40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.604+0000 7fc6da59c700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc6c40a1b50 0x7fc6c40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.604+0000 7fc6cb7fe700 1 -- 172.21.15.42:0/2790306375 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc6d4006e40 con 0x7fc6dc107ca0 2024-03-12T04:14:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.604+0000 7fc6da59c700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc6c40a1b50 0x7fc6c40a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006fd0 tx=0x7fc6cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.609+0000 7fc6cb7fe700 1 -- 172.21.15.42:0/2790306375 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6d4095d90 con 0x7fc6dc107ca0 2024-03-12T04:14:07.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:07 smithi136 ceph-mon[35571]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:07.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.811+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc6dc111c50 con 0x7fc6dc107ca0 2024-03-12T04:14:07.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.812+0000 7fc6cb7fe700 1 -- 172.21.15.42:0/2790306375 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6d4099450 con 0x7fc6dc107ca0 2024-03-12T04:14:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:141733920832 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc6c40a1b50 msgr2=0x7fc6c40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc6c40a1b50 0x7fc6c40a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006fd0 tx=0x7fc6cc008040 comp rx=0 tx=0).stop 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc107ca0 msgr2=0x7fc6dc19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc107ca0 0x7fc6dc19c9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400b970 tx=0x7fc6d400bd30 comp rx=0 tx=0).stop 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 shutdown_connections 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc6c40a1b50 0x7fc6c40a4010 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6dc107ca0 0x7fc6dc19c9f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6dc06a3c0 0x7fc6dc198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 --2- 172.21.15.42:0/2790306375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6dc06ad80 0x7fc6dc198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 >> 172.21.15.42:0/2790306375 conn(0x7fc6dc077b80 msgr2=0x7fc6dc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 shutdown_connections 2024-03-12T04:14:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:07.814+0000 7fc6e0f56700 1 -- 172.21.15.42:0/2790306375 wait complete. 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 -- 172.21.15.42:0/4292041509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508101cc0 msgr2=0x7ff508102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/4292041509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508101cc0 0x7ff508102140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0099c0 tx=0x7ff4fc009cd0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 -- 172.21.15.42:0/4292041509 shutdown_connections 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/4292041509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff508102680 0x7ff50810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/4292041509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff508107ca0 0x7ff508108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/4292041509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508101cc0 0x7ff508102140 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 -- 172.21.15.42:0/4292041509 >> 172.21.15.42:0/4292041509 conn(0x7ff508078da0 msgr2=0x7ff5080791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.033+0000 7ff50e3e4700 1 -- 172.21.15.42:0/4292041509 shutdown_connections 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 -- 172.21.15.42:0/4292041509 wait complete. 2024-03-12T04:14:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 Processor -- start 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 -- start start 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff508101cc0 0x7ff508198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff508102680 0x7ff5081985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508107ca0 0x7ff50819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50819cf50 con 0x7ff508107ca0 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50819d0c0 con 0x7ff508102680 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.034+0000 7ff50e3e4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50819d280 con 0x7ff508101cc0 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508107ca0 0x7ff50819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff507fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff508101cc0 0x7ff508198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508107ca0 0x7ff50819c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58662/0 (socket says 172.21.15.42:58662) 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff507fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff508101cc0 0x7ff508198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:49232/0 (socket says 172.21.15.42:49232) 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 -- 172.21.15.42:0/2327898561 learned_addr learned my addr 172.21.15.42:0/2327898561 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff5077fe700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff508102680 0x7ff5081985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 -- 172.21.15.42:0/2327898561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff508101cc0 msgr2=0x7ff508198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff508101cc0 0x7ff508198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 -- 172.21.15.42:0/2327898561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff508102680 msgr2=0x7ff5081985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff508102680 0x7ff5081985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 -- 172.21.15.42:0/2327898561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc0095f0 con 0x7ff508107ca0 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50c981700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508107ca0 0x7ff50819c940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff50000ada0 tx=0x7ff50000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff5057fa700 1 -- 172.21.15.42:0/2327898561 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000f070 con 0x7ff508107ca0 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff5057fa700 1 -- 172.21.15.42:0/2327898561 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff500007b30 con 0x7ff508107ca0 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50819d560 con 0x7ff508107ca0 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff5057fa700 1 -- 172.21.15.42:0/2327898561 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500005280 con 0x7ff508107ca0 2024-03-12T04:14:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.035+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5081a4dc0 con 0x7ff508107ca0 2024-03-12T04:14:08.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.036+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50810a180 con 0x7ff508107ca0 2024-03-12T04:14:08.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.038+0000 7ff5057fa700 1 -- 172.21.15.42:0/2327898561 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff50000c8d0 con 0x7ff508107ca0 2024-03-12T04:14:08.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.039+0000 7ff5057fa700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4f00a1b50 0x7ff4f00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:08.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.039+0000 7ff507fff700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4f00a1b50 0x7ff4f00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:08.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.039+0000 7ff5057fa700 1 -- 172.21.15.42:0/2327898561 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff5000ccee0 con 0x7ff508107ca0 2024-03-12T04:14:08.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.039+0000 7ff507fff700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4f00a1b50 0x7ff4f00a4010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005950 tx=0x7ff4f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:08.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.042+0000 7ff5057fa700 1 -- 172.21.15.42:0/2327898561 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff500095ff0 con 0x7ff508107ca0 2024-03-12T04:14:08.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.247+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7ff5081993d0 con 0x7ff508107ca0 2024-03-12T04:14:08.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.247+0000 7ff5057fa700 1 -- 172.21.15.42:0/2327898561 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff5000996b0 con 0x7ff508107ca0 2024-03-12T04:14:08.250 INFO:teuthology.orchestra.run.smithi042.stdout:352187318300 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4f00a1b50 msgr2=0x7ff4f00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4f00a1b50 0x7ff4f00a4010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005950 tx=0x7ff4f800b410 comp rx=0 tx=0).stop 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508107ca0 msgr2=0x7ff50819c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508107ca0 0x7ff50819c940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff50000ada0 tx=0x7ff50000c330 comp rx=0 tx=0).stop 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 shutdown_connections 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff4f00a1b50 0x7ff4f00a4010 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff508107ca0 0x7ff50819c940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff508101cc0 0x7ff508198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 --2- 172.21.15.42:0/2327898561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff508102680 0x7ff5081985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 >> 172.21.15.42:0/2327898561 conn(0x7ff508078da0 msgr2=0x7ff508100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 shutdown_connections 2024-03-12T04:14:08.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:08.249+0000 7ff50e3e4700 1 -- 172.21.15.42:0/2327898561 wait complete. 2024-03-12T04:14:08.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:08 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2790306375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T04:14:08.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:08 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2790306375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T04:14:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:08 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2790306375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T04:14:09.114 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920827 got 141733920832 for osd.4 2024-03-12T04:14:09.115 DEBUG:teuthology.parallel:result is None 2024-03-12T04:14:09.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:09 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2327898561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-12T04:14:09.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:09 smithi042 ceph-mon[32260]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:14:09.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:09 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2327898561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-12T04:14:09.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:09 smithi119 ceph-mon[36119]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:14:09.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:09 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2327898561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-12T04:14:09.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:09 smithi136 ceph-mon[35571]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:14:09.745 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579930 got 317827579937 for osd.10 2024-03-12T04:14:09.745 DEBUG:teuthology.parallel:result is None 2024-03-12T04:14:10.219 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318294 got 352187318300 for osd.11 2024-03-12T04:14:10.219 DEBUG:teuthology.parallel:result is None 2024-03-12T04:14:10.219 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-12T04:14:10.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-12T04:14:10.501 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:11.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:10 smithi042 ceph-mon[32260]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:11.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:10 smithi119 ceph-mon[36119]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:11.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:10 smithi136 ceph-mon[35571]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 -- 172.21.15.42:0/1502549061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65281015d0 msgr2=0x7f65281019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 --2- 172.21.15.42:0/1502549061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65281015d0 0x7f65281019b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6524009990 tx=0x7f6524009ca0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 -- 172.21.15.42:0/1502549061 shutdown_connections 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 --2- 172.21.15.42:0/1502549061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6528105ef0 0x7f652810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 --2- 172.21.15.42:0/1502549061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65281015d0 0x7f65281019b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 --2- 172.21.15.42:0/1502549061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6528101f80 0x7f6528105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 -- 172.21.15.42:0/1502549061 >> 172.21.15.42:0/1502549061 conn(0x7f6528077170 msgr2=0x7f65280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 -- 172.21.15.42:0/1502549061 shutdown_connections 2024-03-12T04:14:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.268+0000 7f6530665700 1 -- 172.21.15.42:0/1502549061 wait complete. 2024-03-12T04:14:12.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 Processor -- start 2024-03-12T04:14:12.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 -- start start 2024-03-12T04:14:12.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65281015d0 0x7f6528197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6528101f80 0x7f6528197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6528105ef0 0x7f652819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652819bf20 con 0x7f6528105ef0 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652819c090 con 0x7f6528101f80 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f6530665700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652819c250 con 0x7f65281015d0 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6528105ef0 0x7f652819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6528105ef0 0x7f652819b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36090/0 (socket says 172.21.15.42:36090) 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 -- 172.21.15.42:0/1278062684 learned_addr learned my addr 172.21.15.42:0/1278062684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652e401700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65281015d0 0x7f6528197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652dc00700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6528101f80 0x7f6528197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 -- 172.21.15.42:0/1278062684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65281015d0 msgr2=0x7f6528197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65281015d0 0x7f6528197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 -- 172.21.15.42:0/1278062684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6528101f80 msgr2=0x7f6528197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6528101f80 0x7f6528197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.269+0000 7f652ec02700 1 -- 172.21.15.42:0/1278062684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240095f0 con 0x7f6528105ef0 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.270+0000 7f652ec02700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6528105ef0 0x7f652819b910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6520007f10 tx=0x7f652000f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.270+0000 7f651f7fe700 1 -- 172.21.15.42:0/1278062684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000fd30 con 0x7f6528105ef0 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.270+0000 7f651f7fe700 1 -- 172.21.15.42:0/1278062684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f652000fe90 con 0x7f6528105ef0 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.270+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652819c530 con 0x7f6528105ef0 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.270+0000 7f651f7fe700 1 -- 172.21.15.42:0/1278062684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520005020 con 0x7f6528105ef0 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.270+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6528100430 con 0x7f6528105ef0 2024-03-12T04:14:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.271+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6528102380 con 0x7f6528105ef0 2024-03-12T04:14:12.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.272+0000 7f651f7fe700 1 -- 172.21.15.42:0/1278062684 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f6520004ad0 con 0x7f6528105ef0 2024-03-12T04:14:12.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.273+0000 7f651f7fe700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f65140a1bd0 0x7f65140a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:12.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.273+0000 7f652e401700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f65140a1bd0 0x7f65140a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:12.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.273+0000 7f651f7fe700 1 -- 172.21.15.42:0/1278062684 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f65200cfac0 con 0x7f6528105ef0 2024-03-12T04:14:12.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.273+0000 7f652e401700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f65140a1bd0 0x7f65140a4090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f652400b5c0 tx=0x7f6524005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:12.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.280+0000 7f651f7fe700 1 -- 172.21.15.42:0/1278062684 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6520098ba0 con 0x7f6528105ef0 2024-03-12T04:14:12.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.482+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6528110af0 con 0x7f65140a1bd0 2024-03-12T04:14:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.489+0000 7f651f7fe700 1 -- 172.21.15.42:0/1278062684 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258972 (secure 0 0 0) 0x7f6528110af0 con 0x7f65140a1bd0 2024-03-12T04:14:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f65140a1bd0 msgr2=0x7f65140a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f65140a1bd0 0x7f65140a4090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f652400b5c0 tx=0x7f6524005960 comp rx=0 tx=0).stop 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6528105ef0 msgr2=0x7f652819b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6528105ef0 0x7f652819b910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6520007f10 tx=0x7f652000f5c0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 shutdown_connections 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f65140a1bd0 0x7f65140a4090 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6528105ef0 0x7f652819b910 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65281015d0 0x7f6528197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.492+0000 7f6530665700 1 --2- 172.21.15.42:0/1278062684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6528101f80 0x7f6528197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.493+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 >> 172.21.15.42:0/1278062684 conn(0x7f6528077170 msgr2=0x7f65280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.493+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 shutdown_connections 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:12.493+0000 7f6530665700 1 -- 172.21.15.42:0/1278062684 wait complete. 2024-03-12T04:14:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-03-12T04:14:13.216 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-12T04:14:10.917651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656572,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":860,"num_read_kb":551,"num_write":420,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373704,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568120,"statfs":{"total":1151940427776,"available":1151557754880,"internally_reserved":0,"allocated":12275712,"data_stored":7569828,"data_compressed":114537,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003117"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933854+0000","last_change":"2024-03-12T04:12:54.788665+0000","last_active":"2024-03-12T04:13:45.933854+0000","last_peered":"2024-03-12T04:13:45.933854+0000","last_clean":"2024-03-12T04:13:45.933854+0000","last_became_active":"2024-03-12T04:12:54.788334+0000","last_became_peered":"2024-03-12T04:12:54.788334+0000","last_unstale":"2024-03-12T04:13:45.933854+0000","last_undegraded":"2024-03-12T04:13:45.933854+0000","last_fullsized":"2024-03-12T04:13:45.933854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:02.442543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936756+0000","last_change":"2024-03-12T04:12:54.791942+0000","last_active":"2024-03-12T04:13:45.936756+0000","last_peered":"2024-03-12T04:13:45.936756+0000","last_clean":"2024-03-12T04:13:45.936756+0000","last_became_active":"2024-03-12T04:12:54.791579+0000","last_became_peered":"2024-03-12T04:12:54.791579+0000","last_unstale":"2024-03-12T04:13:45.936756+0000","last_undegraded":"2024-03-12T04:13:45.936756+0000","last_fullsized":"2024-03-12T04:13:45.936756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:51:08.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936570+0000","last_change":"2024-03-12T04:12:54.793238+0000","last_active":"2024-03-12T04:13:45.936570+0000","last_peered":"2024-03-12T04:13:45.936570+0000","last_clean":"2024-03-12T04:13:45.936570+0000","last_became_active":"2024-03-12T04:12:54.792877+0000","last_became_peered":"2024-03-12T04:12:54.792877+0000","last_unstale":"2024-03-12T04:13:45.936570+0000","last_undegraded":"2024-03-12T04:13:45.936570+0000","last_fullsized":"2024-03-12T04:13:45.936570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:36:46.914204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099374+0000","last_change":"2024-03-12T04:12:54.797573+0000","last_active":"2024-03-12T04:13:46.099374+0000","last_peered":"2024-03-12T04:13:46.099374+0000","last_clean":"2024-03-12T04:13:46.099374+0000","last_became_active":"2024-03-12T04:12:54.796939+0000","last_became_peered":"2024-03-12T04:12:54.796939+0000","last_unstale":"2024-03-12T04:13:46.099374+0000","last_undegraded":"2024-03-12T04:13:46.099374+0000","last_fullsized":"2024-03-12T04:13:46.099374+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:34:58.047492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936729+0000","last_change":"2024-03-12T04:12:54.780595+0000","last_active":"2024-03-12T04:13:45.936729+0000","last_peered":"2024-03-12T04:13:45.936729+0000","last_clean":"2024-03-12T04:13:45.936729+0000","last_became_active":"2024-03-12T04:12:54.780256+0000","last_became_peered":"2024-03-12T04:12:54.780256+0000","last_unstale":"2024-03-12T04:13:45.936729+0000","last_undegraded":"2024-03-12T04:13:45.936729+0000","last_fullsized":"2024-03-12T04:13:45.936729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:42.492006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934954+0000","last_change":"2024-03-12T04:13:45.934954+0000","last_active":"2024-03-12T04:13:45.934954+0000","last_peered":"2024-03-12T04:13:45.934954+0000","last_clean":"2024-03-12T04:13:45.934954+0000","last_became_active":"2024-03-12T04:13:45.934579+0000","last_became_peered":"2024-03-12T04:13:45.934579+0000","last_unstale":"2024-03-12T04:13:45.934954+0000","last_undegraded":"2024-03-12T04:13:45.934954+0000","last_fullsized":"2024-03-12T04:13:45.934954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:39:10.062304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099840+0000","last_change":"2024-03-12T04:12:54.781510+0000","last_active":"2024-03-12T04:13:46.099840+0000","last_peered":"2024-03-12T04:13:46.099840+0000","last_clean":"2024-03-12T04:13:46.099840+0000","last_became_active":"2024-03-12T04:12:54.781141+0000","last_became_peered":"2024-03-12T04:12:54.781141+0000","last_unstale":"2024-03-12T04:13:46.099840+0000","last_undegraded":"2024-03-12T04:13:46.099840+0000","last_fullsized":"2024-03-12T04:13:46.099840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:42:05.055308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937029+0000","last_change":"2024-03-12T04:12:54.782541+0000","last_active":"2024-03-12T04:13:45.937029+0000","last_peered":"2024-03-12T04:13:45.937029+0000","last_clean":"2024-03-12T04:13:45.937029+0000","last_became_active":"2024-03-12T04:12:54.781582+0000","last_became_peered":"2024-03-12T04:12:54.781582+0000","last_unstale":"2024-03-12T04:13:45.937029+0000","last_undegraded":"2024-03-12T04:13:45.937029+0000","last_fullsized":"2024-03-12T04:13:45.937029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:15.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.103255+0000","last_change":"2024-03-12T04:12:54.788201+0000","last_active":"2024-03-12T04:13:46.103255+0000","last_peered":"2024-03-12T04:13:46.103255+0000","last_clean":"2024-03-12T04:13:46.103255+0000","last_became_active":"2024-03-12T04:12:54.787875+0000","last_became_peered":"2024-03-12T04:12:54.787875+0000","last_unstale":"2024-03-12T04:13:46.103255+0000","last_undegraded":"2024-03-12T04:13:46.103255+0000","last_fullsized":"2024-03-12T04:13:46.103255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:40:39.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932924+0000","last_change":"2024-03-12T04:12:54.789748+0000","last_active":"2024-03-12T04:13:45.932924+0000","last_peered":"2024-03-12T04:13:45.932924+0000","last_clean":"2024-03-12T04:13:45.932924+0000","last_became_active":"2024-03-12T04:12:54.788838+0000","last_became_peered":"2024-03-12T04:12:54.788838+0000","last_unstale":"2024-03-12T04:13:45.932924+0000","last_undegraded":"2024-03-12T04:13:45.932924+0000","last_fullsized":"2024-03-12T04:13:45.932924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:39:51.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100115+0000","last_change":"2024-03-12T04:12:54.793101+0000","last_active":"2024-03-12T04:13:46.100115+0000","last_peered":"2024-03-12T04:13:46.100115+0000","last_clean":"2024-03-12T04:13:46.100115+0000","last_became_active":"2024-03-12T04:12:54.792488+0000","last_became_peered":"2024-03-12T04:12:54.792488+0000","last_unstale":"2024-03-12T04:13:46.100115+0000","last_undegraded":"2024-03-12T04:13:46.100115+0000","last_fullsized":"2024-03-12T04:13:46.100115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:33.862011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099333+0000","last_change":"2024-03-12T04:12:54.797953+0000","last_active":"2024-03-12T04:13:46.099333+0000","last_peered":"2024-03-12T04:13:46.099333+0000","last_clean":"2024-03-12T04:13:46.099333+0000","last_became_active":"2024-03-12T04:12:54.797413+0000","last_became_peered":"2024-03-12T04:12:54.797413+0000","last_unstale":"2024-03-12T04:13:46.099333+0000","last_undegraded":"2024-03-12T04:13:46.099333+0000","last_fullsized":"2024-03-12T04:13:46.099333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:18:01.140872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933743+0000","last_change":"2024-03-12T04:12:54.775570+0000","last_active":"2024-03-12T04:13:45.933743+0000","last_peered":"2024-03-12T04:13:45.933743+0000","last_clean":"2024-03-12T04:13:45.933743+0000","last_became_active":"2024-03-12T04:12:54.775186+0000","last_became_peered":"2024-03-12T04:12:54.775186+0000","last_unstale":"2024-03-12T04:13:45.933743+0000","last_undegraded":"2024-03-12T04:13:45.933743+0000","last_fullsized":"2024-03-12T04:13:45.933743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:26.004458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936505+0000","last_change":"2024-03-12T04:12:54.780859+0000","last_active":"2024-03-12T04:13:45.936505+0000","last_peered":"2024-03-12T04:13:45.936505+0000","last_clean":"2024-03-12T04:13:45.936505+0000","last_became_active":"2024-03-12T04:12:54.780484+0000","last_became_peered":"2024-03-12T04:12:54.780484+0000","last_unstale":"2024-03-12T04:13:45.936505+0000","last_undegraded":"2024-03-12T04:13:45.936505+0000","last_fullsized":"2024-03-12T04:13:45.936505+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:24:00.575324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936336+0000","last_change":"2024-03-12T04:12:54.796087+0000","last_active":"2024-03-12T04:13:45.936336+0000","last_peered":"2024-03-12T04:13:45.936336+0000","last_clean":"2024-03-12T04:13:45.936336+0000","last_became_active":"2024-03-12T04:12:54.793702+0000","last_became_peered":"2024-03-12T04:12:54.793702+0000","last_unstale":"2024-03-12T04:13:45.936336+0000","last_undegraded":"2024-03-12T04:13:45.936336+0000","last_fullsized":"2024-03-12T04:13:45.936336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:09:27.334791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102715+0000","last_change":"2024-03-12T04:12:54.783146+0000","last_active":"2024-03-12T04:13:46.102715+0000","last_peered":"2024-03-12T04:13:46.102715+0000","last_clean":"2024-03-12T04:13:46.102715+0000","last_became_active":"2024-03-12T04:12:54.782834+0000","last_became_peered":"2024-03-12T04:12:54.782834+0000","last_unstale":"2024-03-12T04:13:46.102715+0000","last_undegraded":"2024-03-12T04:13:46.102715+0000","last_fullsized":"2024-03-12T04:13:46.102715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:35:11.126629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934082+0000","last_change":"2024-03-12T04:12:54.794879+0000","last_active":"2024-03-12T04:13:45.934082+0000","last_peered":"2024-03-12T04:13:45.934082+0000","last_clean":"2024-03-12T04:13:45.934082+0000","last_became_active":"2024-03-12T04:12:54.794251+0000","last_became_peered":"2024-03-12T04:12:54.794251+0000","last_unstale":"2024-03-12T04:13:45.934082+0000","last_undegraded":"2024-03-12T04:13:45.934082+0000","last_fullsized":"2024-03-12T04:13:45.934082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:33.754426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930863+0000","last_change":"2024-03-12T04:12:54.791301+0000","last_active":"2024-03-12T04:13:45.930863+0000","last_peered":"2024-03-12T04:13:45.930863+0000","last_clean":"2024-03-12T04:13:45.930863+0000","last_became_active":"2024-03-12T04:12:54.790850+0000","last_became_peered":"2024-03-12T04:12:54.790850+0000","last_unstale":"2024-03-12T04:13:45.930863+0000","last_undegraded":"2024-03-12T04:13:45.930863+0000","last_fullsized":"2024-03-12T04:13:45.930863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:26:04.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932237+0000","last_change":"2024-03-12T04:12:54.793419+0000","last_active":"2024-03-12T04:13:45.932237+0000","last_peered":"2024-03-12T04:13:45.932237+0000","last_clean":"2024-03-12T04:13:45.932237+0000","last_became_active":"2024-03-12T04:12:54.793080+0000","last_became_peered":"2024-03-12T04:12:54.793080+0000","last_unstale":"2024-03-12T04:13:45.932237+0000","last_undegraded":"2024-03-12T04:13:45.932237+0000","last_fullsized":"2024-03-12T04:13:45.932237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:46:26.506899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:08.127915+0000","last_change":"2024-03-12T04:12:53.774893+0000","last_active":"2024-03-12T04:14:08.127915+0000","last_peered":"2024-03-12T04:14:08.127915+0000","last_clean":"2024-03-12T04:14:08.127915+0000","last_became_active":"2024-03-12T04:12:53.773801+0000","last_became_peered":"2024-03-12T04:12:53.773801+0000","last_unstale":"2024-03-12T04:14:08.127915+0000","last_undegraded":"2024-03-12T04:14:08.127915+0000","last_fullsized":"2024-03-12T04:14:08.127915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:30:19.995765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935920+0000","last_change":"2024-03-12T04:12:54.786795+0000","last_active":"2024-03-12T04:13:45.935920+0000","last_peered":"2024-03-12T04:13:45.935920+0000","last_clean":"2024-03-12T04:13:45.935920+0000","last_became_active":"2024-03-12T04:12:54.786481+0000","last_became_peered":"2024-03-12T04:12:54.786481+0000","last_unstale":"2024-03-12T04:13:45.935920+0000","last_undegraded":"2024-03-12T04:13:45.935920+0000","last_fullsized":"2024-03-12T04:13:45.935920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:06.849644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936174+0000","last_change":"2024-03-12T04:12:53.779582+0000","last_active":"2024-03-12T04:13:45.936174+0000","last_peered":"2024-03-12T04:13:45.936174+0000","last_clean":"2024-03-12T04:13:45.936174+0000","last_became_active":"2024-03-12T04:12:53.779062+0000","last_became_peered":"2024-03-12T04:12:53.779062+0000","last_unstale":"2024-03-12T04:13:45.936174+0000","last_undegraded":"2024-03-12T04:13:45.936174+0000","last_fullsized":"2024-03-12T04:13:45.936174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:44:01.065183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936867+0000","last_change":"2024-03-12T04:12:54.789198+0000","last_active":"2024-03-12T04:13:45.936867+0000","last_peered":"2024-03-12T04:13:45.936867+0000","last_clean":"2024-03-12T04:13:45.936867+0000","last_became_active":"2024-03-12T04:12:54.788213+0000","last_became_peered":"2024-03-12T04:12:54.788213+0000","last_unstale":"2024-03-12T04:13:45.936867+0000","last_undegraded":"2024-03-12T04:13:45.936867+0000","last_fullsized":"2024-03-12T04:13:45.936867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:24:36.561553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937326+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:45.937326+0000","last_peered":"2024-03-12T04:13:45.937326+0000","last_clean":"2024-03-12T04:13:45.937326+0000","last_became_active":"2024-03-12T04:12:53.771178+0000","last_became_peered":"2024-03-12T04:12:53.771178+0000","last_unstale":"2024-03-12T04:13:45.937326+0000","last_undegraded":"2024-03-12T04:13:45.937326+0000","last_fullsized":"2024-03-12T04:13:45.937326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:12.882128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102922+0000","last_change":"2024-03-12T04:12:54.799758+0000","last_active":"2024-03-12T04:13:46.102922+0000","last_peered":"2024-03-12T04:13:46.102922+0000","last_clean":"2024-03-12T04:13:46.102922+0000","last_became_active":"2024-03-12T04:12:54.799338+0000","last_became_peered":"2024-03-12T04:12:54.799338+0000","last_unstale":"2024-03-12T04:13:46.102922+0000","last_undegraded":"2024-03-12T04:13:46.102922+0000","last_fullsized":"2024-03-12T04:13:46.102922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:32:54.087890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937112+0000","last_change":"2024-03-12T04:12:53.774390+0000","last_active":"2024-03-12T04:13:45.937112+0000","last_peered":"2024-03-12T04:13:45.937112+0000","last_clean":"2024-03-12T04:13:45.937112+0000","last_became_active":"2024-03-12T04:12:53.774042+0000","last_became_peered":"2024-03-12T04:12:53.774042+0000","last_unstale":"2024-03-12T04:13:45.937112+0000","last_undegraded":"2024-03-12T04:13:45.937112+0000","last_fullsized":"2024-03-12T04:13:45.937112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:37.361505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.104074+0000","last_change":"2024-03-12T04:13:46.104074+0000","last_active":"2024-03-12T04:13:46.104074+0000","last_peered":"2024-03-12T04:13:46.104074+0000","last_clean":"2024-03-12T04:13:46.104074+0000","last_became_active":"2024-03-12T04:13:46.103719+0000","last_became_peered":"2024-03-12T04:13:46.103719+0000","last_unstale":"2024-03-12T04:13:46.104074+0000","last_undegraded":"2024-03-12T04:13:46.104074+0000","last_fullsized":"2024-03-12T04:13:46.104074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:29.979576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:08.129881+0000","last_change":"2024-03-12T04:12:53.772182+0000","last_active":"2024-03-12T04:14:08.129881+0000","last_peered":"2024-03-12T04:14:08.129881+0000","last_clean":"2024-03-12T04:14:08.129881+0000","last_became_active":"2024-03-12T04:12:53.771858+0000","last_became_peered":"2024-03-12T04:12:53.771858+0000","last_unstale":"2024-03-12T04:14:08.129881+0000","last_undegraded":"2024-03-12T04:14:08.129881+0000","last_fullsized":"2024-03-12T04:14:08.129881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:42:41.466311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936502+0000","last_change":"2024-03-12T04:12:54.779869+0000","last_active":"2024-03-12T04:13:45.936502+0000","last_peered":"2024-03-12T04:13:45.936502+0000","last_clean":"2024-03-12T04:13:45.936502+0000","last_became_active":"2024-03-12T04:12:54.779500+0000","last_became_peered":"2024-03-12T04:12:54.779500+0000","last_unstale":"2024-03-12T04:13:45.936502+0000","last_undegraded":"2024-03-12T04:13:45.936502+0000","last_fullsized":"2024-03-12T04:13:45.936502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:29.538570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932326+0000","last_change":"2024-03-12T04:12:53.774056+0000","last_active":"2024-03-12T04:13:45.932326+0000","last_peered":"2024-03-12T04:13:45.932326+0000","last_clean":"2024-03-12T04:13:45.932326+0000","last_became_active":"2024-03-12T04:12:53.773675+0000","last_became_peered":"2024-03-12T04:12:53.773675+0000","last_unstale":"2024-03-12T04:13:45.932326+0000","last_undegraded":"2024-03-12T04:13:45.932326+0000","last_fullsized":"2024-03-12T04:13:45.932326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:33.576947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935472+0000","last_change":"2024-03-12T04:12:54.790583+0000","last_active":"2024-03-12T04:13:45.935472+0000","last_peered":"2024-03-12T04:13:45.935472+0000","last_clean":"2024-03-12T04:13:45.935472+0000","last_became_active":"2024-03-12T04:12:54.788364+0000","last_became_peered":"2024-03-12T04:12:54.788364+0000","last_unstale":"2024-03-12T04:13:45.935472+0000","last_undegraded":"2024-03-12T04:13:45.935472+0000","last_fullsized":"2024-03-12T04:13:45.935472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:05:13.542279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933518+0000","last_change":"2024-03-12T04:12:53.769619+0000","last_active":"2024-03-12T04:13:45.933518+0000","last_peered":"2024-03-12T04:13:45.933518+0000","last_clean":"2024-03-12T04:13:45.933518+0000","last_became_active":"2024-03-12T04:12:53.769293+0000","last_became_peered":"2024-03-12T04:12:53.769293+0000","last_unstale":"2024-03-12T04:13:45.933518+0000","last_undegraded":"2024-03-12T04:13:45.933518+0000","last_fullsized":"2024-03-12T04:13:45.933518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:44.618499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935485+0000","last_change":"2024-03-12T04:12:54.776493+0000","last_active":"2024-03-12T04:13:45.935485+0000","last_peered":"2024-03-12T04:13:45.935485+0000","last_clean":"2024-03-12T04:13:45.935485+0000","last_became_active":"2024-03-12T04:12:54.775967+0000","last_became_peered":"2024-03-12T04:12:54.775967+0000","last_unstale":"2024-03-12T04:13:45.935485+0000","last_undegraded":"2024-03-12T04:13:45.935485+0000","last_fullsized":"2024-03-12T04:13:45.935485+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:19:46.824145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934388+0000","last_change":"2024-03-12T04:12:53.773523+0000","last_active":"2024-03-12T04:13:45.934388+0000","last_peered":"2024-03-12T04:13:45.934388+0000","last_clean":"2024-03-12T04:13:45.934388+0000","last_became_active":"2024-03-12T04:12:53.773161+0000","last_became_peered":"2024-03-12T04:12:53.773161+0000","last_unstale":"2024-03-12T04:13:45.934388+0000","last_undegraded":"2024-03-12T04:13:45.934388+0000","last_fullsized":"2024-03-12T04:13:45.934388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:41:40.810994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930799+0000","last_change":"2024-03-12T04:12:54.791688+0000","last_active":"2024-03-12T04:13:45.930799+0000","last_peered":"2024-03-12T04:13:45.930799+0000","last_clean":"2024-03-12T04:13:45.930799+0000","last_became_active":"2024-03-12T04:12:54.790854+0000","last_became_peered":"2024-03-12T04:12:54.790854+0000","last_unstale":"2024-03-12T04:13:45.930799+0000","last_undegraded":"2024-03-12T04:13:45.930799+0000","last_fullsized":"2024-03-12T04:13:45.930799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:42:50.343524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":161,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:08.129092+0000","last_change":"2024-03-12T04:12:53.777534+0000","last_active":"2024-03-12T04:14:08.129092+0000","last_peered":"2024-03-12T04:14:08.129092+0000","last_clean":"2024-03-12T04:14:08.129092+0000","last_became_active":"2024-03-12T04:12:53.777201+0000","last_became_peered":"2024-03-12T04:12:53.777201+0000","last_unstale":"2024-03-12T04:14:08.129092+0000","last_undegraded":"2024-03-12T04:14:08.129092+0000","last_fullsized":"2024-03-12T04:14:08.129092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:54.708892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.103229+0000","last_change":"2024-03-12T04:12:54.798962+0000","last_active":"2024-03-12T04:13:46.103229+0000","last_peered":"2024-03-12T04:13:46.103229+0000","last_clean":"2024-03-12T04:13:46.103229+0000","last_became_active":"2024-03-12T04:12:54.798650+0000","last_became_peered":"2024-03-12T04:12:54.798650+0000","last_unstale":"2024-03-12T04:13:46.103229+0000","last_undegraded":"2024-03-12T04:13:46.103229+0000","last_fullsized":"2024-03-12T04:13:46.103229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:05:29.270242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100119+0000","last_change":"2024-03-12T04:12:53.777791+0000","last_active":"2024-03-12T04:13:46.100119+0000","last_peered":"2024-03-12T04:13:46.100119+0000","last_clean":"2024-03-12T04:13:46.100119+0000","last_became_active":"2024-03-12T04:12:53.777430+0000","last_became_peered":"2024-03-12T04:12:53.777430+0000","last_unstale":"2024-03-12T04:13:46.100119+0000","last_undegraded":"2024-03-12T04:13:46.100119+0000","last_fullsized":"2024-03-12T04:13:46.100119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:38:21.858857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936587+0000","last_change":"2024-03-12T04:12:54.784710+0000","last_active":"2024-03-12T04:13:45.936587+0000","last_peered":"2024-03-12T04:13:45.936587+0000","last_clean":"2024-03-12T04:13:45.936587+0000","last_became_active":"2024-03-12T04:12:54.784342+0000","last_became_peered":"2024-03-12T04:12:54.784342+0000","last_unstale":"2024-03-12T04:13:45.936587+0000","last_undegraded":"2024-03-12T04:13:45.936587+0000","last_fullsized":"2024-03-12T04:13:45.936587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:21.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935890+0000","last_change":"2024-03-12T04:12:53.779289+0000","last_active":"2024-03-12T04:13:45.935890+0000","last_peered":"2024-03-12T04:13:45.935890+0000","last_clean":"2024-03-12T04:13:45.935890+0000","last_became_active":"2024-03-12T04:12:53.778894+0000","last_became_peered":"2024-03-12T04:12:53.778894+0000","last_unstale":"2024-03-12T04:13:45.935890+0000","last_undegraded":"2024-03-12T04:13:45.935890+0000","last_fullsized":"2024-03-12T04:13:45.935890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:13:33.328977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.103025+0000","last_change":"2024-03-12T04:12:54.794102+0000","last_active":"2024-03-12T04:13:46.103025+0000","last_peered":"2024-03-12T04:13:46.103025+0000","last_clean":"2024-03-12T04:13:46.103025+0000","last_became_active":"2024-03-12T04:12:54.793789+0000","last_became_peered":"2024-03-12T04:12:54.793789+0000","last_unstale":"2024-03-12T04:13:46.103025+0000","last_undegraded":"2024-03-12T04:13:46.103025+0000","last_fullsized":"2024-03-12T04:13:46.103025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:55:33.632297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:08.143239+0000","last_change":"2024-03-12T04:12:53.774847+0000","last_active":"2024-03-12T04:14:08.143239+0000","last_peered":"2024-03-12T04:14:08.143239+0000","last_clean":"2024-03-12T04:14:08.143239+0000","last_became_active":"2024-03-12T04:12:53.774457+0000","last_became_peered":"2024-03-12T04:12:53.774457+0000","last_unstale":"2024-03-12T04:14:08.143239+0000","last_undegraded":"2024-03-12T04:14:08.143239+0000","last_fullsized":"2024-03-12T04:14:08.143239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:17.350636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58766,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":145,"num_read_kb":129,"num_write":32,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937153+0000","last_change":"2024-03-12T04:12:54.783671+0000","last_active":"2024-03-12T04:13:45.937153+0000","last_peered":"2024-03-12T04:13:45.937153+0000","last_clean":"2024-03-12T04:13:45.937153+0000","last_became_active":"2024-03-12T04:12:54.782919+0000","last_became_peered":"2024-03-12T04:12:54.782919+0000","last_unstale":"2024-03-12T04:13:45.937153+0000","last_undegraded":"2024-03-12T04:13:45.937153+0000","last_fullsized":"2024-03-12T04:13:45.937153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:38:19.445096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.948316+0000","last_change":"2024-03-12T04:13:45.947951+0000","last_active":"2024-03-12T04:13:45.948316+0000","last_peered":"2024-03-12T04:13:45.948316+0000","last_clean":"2024-03-12T04:13:45.948316+0000","last_became_active":"2024-03-12T04:13:45.938099+0000","last_became_peered":"2024-03-12T04:13:45.938099+0000","last_unstale":"2024-03-12T04:13:45.948316+0000","last_undegraded":"2024-03-12T04:13:45.948316+0000","last_fullsized":"2024-03-12T04:13:45.948316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:03.905076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933675+0000","last_change":"2024-03-12T04:12:54.779319+0000","last_active":"2024-03-12T04:13:45.933675+0000","last_peered":"2024-03-12T04:13:45.933675+0000","last_clean":"2024-03-12T04:13:45.933675+0000","last_became_active":"2024-03-12T04:12:54.778927+0000","last_became_peered":"2024-03-12T04:12:54.778927+0000","last_unstale":"2024-03-12T04:13:45.933675+0000","last_undegraded":"2024-03-12T04:13:45.933675+0000","last_fullsized":"2024-03-12T04:13:45.933675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:20:53.947257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934971+0000","last_change":"2024-03-12T04:12:53.774754+0000","last_active":"2024-03-12T04:13:45.934971+0000","last_peered":"2024-03-12T04:13:45.934971+0000","last_clean":"2024-03-12T04:13:45.934971+0000","last_became_active":"2024-03-12T04:12:53.774395+0000","last_became_peered":"2024-03-12T04:12:53.774395+0000","last_unstale":"2024-03-12T04:13:45.934971+0000","last_undegraded":"2024-03-12T04:13:45.934971+0000","last_fullsized":"2024-03-12T04:13:45.934971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:13.286204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099966+0000","last_change":"2024-03-12T04:12:54.786891+0000","last_active":"2024-03-12T04:13:46.099966+0000","last_peered":"2024-03-12T04:13:46.099966+0000","last_clean":"2024-03-12T04:13:46.099966+0000","last_became_active":"2024-03-12T04:12:54.786591+0000","last_became_peered":"2024-03-12T04:12:54.786591+0000","last_unstale":"2024-03-12T04:13:46.099966+0000","last_undegraded":"2024-03-12T04:13:46.099966+0000","last_fullsized":"2024-03-12T04:13:46.099966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:16:09.188557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:08.129040+0000","last_change":"2024-03-12T04:12:53.769839+0000","last_active":"2024-03-12T04:14:08.129040+0000","last_peered":"2024-03-12T04:14:08.129040+0000","last_clean":"2024-03-12T04:14:08.129040+0000","last_became_active":"2024-03-12T04:12:53.769486+0000","last_became_peered":"2024-03-12T04:12:53.769486+0000","last_unstale":"2024-03-12T04:14:08.129040+0000","last_undegraded":"2024-03-12T04:14:08.129040+0000","last_fullsized":"2024-03-12T04:14:08.129040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:46.050018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.098955+0000","last_change":"2024-03-12T04:12:54.782447+0000","last_active":"2024-03-12T04:13:46.098955+0000","last_peered":"2024-03-12T04:13:46.098955+0000","last_clean":"2024-03-12T04:13:46.098955+0000","last_became_active":"2024-03-12T04:12:54.782091+0000","last_became_peered":"2024-03-12T04:12:54.782091+0000","last_unstale":"2024-03-12T04:13:46.098955+0000","last_undegraded":"2024-03-12T04:13:46.098955+0000","last_fullsized":"2024-03-12T04:13:46.098955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:22.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.959032+0000","last_change":"2024-03-12T04:13:45.958615+0000","last_active":"2024-03-12T04:13:45.959032+0000","last_peered":"2024-03-12T04:13:45.959032+0000","last_clean":"2024-03-12T04:13:45.959032+0000","last_became_active":"2024-03-12T04:13:45.938341+0000","last_became_peered":"2024-03-12T04:13:45.938341+0000","last_unstale":"2024-03-12T04:13:45.959032+0000","last_undegraded":"2024-03-12T04:13:45.959032+0000","last_fullsized":"2024-03-12T04:13:45.959032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:17:44.688344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935203+0000","last_change":"2024-03-12T04:12:54.784282+0000","last_active":"2024-03-12T04:13:45.935203+0000","last_peered":"2024-03-12T04:13:45.935203+0000","last_clean":"2024-03-12T04:13:45.935203+0000","last_became_active":"2024-03-12T04:12:54.783953+0000","last_became_peered":"2024-03-12T04:12:54.783953+0000","last_unstale":"2024-03-12T04:13:45.935203+0000","last_undegraded":"2024-03-12T04:13:45.935203+0000","last_fullsized":"2024-03-12T04:13:45.935203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:14:02.182546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937257+0000","last_change":"2024-03-12T04:12:53.770230+0000","last_active":"2024-03-12T04:13:45.937257+0000","last_peered":"2024-03-12T04:13:45.937257+0000","last_clean":"2024-03-12T04:13:45.937257+0000","last_became_active":"2024-03-12T04:12:53.768817+0000","last_became_peered":"2024-03-12T04:12:53.768817+0000","last_unstale":"2024-03-12T04:13:45.937257+0000","last_undegraded":"2024-03-12T04:13:45.937257+0000","last_fullsized":"2024-03-12T04:13:45.937257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:09:12.656489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933705+0000","last_change":"2024-03-12T04:12:54.785654+0000","last_active":"2024-03-12T04:13:45.933705+0000","last_peered":"2024-03-12T04:13:45.933705+0000","last_clean":"2024-03-12T04:13:45.933705+0000","last_became_active":"2024-03-12T04:12:54.785244+0000","last_became_peered":"2024-03-12T04:12:54.785244+0000","last_unstale":"2024-03-12T04:13:45.933705+0000","last_undegraded":"2024-03-12T04:13:45.933705+0000","last_fullsized":"2024-03-12T04:13:45.933705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:00:41.961637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930880+0000","last_change":"2024-03-12T04:12:53.780962+0000","last_active":"2024-03-12T04:13:45.930880+0000","last_peered":"2024-03-12T04:13:45.930880+0000","last_clean":"2024-03-12T04:13:45.930880+0000","last_became_active":"2024-03-12T04:12:53.780623+0000","last_became_peered":"2024-03-12T04:12:53.780623+0000","last_unstale":"2024-03-12T04:13:45.930880+0000","last_undegraded":"2024-03-12T04:13:45.930880+0000","last_fullsized":"2024-03-12T04:13:45.930880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932748+0000","last_change":"2024-03-12T04:12:54.775065+0000","last_active":"2024-03-12T04:13:45.932748+0000","last_peered":"2024-03-12T04:13:45.932748+0000","last_clean":"2024-03-12T04:13:45.932748+0000","last_became_active":"2024-03-12T04:12:54.774686+0000","last_became_peered":"2024-03-12T04:12:54.774686+0000","last_unstale":"2024-03-12T04:13:45.932748+0000","last_undegraded":"2024-03-12T04:13:45.932748+0000","last_fullsized":"2024-03-12T04:13:45.932748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:56:25.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:05.124888+0000","last_change":"2024-03-12T04:12:53.777702+0000","last_active":"2024-03-12T04:14:05.124888+0000","last_peered":"2024-03-12T04:14:05.124888+0000","last_clean":"2024-03-12T04:14:05.124888+0000","last_became_active":"2024-03-12T04:12:53.777346+0000","last_became_peered":"2024-03-12T04:12:53.777346+0000","last_unstale":"2024-03-12T04:14:05.124888+0000","last_undegraded":"2024-03-12T04:14:05.124888+0000","last_fullsized":"2024-03-12T04:14:05.124888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:09.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932521+0000","last_change":"2024-03-12T04:12:54.789087+0000","last_active":"2024-03-12T04:13:45.932521+0000","last_peered":"2024-03-12T04:13:45.932521+0000","last_clean":"2024-03-12T04:13:45.932521+0000","last_became_active":"2024-03-12T04:12:54.788735+0000","last_became_peered":"2024-03-12T04:12:54.788735+0000","last_unstale":"2024-03-12T04:13:45.932521+0000","last_undegraded":"2024-03-12T04:13:45.932521+0000","last_fullsized":"2024-03-12T04:13:45.932521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:14.947137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934523+0000","last_change":"2024-03-12T04:12:54.789974+0000","last_active":"2024-03-12T04:13:45.934523+0000","last_peered":"2024-03-12T04:13:45.934523+0000","last_clean":"2024-03-12T04:13:45.934523+0000","last_became_active":"2024-03-12T04:12:54.789578+0000","last_became_peered":"2024-03-12T04:12:54.789578+0000","last_unstale":"2024-03-12T04:13:45.934523+0000","last_undegraded":"2024-03-12T04:13:45.934523+0000","last_fullsized":"2024-03-12T04:13:45.934523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:22:25.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:05.124196+0000","last_change":"2024-03-12T04:12:53.773601+0000","last_active":"2024-03-12T04:14:05.124196+0000","last_peered":"2024-03-12T04:14:05.124196+0000","last_clean":"2024-03-12T04:14:05.124196+0000","last_became_active":"2024-03-12T04:12:53.773264+0000","last_became_peered":"2024-03-12T04:12:53.773264+0000","last_unstale":"2024-03-12T04:14:05.124196+0000","last_undegraded":"2024-03-12T04:14:05.124196+0000","last_fullsized":"2024-03-12T04:14:05.124196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:07:03.580157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937019+0000","last_change":"2024-03-12T04:12:54.782496+0000","last_active":"2024-03-12T04:13:45.937019+0000","last_peered":"2024-03-12T04:13:45.937019+0000","last_clean":"2024-03-12T04:13:45.937019+0000","last_became_active":"2024-03-12T04:12:54.782014+0000","last_became_peered":"2024-03-12T04:12:54.782014+0000","last_unstale":"2024-03-12T04:13:45.937019+0000","last_undegraded":"2024-03-12T04:13:45.937019+0000","last_fullsized":"2024-03-12T04:13:45.937019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:08.580655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099322+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:46.099322+0000","last_peered":"2024-03-12T04:13:46.099322+0000","last_clean":"2024-03-12T04:13:46.099322+0000","last_became_active":"2024-03-12T04:12:53.770855+0000","last_became_peered":"2024-03-12T04:12:53.770855+0000","last_unstale":"2024-03-12T04:13:46.099322+0000","last_undegraded":"2024-03-12T04:13:46.099322+0000","last_fullsized":"2024-03-12T04:13:46.099322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:32:33.969240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100244+0000","last_change":"2024-03-12T04:11:37.642780+0000","last_active":"2024-03-12T04:13:46.100244+0000","last_peered":"2024-03-12T04:13:46.100244+0000","last_clean":"2024-03-12T04:13:46.100244+0000","last_became_active":"2024-03-12T04:11:37.642145+0000","last_became_peered":"2024-03-12T04:11:37.642145+0000","last_unstale":"2024-03-12T04:13:46.100244+0000","last_undegraded":"2024-03-12T04:13:46.100244+0000","last_fullsized":"2024-03-12T04:13:46.100244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_clean_scrub_stamp":"2024-03-12T04:08:09.604393+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:46:36.666483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933359+0000","last_change":"2024-03-12T04:12:54.790464+0000","last_active":"2024-03-12T04:13:45.933359+0000","last_peered":"2024-03-12T04:13:45.933359+0000","last_clean":"2024-03-12T04:13:45.933359+0000","last_became_active":"2024-03-12T04:12:54.788440+0000","last_became_peered":"2024-03-12T04:12:54.788440+0000","last_unstale":"2024-03-12T04:13:45.933359+0000","last_undegraded":"2024-03-12T04:13:45.933359+0000","last_fullsized":"2024-03-12T04:13:45.933359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:54.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100091+0000","last_change":"2024-03-12T04:12:53.773334+0000","last_active":"2024-03-12T04:13:46.100091+0000","last_peered":"2024-03-12T04:13:46.100091+0000","last_clean":"2024-03-12T04:13:46.100091+0000","last_became_active":"2024-03-12T04:12:53.772759+0000","last_became_peered":"2024-03-12T04:12:53.772759+0000","last_unstale":"2024-03-12T04:13:46.100091+0000","last_undegraded":"2024-03-12T04:13:46.100091+0000","last_fullsized":"2024-03-12T04:13:46.100091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:03:17.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937169+0000","last_change":"2024-03-12T04:12:53.777035+0000","last_active":"2024-03-12T04:13:45.937169+0000","last_peered":"2024-03-12T04:13:45.937169+0000","last_clean":"2024-03-12T04:13:45.937169+0000","last_became_active":"2024-03-12T04:12:53.776587+0000","last_became_peered":"2024-03-12T04:12:53.776587+0000","last_unstale":"2024-03-12T04:13:45.937169+0000","last_undegraded":"2024-03-12T04:13:45.937169+0000","last_fullsized":"2024-03-12T04:13:45.937169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:21.390525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933689+0000","last_change":"2024-03-12T04:12:54.790835+0000","last_active":"2024-03-12T04:13:45.933689+0000","last_peered":"2024-03-12T04:13:45.933689+0000","last_clean":"2024-03-12T04:13:45.933689+0000","last_became_active":"2024-03-12T04:12:54.790512+0000","last_became_peered":"2024-03-12T04:12:54.790512+0000","last_unstale":"2024-03-12T04:13:45.933689+0000","last_undegraded":"2024-03-12T04:13:45.933689+0000","last_fullsized":"2024-03-12T04:13:45.933689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:37:58.029450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931022+0000","last_change":"2024-03-12T04:12:53.774604+0000","last_active":"2024-03-12T04:13:45.931022+0000","last_peered":"2024-03-12T04:13:45.931022+0000","last_clean":"2024-03-12T04:13:45.931022+0000","last_became_active":"2024-03-12T04:12:53.774270+0000","last_became_peered":"2024-03-12T04:12:53.774270+0000","last_unstale":"2024-03-12T04:13:45.931022+0000","last_undegraded":"2024-03-12T04:13:45.931022+0000","last_fullsized":"2024-03-12T04:13:45.931022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:23.635179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935988+0000","last_change":"2024-03-12T04:12:54.786259+0000","last_active":"2024-03-12T04:13:45.935988+0000","last_peered":"2024-03-12T04:13:45.935988+0000","last_clean":"2024-03-12T04:13:45.935988+0000","last_became_active":"2024-03-12T04:12:54.783114+0000","last_became_peered":"2024-03-12T04:12:54.783114+0000","last_unstale":"2024-03-12T04:13:45.935988+0000","last_undegraded":"2024-03-12T04:13:45.935988+0000","last_fullsized":"2024-03-12T04:13:45.935988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:53.036505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099781+0000","last_change":"2024-03-12T04:12:53.779618+0000","last_active":"2024-03-12T04:13:46.099781+0000","last_peered":"2024-03-12T04:13:46.099781+0000","last_clean":"2024-03-12T04:13:46.099781+0000","last_became_active":"2024-03-12T04:12:53.779264+0000","last_became_peered":"2024-03-12T04:12:53.779264+0000","last_unstale":"2024-03-12T04:13:46.099781+0000","last_undegraded":"2024-03-12T04:13:46.099781+0000","last_fullsized":"2024-03-12T04:13:46.099781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:57.633014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936675+0000","last_change":"2024-03-12T04:12:54.789012+0000","last_active":"2024-03-12T04:13:45.936675+0000","last_peered":"2024-03-12T04:13:45.936675+0000","last_clean":"2024-03-12T04:13:45.936675+0000","last_became_active":"2024-03-12T04:12:54.788441+0000","last_became_peered":"2024-03-12T04:12:54.788441+0000","last_unstale":"2024-03-12T04:13:45.936675+0000","last_undegraded":"2024-03-12T04:13:45.936675+0000","last_fullsized":"2024-03-12T04:13:45.936675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:55.858926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102816+0000","last_change":"2024-03-12T04:12:53.776621+0000","last_active":"2024-03-12T04:13:46.102816+0000","last_peered":"2024-03-12T04:13:46.102816+0000","last_clean":"2024-03-12T04:13:46.102816+0000","last_became_active":"2024-03-12T04:12:53.776313+0000","last_became_peered":"2024-03-12T04:12:53.776313+0000","last_unstale":"2024-03-12T04:13:46.102816+0000","last_undegraded":"2024-03-12T04:13:46.102816+0000","last_fullsized":"2024-03-12T04:13:46.102816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:43.536159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936189+0000","last_change":"2024-03-12T04:12:54.786929+0000","last_active":"2024-03-12T04:13:45.936189+0000","last_peered":"2024-03-12T04:13:45.936189+0000","last_clean":"2024-03-12T04:13:45.936189+0000","last_became_active":"2024-03-12T04:12:54.786580+0000","last_became_peered":"2024-03-12T04:12:54.786580+0000","last_unstale":"2024-03-12T04:13:45.936189+0000","last_undegraded":"2024-03-12T04:13:45.936189+0000","last_fullsized":"2024-03-12T04:13:45.936189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:58.955293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935801+0000","last_change":"2024-03-12T04:12:53.778577+0000","last_active":"2024-03-12T04:13:45.935801+0000","last_peered":"2024-03-12T04:13:45.935801+0000","last_clean":"2024-03-12T04:13:45.935801+0000","last_became_active":"2024-03-12T04:12:53.778033+0000","last_became_peered":"2024-03-12T04:12:53.778033+0000","last_unstale":"2024-03-12T04:13:45.935801+0000","last_undegraded":"2024-03-12T04:13:45.935801+0000","last_fullsized":"2024-03-12T04:13:45.935801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:02.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934684+0000","last_change":"2024-03-12T04:12:54.785861+0000","last_active":"2024-03-12T04:13:45.934684+0000","last_peered":"2024-03-12T04:13:45.934684+0000","last_clean":"2024-03-12T04:13:45.934684+0000","last_became_active":"2024-03-12T04:12:54.784682+0000","last_became_peered":"2024-03-12T04:12:54.784682+0000","last_unstale":"2024-03-12T04:13:45.934684+0000","last_undegraded":"2024-03-12T04:13:45.934684+0000","last_fullsized":"2024-03-12T04:13:45.934684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:35:12.288309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931757+0000","last_change":"2024-03-12T04:12:53.772865+0000","last_active":"2024-03-12T04:13:45.931757+0000","last_peered":"2024-03-12T04:13:45.931757+0000","last_clean":"2024-03-12T04:13:45.931757+0000","last_became_active":"2024-03-12T04:12:53.772502+0000","last_became_peered":"2024-03-12T04:12:53.772502+0000","last_unstale":"2024-03-12T04:13:45.931757+0000","last_undegraded":"2024-03-12T04:13:45.931757+0000","last_fullsized":"2024-03-12T04:13:45.931757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:40:24.112197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935763+0000","last_change":"2024-03-12T04:12:54.780722+0000","last_active":"2024-03-12T04:13:45.935763+0000","last_peered":"2024-03-12T04:13:45.935763+0000","last_clean":"2024-03-12T04:13:45.935763+0000","last_became_active":"2024-03-12T04:12:54.780404+0000","last_became_peered":"2024-03-12T04:12:54.780404+0000","last_unstale":"2024-03-12T04:13:45.935763+0000","last_undegraded":"2024-03-12T04:13:45.935763+0000","last_fullsized":"2024-03-12T04:13:45.935763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.068094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936882+0000","last_change":"2024-03-12T04:12:53.771471+0000","last_active":"2024-03-12T04:13:45.936882+0000","last_peered":"2024-03-12T04:13:45.936882+0000","last_clean":"2024-03-12T04:13:45.936882+0000","last_became_active":"2024-03-12T04:12:53.771147+0000","last_became_peered":"2024-03-12T04:12:53.771147+0000","last_unstale":"2024-03-12T04:13:45.936882+0000","last_undegraded":"2024-03-12T04:13:45.936882+0000","last_fullsized":"2024-03-12T04:13:45.936882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:46.159168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100077+0000","last_change":"2024-03-12T04:12:54.796253+0000","last_active":"2024-03-12T04:13:46.100077+0000","last_peered":"2024-03-12T04:13:46.100077+0000","last_clean":"2024-03-12T04:13:46.100077+0000","last_became_active":"2024-03-12T04:12:54.795656+0000","last_became_peered":"2024-03-12T04:12:54.795656+0000","last_unstale":"2024-03-12T04:13:46.100077+0000","last_undegraded":"2024-03-12T04:13:46.100077+0000","last_fullsized":"2024-03-12T04:13:46.100077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:39:32.788874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:08.129949+0000","last_change":"2024-03-12T04:12:53.780257+0000","last_active":"2024-03-12T04:14:08.129949+0000","last_peered":"2024-03-12T04:14:08.129949+0000","last_clean":"2024-03-12T04:14:08.129949+0000","last_became_active":"2024-03-12T04:12:53.779891+0000","last_became_peered":"2024-03-12T04:12:53.779891+0000","last_unstale":"2024-03-12T04:14:08.129949+0000","last_undegraded":"2024-03-12T04:14:08.129949+0000","last_fullsized":"2024-03-12T04:14:08.129949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:49:30.218372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100240+0000","last_change":"2024-03-12T04:12:54.792760+0000","last_active":"2024-03-12T04:13:46.100240+0000","last_peered":"2024-03-12T04:13:46.100240+0000","last_clean":"2024-03-12T04:13:46.100240+0000","last_became_active":"2024-03-12T04:12:54.792363+0000","last_became_peered":"2024-03-12T04:12:54.792363+0000","last_unstale":"2024-03-12T04:13:46.100240+0000","last_undegraded":"2024-03-12T04:13:46.100240+0000","last_fullsized":"2024-03-12T04:13:46.100240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:30:15.394318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935702+0000","last_change":"2024-03-12T04:12:53.775556+0000","last_active":"2024-03-12T04:13:45.935702+0000","last_peered":"2024-03-12T04:13:45.935702+0000","last_clean":"2024-03-12T04:13:45.935702+0000","last_became_active":"2024-03-12T04:12:53.775182+0000","last_became_peered":"2024-03-12T04:12:53.775182+0000","last_unstale":"2024-03-12T04:13:45.935702+0000","last_undegraded":"2024-03-12T04:13:45.935702+0000","last_fullsized":"2024-03-12T04:13:45.935702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:13:50.076672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100091+0000","last_change":"2024-03-12T04:12:54.795876+0000","last_active":"2024-03-12T04:13:46.100091+0000","last_peered":"2024-03-12T04:13:46.100091+0000","last_clean":"2024-03-12T04:13:46.100091+0000","last_became_active":"2024-03-12T04:12:54.793874+0000","last_became_peered":"2024-03-12T04:12:54.793874+0000","last_unstale":"2024-03-12T04:13:46.100091+0000","last_undegraded":"2024-03-12T04:13:46.100091+0000","last_fullsized":"2024-03-12T04:13:46.100091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:33:31.165974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933769+0000","last_change":"2024-03-12T04:12:53.774879+0000","last_active":"2024-03-12T04:13:45.933769+0000","last_peered":"2024-03-12T04:13:45.933769+0000","last_clean":"2024-03-12T04:13:45.933769+0000","last_became_active":"2024-03-12T04:12:53.774530+0000","last_became_peered":"2024-03-12T04:12:53.774530+0000","last_unstale":"2024-03-12T04:13:45.933769+0000","last_undegraded":"2024-03-12T04:13:45.933769+0000","last_fullsized":"2024-03-12T04:13:45.933769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:25.858165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100242+0000","last_change":"2024-03-12T04:12:54.790140+0000","last_active":"2024-03-12T04:13:46.100242+0000","last_peered":"2024-03-12T04:13:46.100242+0000","last_clean":"2024-03-12T04:13:46.100242+0000","last_became_active":"2024-03-12T04:12:54.789798+0000","last_became_peered":"2024-03-12T04:12:54.789798+0000","last_unstale":"2024-03-12T04:13:46.100242+0000","last_undegraded":"2024-03-12T04:13:46.100242+0000","last_fullsized":"2024-03-12T04:13:46.100242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:12:38.722638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934696+0000","last_change":"2024-03-12T04:12:54.785029+0000","last_active":"2024-03-12T04:13:45.934696+0000","last_peered":"2024-03-12T04:13:45.934696+0000","last_clean":"2024-03-12T04:13:45.934696+0000","last_became_active":"2024-03-12T04:12:54.784197+0000","last_became_peered":"2024-03-12T04:12:54.784197+0000","last_unstale":"2024-03-12T04:13:45.934696+0000","last_undegraded":"2024-03-12T04:13:45.934696+0000","last_fullsized":"2024-03-12T04:13:45.934696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:22:41.352093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931717+0000","last_change":"2024-03-12T04:12:54.793132+0000","last_active":"2024-03-12T04:13:45.931717+0000","last_peered":"2024-03-12T04:13:45.931717+0000","last_clean":"2024-03-12T04:13:45.931717+0000","last_became_active":"2024-03-12T04:12:54.792838+0000","last_became_peered":"2024-03-12T04:12:54.792838+0000","last_unstale":"2024-03-12T04:13:45.931717+0000","last_undegraded":"2024-03-12T04:13:45.931717+0000","last_fullsized":"2024-03-12T04:13:45.931717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:25:22.641963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.938126+0000","last_change":"2024-03-12T04:13:45.938126+0000","last_active":"2024-03-12T04:13:45.938126+0000","last_peered":"2024-03-12T04:13:45.938126+0000","last_clean":"2024-03-12T04:13:45.938126+0000","last_became_active":"2024-03-12T04:13:45.937698+0000","last_became_peered":"2024-03-12T04:13:45.937698+0000","last_unstale":"2024-03-12T04:13:45.938126+0000","last_undegraded":"2024-03-12T04:13:45.938126+0000","last_fullsized":"2024-03-12T04:13:45.938126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:10:25.378634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936928+0000","last_change":"2024-03-12T04:12:54.785830+0000","last_active":"2024-03-12T04:13:45.936928+0000","last_peered":"2024-03-12T04:13:45.936928+0000","last_clean":"2024-03-12T04:13:45.936928+0000","last_became_active":"2024-03-12T04:12:54.785087+0000","last_became_peered":"2024-03-12T04:12:54.785087+0000","last_unstale":"2024-03-12T04:13:45.936928+0000","last_undegraded":"2024-03-12T04:13:45.936928+0000","last_fullsized":"2024-03-12T04:13:45.936928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:08:09.860311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937883+0000","last_change":"2024-03-12T04:13:45.937883+0000","last_active":"2024-03-12T04:13:45.937883+0000","last_peered":"2024-03-12T04:13:45.937883+0000","last_clean":"2024-03-12T04:13:45.937883+0000","last_became_active":"2024-03-12T04:13:45.937471+0000","last_became_peered":"2024-03-12T04:13:45.937471+0000","last_unstale":"2024-03-12T04:13:45.937883+0000","last_undegraded":"2024-03-12T04:13:45.937883+0000","last_fullsized":"2024-03-12T04:13:45.937883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:28:21.997033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.938864+0000","last_change":"2024-03-12T04:13:45.938864+0000","last_active":"2024-03-12T04:13:45.938864+0000","last_peered":"2024-03-12T04:13:45.938864+0000","last_clean":"2024-03-12T04:13:45.938864+0000","last_became_active":"2024-03-12T04:13:45.938336+0000","last_became_peered":"2024-03-12T04:13:45.938336+0000","last_unstale":"2024-03-12T04:13:45.938864+0000","last_undegraded":"2024-03-12T04:13:45.938864+0000","last_fullsized":"2024-03-12T04:13:45.938864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:44.990282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932601+0000","last_change":"2024-03-12T04:12:54.796733+0000","last_active":"2024-03-12T04:13:45.932601+0000","last_peered":"2024-03-12T04:13:45.932601+0000","last_clean":"2024-03-12T04:13:45.932601+0000","last_became_active":"2024-03-12T04:12:54.796388+0000","last_became_peered":"2024-03-12T04:12:54.796388+0000","last_unstale":"2024-03-12T04:13:45.932601+0000","last_undegraded":"2024-03-12T04:13:45.932601+0000","last_fullsized":"2024-03-12T04:13:45.932601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:01.580286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935882+0000","last_change":"2024-03-12T04:12:54.787200+0000","last_active":"2024-03-12T04:13:45.935882+0000","last_peered":"2024-03-12T04:13:45.935882+0000","last_clean":"2024-03-12T04:13:45.935882+0000","last_became_active":"2024-03-12T04:12:54.786657+0000","last_became_peered":"2024-03-12T04:12:54.786657+0000","last_unstale":"2024-03-12T04:13:45.935882+0000","last_undegraded":"2024-03-12T04:13:45.935882+0000","last_fullsized":"2024-03-12T04:13:45.935882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:15:35.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934468+0000","last_change":"2024-03-12T04:12:54.785132+0000","last_active":"2024-03-12T04:13:45.934468+0000","last_peered":"2024-03-12T04:13:45.934468+0000","last_clean":"2024-03-12T04:13:45.934468+0000","last_became_active":"2024-03-12T04:12:54.782548+0000","last_became_peered":"2024-03-12T04:12:54.782548+0000","last_unstale":"2024-03-12T04:13:45.934468+0000","last_undegraded":"2024-03-12T04:13:45.934468+0000","last_fullsized":"2024-03-12T04:13:45.934468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:40:49.284778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935692+0000","last_change":"2024-03-12T04:12:54.790227+0000","last_active":"2024-03-12T04:13:45.935692+0000","last_peered":"2024-03-12T04:13:45.935692+0000","last_clean":"2024-03-12T04:13:45.935692+0000","last_became_active":"2024-03-12T04:12:54.789853+0000","last_became_peered":"2024-03-12T04:12:54.789853+0000","last_unstale":"2024-03-12T04:13:45.935692+0000","last_undegraded":"2024-03-12T04:13:45.935692+0000","last_fullsized":"2024-03-12T04:13:45.935692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:38:16.683661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102908+0000","last_change":"2024-03-12T04:12:54.796039+0000","last_active":"2024-03-12T04:13:46.102908+0000","last_peered":"2024-03-12T04:13:46.102908+0000","last_clean":"2024-03-12T04:13:46.102908+0000","last_became_active":"2024-03-12T04:12:54.795568+0000","last_became_peered":"2024-03-12T04:12:54.795568+0000","last_unstale":"2024-03-12T04:13:46.102908+0000","last_undegraded":"2024-03-12T04:13:46.102908+0000","last_fullsized":"2024-03-12T04:13:46.102908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:46.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933642+0000","last_change":"2024-03-12T04:12:54.785610+0000","last_active":"2024-03-12T04:13:45.933642+0000","last_peered":"2024-03-12T04:13:45.933642+0000","last_clean":"2024-03-12T04:13:45.933642+0000","last_became_active":"2024-03-12T04:12:54.785256+0000","last_became_peered":"2024-03-12T04:12:54.785256+0000","last_unstale":"2024-03-12T04:13:45.933642+0000","last_undegraded":"2024-03-12T04:13:45.933642+0000","last_fullsized":"2024-03-12T04:13:45.933642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:34.152361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933490+0000","last_change":"2024-03-12T04:12:54.787064+0000","last_active":"2024-03-12T04:13:45.933490+0000","last_peered":"2024-03-12T04:13:45.933490+0000","last_clean":"2024-03-12T04:13:45.933490+0000","last_became_active":"2024-03-12T04:12:54.786701+0000","last_became_peered":"2024-03-12T04:12:54.786701+0000","last_unstale":"2024-03-12T04:13:45.933490+0000","last_undegraded":"2024-03-12T04:13:45.933490+0000","last_fullsized":"2024-03-12T04:13:45.933490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:44:43.582171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250964,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65970,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":724,"num_read_kb":432,"num_write":237,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":197910,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":468048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1126400,"data_stored":1118054,"data_compressed":38179,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743464,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":467798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005101,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":468064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":1061085,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":467515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":527194,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":468854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":527737,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":467340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":467767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1060372,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60518,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61061,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60893,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T04:14:13.221 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-12T04:14:13.471 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:13.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:13.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:13 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:13.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:13 smithi042 ceph-mon[32260]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:14:13.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:13 smithi042 ceph-mon[32260]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:13.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:13.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:13 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:13.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:13 smithi119 ceph-mon[36119]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:14:13.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:13 smithi119 ceph-mon[36119]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:13.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:13.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:13 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:13.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:13 smithi136 ceph-mon[35571]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:14:13.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:13 smithi136 ceph-mon[35571]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:14.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:14:14.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:14:14.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:14 smithi042 ceph-mon[32260]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:14:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:14:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:14 smithi119 ceph-mon[36119]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:14:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:14:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:14 smithi136 ceph-mon[35571]: from='mgr.14150 172.21.15.42:0/1218989043' entity='mgr.x' 2024-03-12T04:14:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.189+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/579382764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18107020 msgr2=0x7f6f1810eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.189+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/579382764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18107020 0x7f6f1810eba0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000b0a0 tx=0x7f6f1000b3b0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/579382764 shutdown_connections 2024-03-12T04:14:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/579382764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18107020 0x7f6f1810eba0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/579382764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f18100800 0x7f6f18100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/579382764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f181011b0 0x7f6f181068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/579382764 >> 172.21.15.42:0/579382764 conn(0x7f6f18075ea0 msgr2=0x7f6f180762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/579382764 shutdown_connections 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/579382764 wait complete. 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.190+0000 7f6f1d0ea700 1 Processor -- start 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1d0ea700 1 -- start start 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1d0ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18100800 0x7f6f181981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1d0ea700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f181011b0 0x7f6f18198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1d0ea700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f18107020 0x7f6f1819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1d0ea700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1819d070 con 0x7f6f18100800 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1d0ea700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1819d1e0 con 0x7f6f181011b0 2024-03-12T04:14:15.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1d0ea700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1819d3a0 con 0x7f6f18107020 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18100800 0x7f6f181981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18100800 0x7f6f181981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36114/0 (socket says 172.21.15.42:36114) 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 -- 172.21.15.42:0/550136805 learned_addr learned my addr 172.21.15.42:0/550136805 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1659c700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f181011b0 0x7f6f18198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1759e700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f18107020 0x7f6f1819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 -- 172.21.15.42:0/550136805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f18107020 msgr2=0x7f6f1819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f18107020 0x7f6f1819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 -- 172.21.15.42:0/550136805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f181011b0 msgr2=0x7f6f18198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f181011b0 0x7f6f18198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 -- 172.21.15.42:0/550136805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10009d00 con 0x7f6f18100800 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1659c700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f181011b0 0x7f6f18198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:15.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f1759e700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f18107020 0x7f6f1819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:15.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.191+0000 7f6f16d9d700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18100800 0x7f6f181981e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c920 tx=0x7f6f0800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:15.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.192+0000 7f6efffff700 1 -- 172.21.15.42:0/550136805 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08007b00 con 0x7f6f18100800 2024-03-12T04:14:15.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.192+0000 7f6efffff700 1 -- 172.21.15.42:0/550136805 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f08004ce0 con 0x7f6f18100800 2024-03-12T04:14:15.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.192+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1819d680 con 0x7f6f18100800 2024-03-12T04:14:15.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.192+0000 7f6efffff700 1 -- 172.21.15.42:0/550136805 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080055e0 con 0x7f6f18100800 2024-03-12T04:14:15.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.192+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f181a4ee0 con 0x7f6f18100800 2024-03-12T04:14:15.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.193+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1810a260 con 0x7f6f18100800 2024-03-12T04:14:15.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.194+0000 7f6efffff700 1 -- 172.21.15.42:0/550136805 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f6f08007c60 con 0x7f6f18100800 2024-03-12T04:14:15.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.194+0000 7f6efffff700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6f000a1b50 0x7f6f000a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:15.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.194+0000 7f6f1659c700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6f000a1b50 0x7f6f000a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:15.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.194+0000 7f6efffff700 1 -- 172.21.15.42:0/550136805 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6f080cd4f0 con 0x7f6f18100800 2024-03-12T04:14:15.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.195+0000 7f6f1659c700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6f000a1b50 0x7f6f000a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c005fd0 tx=0x7f6f0c005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:15.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.198+0000 7f6efffff700 1 -- 172.21.15.42:0/550136805 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6f080965d0 con 0x7f6f18100800 2024-03-12T04:14:15.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.398+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f18111bd0 con 0x7f6f000a1b50 2024-03-12T04:14:15.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.405+0000 7f6efffff700 1 -- 172.21.15.42:0/550136805 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258975 (secure 0 0 0) 0x7f6f18111bd0 con 0x7f6f000a1b50 2024-03-12T04:14:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6f000a1b50 msgr2=0x7f6f000a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6f000a1b50 0x7f6f000a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c005fd0 tx=0x7f6f0c005db0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18100800 msgr2=0x7f6f181981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18100800 0x7f6f181981e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c920 tx=0x7f6f0800cce0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 shutdown_connections 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6f000a1b50 0x7f6f000a4010 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f18107020 0x7f6f1819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f18100800 0x7f6f181981e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 --2- 172.21.15.42:0/550136805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f181011b0 0x7f6f18198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 >> 172.21.15.42:0/550136805 conn(0x7f6f18075ea0 msgr2=0x7f6f180fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 shutdown_connections 2024-03-12T04:14:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:15.408+0000 7f6f1d0ea700 1 -- 172.21.15.42:0/550136805 wait complete. 2024-03-12T04:14:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-03-12T04:14:15.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:15 smithi042 ceph-mon[32260]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:15.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:15 smithi119 ceph-mon[36119]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:15.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:15 smithi136 ceph-mon[35571]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:16.090 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-03-12T04:14:14.919092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656572,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":910,"num_read_kb":576,"num_write":420,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373704,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568120,"statfs":{"total":1151940427776,"available":1151557754880,"internally_reserved":0,"allocated":12275712,"data_stored":7569828,"data_compressed":114537,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003357"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933854+0000","last_change":"2024-03-12T04:12:54.788665+0000","last_active":"2024-03-12T04:13:45.933854+0000","last_peered":"2024-03-12T04:13:45.933854+0000","last_clean":"2024-03-12T04:13:45.933854+0000","last_became_active":"2024-03-12T04:12:54.788334+0000","last_became_peered":"2024-03-12T04:12:54.788334+0000","last_unstale":"2024-03-12T04:13:45.933854+0000","last_undegraded":"2024-03-12T04:13:45.933854+0000","last_fullsized":"2024-03-12T04:13:45.933854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:02.442543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936756+0000","last_change":"2024-03-12T04:12:54.791942+0000","last_active":"2024-03-12T04:13:45.936756+0000","last_peered":"2024-03-12T04:13:45.936756+0000","last_clean":"2024-03-12T04:13:45.936756+0000","last_became_active":"2024-03-12T04:12:54.791579+0000","last_became_peered":"2024-03-12T04:12:54.791579+0000","last_unstale":"2024-03-12T04:13:45.936756+0000","last_undegraded":"2024-03-12T04:13:45.936756+0000","last_fullsized":"2024-03-12T04:13:45.936756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:51:08.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936570+0000","last_change":"2024-03-12T04:12:54.793238+0000","last_active":"2024-03-12T04:13:45.936570+0000","last_peered":"2024-03-12T04:13:45.936570+0000","last_clean":"2024-03-12T04:13:45.936570+0000","last_became_active":"2024-03-12T04:12:54.792877+0000","last_became_peered":"2024-03-12T04:12:54.792877+0000","last_unstale":"2024-03-12T04:13:45.936570+0000","last_undegraded":"2024-03-12T04:13:45.936570+0000","last_fullsized":"2024-03-12T04:13:45.936570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:36:46.914204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099374+0000","last_change":"2024-03-12T04:12:54.797573+0000","last_active":"2024-03-12T04:13:46.099374+0000","last_peered":"2024-03-12T04:13:46.099374+0000","last_clean":"2024-03-12T04:13:46.099374+0000","last_became_active":"2024-03-12T04:12:54.796939+0000","last_became_peered":"2024-03-12T04:12:54.796939+0000","last_unstale":"2024-03-12T04:13:46.099374+0000","last_undegraded":"2024-03-12T04:13:46.099374+0000","last_fullsized":"2024-03-12T04:13:46.099374+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:34:58.047492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936729+0000","last_change":"2024-03-12T04:12:54.780595+0000","last_active":"2024-03-12T04:13:45.936729+0000","last_peered":"2024-03-12T04:13:45.936729+0000","last_clean":"2024-03-12T04:13:45.936729+0000","last_became_active":"2024-03-12T04:12:54.780256+0000","last_became_peered":"2024-03-12T04:12:54.780256+0000","last_unstale":"2024-03-12T04:13:45.936729+0000","last_undegraded":"2024-03-12T04:13:45.936729+0000","last_fullsized":"2024-03-12T04:13:45.936729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:42.492006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934954+0000","last_change":"2024-03-12T04:13:45.934954+0000","last_active":"2024-03-12T04:13:45.934954+0000","last_peered":"2024-03-12T04:13:45.934954+0000","last_clean":"2024-03-12T04:13:45.934954+0000","last_became_active":"2024-03-12T04:13:45.934579+0000","last_became_peered":"2024-03-12T04:13:45.934579+0000","last_unstale":"2024-03-12T04:13:45.934954+0000","last_undegraded":"2024-03-12T04:13:45.934954+0000","last_fullsized":"2024-03-12T04:13:45.934954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:39:10.062304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099840+0000","last_change":"2024-03-12T04:12:54.781510+0000","last_active":"2024-03-12T04:13:46.099840+0000","last_peered":"2024-03-12T04:13:46.099840+0000","last_clean":"2024-03-12T04:13:46.099840+0000","last_became_active":"2024-03-12T04:12:54.781141+0000","last_became_peered":"2024-03-12T04:12:54.781141+0000","last_unstale":"2024-03-12T04:13:46.099840+0000","last_undegraded":"2024-03-12T04:13:46.099840+0000","last_fullsized":"2024-03-12T04:13:46.099840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:42:05.055308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937029+0000","last_change":"2024-03-12T04:12:54.782541+0000","last_active":"2024-03-12T04:13:45.937029+0000","last_peered":"2024-03-12T04:13:45.937029+0000","last_clean":"2024-03-12T04:13:45.937029+0000","last_became_active":"2024-03-12T04:12:54.781582+0000","last_became_peered":"2024-03-12T04:12:54.781582+0000","last_unstale":"2024-03-12T04:13:45.937029+0000","last_undegraded":"2024-03-12T04:13:45.937029+0000","last_fullsized":"2024-03-12T04:13:45.937029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:15.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.103255+0000","last_change":"2024-03-12T04:12:54.788201+0000","last_active":"2024-03-12T04:13:46.103255+0000","last_peered":"2024-03-12T04:13:46.103255+0000","last_clean":"2024-03-12T04:13:46.103255+0000","last_became_active":"2024-03-12T04:12:54.787875+0000","last_became_peered":"2024-03-12T04:12:54.787875+0000","last_unstale":"2024-03-12T04:13:46.103255+0000","last_undegraded":"2024-03-12T04:13:46.103255+0000","last_fullsized":"2024-03-12T04:13:46.103255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:40:39.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932924+0000","last_change":"2024-03-12T04:12:54.789748+0000","last_active":"2024-03-12T04:13:45.932924+0000","last_peered":"2024-03-12T04:13:45.932924+0000","last_clean":"2024-03-12T04:13:45.932924+0000","last_became_active":"2024-03-12T04:12:54.788838+0000","last_became_peered":"2024-03-12T04:12:54.788838+0000","last_unstale":"2024-03-12T04:13:45.932924+0000","last_undegraded":"2024-03-12T04:13:45.932924+0000","last_fullsized":"2024-03-12T04:13:45.932924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:39:51.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100115+0000","last_change":"2024-03-12T04:12:54.793101+0000","last_active":"2024-03-12T04:13:46.100115+0000","last_peered":"2024-03-12T04:13:46.100115+0000","last_clean":"2024-03-12T04:13:46.100115+0000","last_became_active":"2024-03-12T04:12:54.792488+0000","last_became_peered":"2024-03-12T04:12:54.792488+0000","last_unstale":"2024-03-12T04:13:46.100115+0000","last_undegraded":"2024-03-12T04:13:46.100115+0000","last_fullsized":"2024-03-12T04:13:46.100115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:33.862011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099333+0000","last_change":"2024-03-12T04:12:54.797953+0000","last_active":"2024-03-12T04:13:46.099333+0000","last_peered":"2024-03-12T04:13:46.099333+0000","last_clean":"2024-03-12T04:13:46.099333+0000","last_became_active":"2024-03-12T04:12:54.797413+0000","last_became_peered":"2024-03-12T04:12:54.797413+0000","last_unstale":"2024-03-12T04:13:46.099333+0000","last_undegraded":"2024-03-12T04:13:46.099333+0000","last_fullsized":"2024-03-12T04:13:46.099333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:18:01.140872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933743+0000","last_change":"2024-03-12T04:12:54.775570+0000","last_active":"2024-03-12T04:13:45.933743+0000","last_peered":"2024-03-12T04:13:45.933743+0000","last_clean":"2024-03-12T04:13:45.933743+0000","last_became_active":"2024-03-12T04:12:54.775186+0000","last_became_peered":"2024-03-12T04:12:54.775186+0000","last_unstale":"2024-03-12T04:13:45.933743+0000","last_undegraded":"2024-03-12T04:13:45.933743+0000","last_fullsized":"2024-03-12T04:13:45.933743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:26.004458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936505+0000","last_change":"2024-03-12T04:12:54.780859+0000","last_active":"2024-03-12T04:13:45.936505+0000","last_peered":"2024-03-12T04:13:45.936505+0000","last_clean":"2024-03-12T04:13:45.936505+0000","last_became_active":"2024-03-12T04:12:54.780484+0000","last_became_peered":"2024-03-12T04:12:54.780484+0000","last_unstale":"2024-03-12T04:13:45.936505+0000","last_undegraded":"2024-03-12T04:13:45.936505+0000","last_fullsized":"2024-03-12T04:13:45.936505+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:24:00.575324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936336+0000","last_change":"2024-03-12T04:12:54.796087+0000","last_active":"2024-03-12T04:13:45.936336+0000","last_peered":"2024-03-12T04:13:45.936336+0000","last_clean":"2024-03-12T04:13:45.936336+0000","last_became_active":"2024-03-12T04:12:54.793702+0000","last_became_peered":"2024-03-12T04:12:54.793702+0000","last_unstale":"2024-03-12T04:13:45.936336+0000","last_undegraded":"2024-03-12T04:13:45.936336+0000","last_fullsized":"2024-03-12T04:13:45.936336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:09:27.334791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102715+0000","last_change":"2024-03-12T04:12:54.783146+0000","last_active":"2024-03-12T04:13:46.102715+0000","last_peered":"2024-03-12T04:13:46.102715+0000","last_clean":"2024-03-12T04:13:46.102715+0000","last_became_active":"2024-03-12T04:12:54.782834+0000","last_became_peered":"2024-03-12T04:12:54.782834+0000","last_unstale":"2024-03-12T04:13:46.102715+0000","last_undegraded":"2024-03-12T04:13:46.102715+0000","last_fullsized":"2024-03-12T04:13:46.102715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:35:11.126629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934082+0000","last_change":"2024-03-12T04:12:54.794879+0000","last_active":"2024-03-12T04:13:45.934082+0000","last_peered":"2024-03-12T04:13:45.934082+0000","last_clean":"2024-03-12T04:13:45.934082+0000","last_became_active":"2024-03-12T04:12:54.794251+0000","last_became_peered":"2024-03-12T04:12:54.794251+0000","last_unstale":"2024-03-12T04:13:45.934082+0000","last_undegraded":"2024-03-12T04:13:45.934082+0000","last_fullsized":"2024-03-12T04:13:45.934082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:33.754426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930863+0000","last_change":"2024-03-12T04:12:54.791301+0000","last_active":"2024-03-12T04:13:45.930863+0000","last_peered":"2024-03-12T04:13:45.930863+0000","last_clean":"2024-03-12T04:13:45.930863+0000","last_became_active":"2024-03-12T04:12:54.790850+0000","last_became_peered":"2024-03-12T04:12:54.790850+0000","last_unstale":"2024-03-12T04:13:45.930863+0000","last_undegraded":"2024-03-12T04:13:45.930863+0000","last_fullsized":"2024-03-12T04:13:45.930863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:26:04.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932237+0000","last_change":"2024-03-12T04:12:54.793419+0000","last_active":"2024-03-12T04:13:45.932237+0000","last_peered":"2024-03-12T04:13:45.932237+0000","last_clean":"2024-03-12T04:13:45.932237+0000","last_became_active":"2024-03-12T04:12:54.793080+0000","last_became_peered":"2024-03-12T04:12:54.793080+0000","last_unstale":"2024-03-12T04:13:45.932237+0000","last_undegraded":"2024-03-12T04:13:45.932237+0000","last_fullsized":"2024-03-12T04:13:45.932237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:46:26.506899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":107,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:13.134500+0000","last_change":"2024-03-12T04:12:53.774893+0000","last_active":"2024-03-12T04:14:13.134500+0000","last_peered":"2024-03-12T04:14:13.134500+0000","last_clean":"2024-03-12T04:14:13.134500+0000","last_became_active":"2024-03-12T04:12:53.773801+0000","last_became_peered":"2024-03-12T04:12:53.773801+0000","last_unstale":"2024-03-12T04:14:13.134500+0000","last_undegraded":"2024-03-12T04:14:13.134500+0000","last_fullsized":"2024-03-12T04:14:13.134500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:30:19.995765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935920+0000","last_change":"2024-03-12T04:12:54.786795+0000","last_active":"2024-03-12T04:13:45.935920+0000","last_peered":"2024-03-12T04:13:45.935920+0000","last_clean":"2024-03-12T04:13:45.935920+0000","last_became_active":"2024-03-12T04:12:54.786481+0000","last_became_peered":"2024-03-12T04:12:54.786481+0000","last_unstale":"2024-03-12T04:13:45.935920+0000","last_undegraded":"2024-03-12T04:13:45.935920+0000","last_fullsized":"2024-03-12T04:13:45.935920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:06.849644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936174+0000","last_change":"2024-03-12T04:12:53.779582+0000","last_active":"2024-03-12T04:13:45.936174+0000","last_peered":"2024-03-12T04:13:45.936174+0000","last_clean":"2024-03-12T04:13:45.936174+0000","last_became_active":"2024-03-12T04:12:53.779062+0000","last_became_peered":"2024-03-12T04:12:53.779062+0000","last_unstale":"2024-03-12T04:13:45.936174+0000","last_undegraded":"2024-03-12T04:13:45.936174+0000","last_fullsized":"2024-03-12T04:13:45.936174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:44:01.065183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936867+0000","last_change":"2024-03-12T04:12:54.789198+0000","last_active":"2024-03-12T04:13:45.936867+0000","last_peered":"2024-03-12T04:13:45.936867+0000","last_clean":"2024-03-12T04:13:45.936867+0000","last_became_active":"2024-03-12T04:12:54.788213+0000","last_became_peered":"2024-03-12T04:12:54.788213+0000","last_unstale":"2024-03-12T04:13:45.936867+0000","last_undegraded":"2024-03-12T04:13:45.936867+0000","last_fullsized":"2024-03-12T04:13:45.936867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:24:36.561553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937326+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:45.937326+0000","last_peered":"2024-03-12T04:13:45.937326+0000","last_clean":"2024-03-12T04:13:45.937326+0000","last_became_active":"2024-03-12T04:12:53.771178+0000","last_became_peered":"2024-03-12T04:12:53.771178+0000","last_unstale":"2024-03-12T04:13:45.937326+0000","last_undegraded":"2024-03-12T04:13:45.937326+0000","last_fullsized":"2024-03-12T04:13:45.937326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:12.882128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102922+0000","last_change":"2024-03-12T04:12:54.799758+0000","last_active":"2024-03-12T04:13:46.102922+0000","last_peered":"2024-03-12T04:13:46.102922+0000","last_clean":"2024-03-12T04:13:46.102922+0000","last_became_active":"2024-03-12T04:12:54.799338+0000","last_became_peered":"2024-03-12T04:12:54.799338+0000","last_unstale":"2024-03-12T04:13:46.102922+0000","last_undegraded":"2024-03-12T04:13:46.102922+0000","last_fullsized":"2024-03-12T04:13:46.102922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:32:54.087890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937112+0000","last_change":"2024-03-12T04:12:53.774390+0000","last_active":"2024-03-12T04:13:45.937112+0000","last_peered":"2024-03-12T04:13:45.937112+0000","last_clean":"2024-03-12T04:13:45.937112+0000","last_became_active":"2024-03-12T04:12:53.774042+0000","last_became_peered":"2024-03-12T04:12:53.774042+0000","last_unstale":"2024-03-12T04:13:45.937112+0000","last_undegraded":"2024-03-12T04:13:45.937112+0000","last_fullsized":"2024-03-12T04:13:45.937112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:37.361505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.104074+0000","last_change":"2024-03-12T04:13:46.104074+0000","last_active":"2024-03-12T04:13:46.104074+0000","last_peered":"2024-03-12T04:13:46.104074+0000","last_clean":"2024-03-12T04:13:46.104074+0000","last_became_active":"2024-03-12T04:13:46.103719+0000","last_became_peered":"2024-03-12T04:13:46.103719+0000","last_unstale":"2024-03-12T04:13:46.104074+0000","last_undegraded":"2024-03-12T04:13:46.104074+0000","last_fullsized":"2024-03-12T04:13:46.104074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:29.979576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:13.136034+0000","last_change":"2024-03-12T04:12:53.772182+0000","last_active":"2024-03-12T04:14:13.136034+0000","last_peered":"2024-03-12T04:14:13.136034+0000","last_clean":"2024-03-12T04:14:13.136034+0000","last_became_active":"2024-03-12T04:12:53.771858+0000","last_became_peered":"2024-03-12T04:12:53.771858+0000","last_unstale":"2024-03-12T04:14:13.136034+0000","last_undegraded":"2024-03-12T04:14:13.136034+0000","last_fullsized":"2024-03-12T04:14:13.136034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:42:41.466311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936502+0000","last_change":"2024-03-12T04:12:54.779869+0000","last_active":"2024-03-12T04:13:45.936502+0000","last_peered":"2024-03-12T04:13:45.936502+0000","last_clean":"2024-03-12T04:13:45.936502+0000","last_became_active":"2024-03-12T04:12:54.779500+0000","last_became_peered":"2024-03-12T04:12:54.779500+0000","last_unstale":"2024-03-12T04:13:45.936502+0000","last_undegraded":"2024-03-12T04:13:45.936502+0000","last_fullsized":"2024-03-12T04:13:45.936502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:29.538570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932326+0000","last_change":"2024-03-12T04:12:53.774056+0000","last_active":"2024-03-12T04:13:45.932326+0000","last_peered":"2024-03-12T04:13:45.932326+0000","last_clean":"2024-03-12T04:13:45.932326+0000","last_became_active":"2024-03-12T04:12:53.773675+0000","last_became_peered":"2024-03-12T04:12:53.773675+0000","last_unstale":"2024-03-12T04:13:45.932326+0000","last_undegraded":"2024-03-12T04:13:45.932326+0000","last_fullsized":"2024-03-12T04:13:45.932326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:33.576947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935472+0000","last_change":"2024-03-12T04:12:54.790583+0000","last_active":"2024-03-12T04:13:45.935472+0000","last_peered":"2024-03-12T04:13:45.935472+0000","last_clean":"2024-03-12T04:13:45.935472+0000","last_became_active":"2024-03-12T04:12:54.788364+0000","last_became_peered":"2024-03-12T04:12:54.788364+0000","last_unstale":"2024-03-12T04:13:45.935472+0000","last_undegraded":"2024-03-12T04:13:45.935472+0000","last_fullsized":"2024-03-12T04:13:45.935472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:05:13.542279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933518+0000","last_change":"2024-03-12T04:12:53.769619+0000","last_active":"2024-03-12T04:13:45.933518+0000","last_peered":"2024-03-12T04:13:45.933518+0000","last_clean":"2024-03-12T04:13:45.933518+0000","last_became_active":"2024-03-12T04:12:53.769293+0000","last_became_peered":"2024-03-12T04:12:53.769293+0000","last_unstale":"2024-03-12T04:13:45.933518+0000","last_undegraded":"2024-03-12T04:13:45.933518+0000","last_fullsized":"2024-03-12T04:13:45.933518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:44.618499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935485+0000","last_change":"2024-03-12T04:12:54.776493+0000","last_active":"2024-03-12T04:13:45.935485+0000","last_peered":"2024-03-12T04:13:45.935485+0000","last_clean":"2024-03-12T04:13:45.935485+0000","last_became_active":"2024-03-12T04:12:54.775967+0000","last_became_peered":"2024-03-12T04:12:54.775967+0000","last_unstale":"2024-03-12T04:13:45.935485+0000","last_undegraded":"2024-03-12T04:13:45.935485+0000","last_fullsized":"2024-03-12T04:13:45.935485+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:19:46.824145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934388+0000","last_change":"2024-03-12T04:12:53.773523+0000","last_active":"2024-03-12T04:13:45.934388+0000","last_peered":"2024-03-12T04:13:45.934388+0000","last_clean":"2024-03-12T04:13:45.934388+0000","last_became_active":"2024-03-12T04:12:53.773161+0000","last_became_peered":"2024-03-12T04:12:53.773161+0000","last_unstale":"2024-03-12T04:13:45.934388+0000","last_undegraded":"2024-03-12T04:13:45.934388+0000","last_fullsized":"2024-03-12T04:13:45.934388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:41:40.810994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930799+0000","last_change":"2024-03-12T04:12:54.791688+0000","last_active":"2024-03-12T04:13:45.930799+0000","last_peered":"2024-03-12T04:13:45.930799+0000","last_clean":"2024-03-12T04:13:45.930799+0000","last_became_active":"2024-03-12T04:12:54.790854+0000","last_became_peered":"2024-03-12T04:12:54.790854+0000","last_unstale":"2024-03-12T04:13:45.930799+0000","last_undegraded":"2024-03-12T04:13:45.930799+0000","last_fullsized":"2024-03-12T04:13:45.930799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:42:50.343524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:13.135649+0000","last_change":"2024-03-12T04:12:53.777534+0000","last_active":"2024-03-12T04:14:13.135649+0000","last_peered":"2024-03-12T04:14:13.135649+0000","last_clean":"2024-03-12T04:14:13.135649+0000","last_became_active":"2024-03-12T04:12:53.777201+0000","last_became_peered":"2024-03-12T04:12:53.777201+0000","last_unstale":"2024-03-12T04:14:13.135649+0000","last_undegraded":"2024-03-12T04:14:13.135649+0000","last_fullsized":"2024-03-12T04:14:13.135649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:54.708892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.103229+0000","last_change":"2024-03-12T04:12:54.798962+0000","last_active":"2024-03-12T04:13:46.103229+0000","last_peered":"2024-03-12T04:13:46.103229+0000","last_clean":"2024-03-12T04:13:46.103229+0000","last_became_active":"2024-03-12T04:12:54.798650+0000","last_became_peered":"2024-03-12T04:12:54.798650+0000","last_unstale":"2024-03-12T04:13:46.103229+0000","last_undegraded":"2024-03-12T04:13:46.103229+0000","last_fullsized":"2024-03-12T04:13:46.103229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:05:29.270242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100119+0000","last_change":"2024-03-12T04:12:53.777791+0000","last_active":"2024-03-12T04:13:46.100119+0000","last_peered":"2024-03-12T04:13:46.100119+0000","last_clean":"2024-03-12T04:13:46.100119+0000","last_became_active":"2024-03-12T04:12:53.777430+0000","last_became_peered":"2024-03-12T04:12:53.777430+0000","last_unstale":"2024-03-12T04:13:46.100119+0000","last_undegraded":"2024-03-12T04:13:46.100119+0000","last_fullsized":"2024-03-12T04:13:46.100119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:38:21.858857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936587+0000","last_change":"2024-03-12T04:12:54.784710+0000","last_active":"2024-03-12T04:13:45.936587+0000","last_peered":"2024-03-12T04:13:45.936587+0000","last_clean":"2024-03-12T04:13:45.936587+0000","last_became_active":"2024-03-12T04:12:54.784342+0000","last_became_peered":"2024-03-12T04:12:54.784342+0000","last_unstale":"2024-03-12T04:13:45.936587+0000","last_undegraded":"2024-03-12T04:13:45.936587+0000","last_fullsized":"2024-03-12T04:13:45.936587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:21.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935890+0000","last_change":"2024-03-12T04:12:53.779289+0000","last_active":"2024-03-12T04:13:45.935890+0000","last_peered":"2024-03-12T04:13:45.935890+0000","last_clean":"2024-03-12T04:13:45.935890+0000","last_became_active":"2024-03-12T04:12:53.778894+0000","last_became_peered":"2024-03-12T04:12:53.778894+0000","last_unstale":"2024-03-12T04:13:45.935890+0000","last_undegraded":"2024-03-12T04:13:45.935890+0000","last_fullsized":"2024-03-12T04:13:45.935890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:13:33.328977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.103025+0000","last_change":"2024-03-12T04:12:54.794102+0000","last_active":"2024-03-12T04:13:46.103025+0000","last_peered":"2024-03-12T04:13:46.103025+0000","last_clean":"2024-03-12T04:13:46.103025+0000","last_became_active":"2024-03-12T04:12:54.793789+0000","last_became_peered":"2024-03-12T04:12:54.793789+0000","last_unstale":"2024-03-12T04:13:46.103025+0000","last_undegraded":"2024-03-12T04:13:46.103025+0000","last_fullsized":"2024-03-12T04:13:46.103025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:55:33.632297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":204,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:13.150429+0000","last_change":"2024-03-12T04:12:53.774847+0000","last_active":"2024-03-12T04:14:13.150429+0000","last_peered":"2024-03-12T04:14:13.150429+0000","last_clean":"2024-03-12T04:14:13.150429+0000","last_became_active":"2024-03-12T04:12:53.774457+0000","last_became_peered":"2024-03-12T04:12:53.774457+0000","last_unstale":"2024-03-12T04:14:13.150429+0000","last_undegraded":"2024-03-12T04:14:13.150429+0000","last_fullsized":"2024-03-12T04:14:13.150429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:17.350636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58766,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":155,"num_read_kb":134,"num_write":32,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937153+0000","last_change":"2024-03-12T04:12:54.783671+0000","last_active":"2024-03-12T04:13:45.937153+0000","last_peered":"2024-03-12T04:13:45.937153+0000","last_clean":"2024-03-12T04:13:45.937153+0000","last_became_active":"2024-03-12T04:12:54.782919+0000","last_became_peered":"2024-03-12T04:12:54.782919+0000","last_unstale":"2024-03-12T04:13:45.937153+0000","last_undegraded":"2024-03-12T04:13:45.937153+0000","last_fullsized":"2024-03-12T04:13:45.937153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:38:19.445096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.948316+0000","last_change":"2024-03-12T04:13:45.947951+0000","last_active":"2024-03-12T04:13:45.948316+0000","last_peered":"2024-03-12T04:13:45.948316+0000","last_clean":"2024-03-12T04:13:45.948316+0000","last_became_active":"2024-03-12T04:13:45.938099+0000","last_became_peered":"2024-03-12T04:13:45.938099+0000","last_unstale":"2024-03-12T04:13:45.948316+0000","last_undegraded":"2024-03-12T04:13:45.948316+0000","last_fullsized":"2024-03-12T04:13:45.948316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:03.905076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933675+0000","last_change":"2024-03-12T04:12:54.779319+0000","last_active":"2024-03-12T04:13:45.933675+0000","last_peered":"2024-03-12T04:13:45.933675+0000","last_clean":"2024-03-12T04:13:45.933675+0000","last_became_active":"2024-03-12T04:12:54.778927+0000","last_became_peered":"2024-03-12T04:12:54.778927+0000","last_unstale":"2024-03-12T04:13:45.933675+0000","last_undegraded":"2024-03-12T04:13:45.933675+0000","last_fullsized":"2024-03-12T04:13:45.933675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:20:53.947257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934971+0000","last_change":"2024-03-12T04:12:53.774754+0000","last_active":"2024-03-12T04:13:45.934971+0000","last_peered":"2024-03-12T04:13:45.934971+0000","last_clean":"2024-03-12T04:13:45.934971+0000","last_became_active":"2024-03-12T04:12:53.774395+0000","last_became_peered":"2024-03-12T04:12:53.774395+0000","last_unstale":"2024-03-12T04:13:45.934971+0000","last_undegraded":"2024-03-12T04:13:45.934971+0000","last_fullsized":"2024-03-12T04:13:45.934971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:13.286204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099966+0000","last_change":"2024-03-12T04:12:54.786891+0000","last_active":"2024-03-12T04:13:46.099966+0000","last_peered":"2024-03-12T04:13:46.099966+0000","last_clean":"2024-03-12T04:13:46.099966+0000","last_became_active":"2024-03-12T04:12:54.786591+0000","last_became_peered":"2024-03-12T04:12:54.786591+0000","last_unstale":"2024-03-12T04:13:46.099966+0000","last_undegraded":"2024-03-12T04:13:46.099966+0000","last_fullsized":"2024-03-12T04:13:46.099966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:16:09.188557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:13.135425+0000","last_change":"2024-03-12T04:12:53.769839+0000","last_active":"2024-03-12T04:14:13.135425+0000","last_peered":"2024-03-12T04:14:13.135425+0000","last_clean":"2024-03-12T04:14:13.135425+0000","last_became_active":"2024-03-12T04:12:53.769486+0000","last_became_peered":"2024-03-12T04:12:53.769486+0000","last_unstale":"2024-03-12T04:14:13.135425+0000","last_undegraded":"2024-03-12T04:14:13.135425+0000","last_fullsized":"2024-03-12T04:14:13.135425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:46.050018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.098955+0000","last_change":"2024-03-12T04:12:54.782447+0000","last_active":"2024-03-12T04:13:46.098955+0000","last_peered":"2024-03-12T04:13:46.098955+0000","last_clean":"2024-03-12T04:13:46.098955+0000","last_became_active":"2024-03-12T04:12:54.782091+0000","last_became_peered":"2024-03-12T04:12:54.782091+0000","last_unstale":"2024-03-12T04:13:46.098955+0000","last_undegraded":"2024-03-12T04:13:46.098955+0000","last_fullsized":"2024-03-12T04:13:46.098955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:22.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.959032+0000","last_change":"2024-03-12T04:13:45.958615+0000","last_active":"2024-03-12T04:13:45.959032+0000","last_peered":"2024-03-12T04:13:45.959032+0000","last_clean":"2024-03-12T04:13:45.959032+0000","last_became_active":"2024-03-12T04:13:45.938341+0000","last_became_peered":"2024-03-12T04:13:45.938341+0000","last_unstale":"2024-03-12T04:13:45.959032+0000","last_undegraded":"2024-03-12T04:13:45.959032+0000","last_fullsized":"2024-03-12T04:13:45.959032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:17:44.688344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935203+0000","last_change":"2024-03-12T04:12:54.784282+0000","last_active":"2024-03-12T04:13:45.935203+0000","last_peered":"2024-03-12T04:13:45.935203+0000","last_clean":"2024-03-12T04:13:45.935203+0000","last_became_active":"2024-03-12T04:12:54.783953+0000","last_became_peered":"2024-03-12T04:12:54.783953+0000","last_unstale":"2024-03-12T04:13:45.935203+0000","last_undegraded":"2024-03-12T04:13:45.935203+0000","last_fullsized":"2024-03-12T04:13:45.935203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:14:02.182546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937257+0000","last_change":"2024-03-12T04:12:53.770230+0000","last_active":"2024-03-12T04:13:45.937257+0000","last_peered":"2024-03-12T04:13:45.937257+0000","last_clean":"2024-03-12T04:13:45.937257+0000","last_became_active":"2024-03-12T04:12:53.768817+0000","last_became_peered":"2024-03-12T04:12:53.768817+0000","last_unstale":"2024-03-12T04:13:45.937257+0000","last_undegraded":"2024-03-12T04:13:45.937257+0000","last_fullsized":"2024-03-12T04:13:45.937257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:09:12.656489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933705+0000","last_change":"2024-03-12T04:12:54.785654+0000","last_active":"2024-03-12T04:13:45.933705+0000","last_peered":"2024-03-12T04:13:45.933705+0000","last_clean":"2024-03-12T04:13:45.933705+0000","last_became_active":"2024-03-12T04:12:54.785244+0000","last_became_peered":"2024-03-12T04:12:54.785244+0000","last_unstale":"2024-03-12T04:13:45.933705+0000","last_undegraded":"2024-03-12T04:13:45.933705+0000","last_fullsized":"2024-03-12T04:13:45.933705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:00:41.961637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930880+0000","last_change":"2024-03-12T04:12:53.780962+0000","last_active":"2024-03-12T04:13:45.930880+0000","last_peered":"2024-03-12T04:13:45.930880+0000","last_clean":"2024-03-12T04:13:45.930880+0000","last_became_active":"2024-03-12T04:12:53.780623+0000","last_became_peered":"2024-03-12T04:12:53.780623+0000","last_unstale":"2024-03-12T04:13:45.930880+0000","last_undegraded":"2024-03-12T04:13:45.930880+0000","last_fullsized":"2024-03-12T04:13:45.930880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932748+0000","last_change":"2024-03-12T04:12:54.775065+0000","last_active":"2024-03-12T04:13:45.932748+0000","last_peered":"2024-03-12T04:13:45.932748+0000","last_clean":"2024-03-12T04:13:45.932748+0000","last_became_active":"2024-03-12T04:12:54.774686+0000","last_became_peered":"2024-03-12T04:12:54.774686+0000","last_unstale":"2024-03-12T04:13:45.932748+0000","last_undegraded":"2024-03-12T04:13:45.932748+0000","last_fullsized":"2024-03-12T04:13:45.932748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:56:25.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:10.131459+0000","last_change":"2024-03-12T04:12:53.777702+0000","last_active":"2024-03-12T04:14:10.131459+0000","last_peered":"2024-03-12T04:14:10.131459+0000","last_clean":"2024-03-12T04:14:10.131459+0000","last_became_active":"2024-03-12T04:12:53.777346+0000","last_became_peered":"2024-03-12T04:12:53.777346+0000","last_unstale":"2024-03-12T04:14:10.131459+0000","last_undegraded":"2024-03-12T04:14:10.131459+0000","last_fullsized":"2024-03-12T04:14:10.131459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:09.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932521+0000","last_change":"2024-03-12T04:12:54.789087+0000","last_active":"2024-03-12T04:13:45.932521+0000","last_peered":"2024-03-12T04:13:45.932521+0000","last_clean":"2024-03-12T04:13:45.932521+0000","last_became_active":"2024-03-12T04:12:54.788735+0000","last_became_peered":"2024-03-12T04:12:54.788735+0000","last_unstale":"2024-03-12T04:13:45.932521+0000","last_undegraded":"2024-03-12T04:13:45.932521+0000","last_fullsized":"2024-03-12T04:13:45.932521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:14.947137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934523+0000","last_change":"2024-03-12T04:12:54.789974+0000","last_active":"2024-03-12T04:13:45.934523+0000","last_peered":"2024-03-12T04:13:45.934523+0000","last_clean":"2024-03-12T04:13:45.934523+0000","last_became_active":"2024-03-12T04:12:54.789578+0000","last_became_peered":"2024-03-12T04:12:54.789578+0000","last_unstale":"2024-03-12T04:13:45.934523+0000","last_undegraded":"2024-03-12T04:13:45.934523+0000","last_fullsized":"2024-03-12T04:13:45.934523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:22:25.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:10.130804+0000","last_change":"2024-03-12T04:12:53.773601+0000","last_active":"2024-03-12T04:14:10.130804+0000","last_peered":"2024-03-12T04:14:10.130804+0000","last_clean":"2024-03-12T04:14:10.130804+0000","last_became_active":"2024-03-12T04:12:53.773264+0000","last_became_peered":"2024-03-12T04:12:53.773264+0000","last_unstale":"2024-03-12T04:14:10.130804+0000","last_undegraded":"2024-03-12T04:14:10.130804+0000","last_fullsized":"2024-03-12T04:14:10.130804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:07:03.580157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937019+0000","last_change":"2024-03-12T04:12:54.782496+0000","last_active":"2024-03-12T04:13:45.937019+0000","last_peered":"2024-03-12T04:13:45.937019+0000","last_clean":"2024-03-12T04:13:45.937019+0000","last_became_active":"2024-03-12T04:12:54.782014+0000","last_became_peered":"2024-03-12T04:12:54.782014+0000","last_unstale":"2024-03-12T04:13:45.937019+0000","last_undegraded":"2024-03-12T04:13:45.937019+0000","last_fullsized":"2024-03-12T04:13:45.937019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:08.580655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099322+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:46.099322+0000","last_peered":"2024-03-12T04:13:46.099322+0000","last_clean":"2024-03-12T04:13:46.099322+0000","last_became_active":"2024-03-12T04:12:53.770855+0000","last_became_peered":"2024-03-12T04:12:53.770855+0000","last_unstale":"2024-03-12T04:13:46.099322+0000","last_undegraded":"2024-03-12T04:13:46.099322+0000","last_fullsized":"2024-03-12T04:13:46.099322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:32:33.969240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100244+0000","last_change":"2024-03-12T04:11:37.642780+0000","last_active":"2024-03-12T04:13:46.100244+0000","last_peered":"2024-03-12T04:13:46.100244+0000","last_clean":"2024-03-12T04:13:46.100244+0000","last_became_active":"2024-03-12T04:11:37.642145+0000","last_became_peered":"2024-03-12T04:11:37.642145+0000","last_unstale":"2024-03-12T04:13:46.100244+0000","last_undegraded":"2024-03-12T04:13:46.100244+0000","last_fullsized":"2024-03-12T04:13:46.100244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_clean_scrub_stamp":"2024-03-12T04:08:09.604393+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:46:36.666483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933359+0000","last_change":"2024-03-12T04:12:54.790464+0000","last_active":"2024-03-12T04:13:45.933359+0000","last_peered":"2024-03-12T04:13:45.933359+0000","last_clean":"2024-03-12T04:13:45.933359+0000","last_became_active":"2024-03-12T04:12:54.788440+0000","last_became_peered":"2024-03-12T04:12:54.788440+0000","last_unstale":"2024-03-12T04:13:45.933359+0000","last_undegraded":"2024-03-12T04:13:45.933359+0000","last_fullsized":"2024-03-12T04:13:45.933359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:54.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100091+0000","last_change":"2024-03-12T04:12:53.773334+0000","last_active":"2024-03-12T04:13:46.100091+0000","last_peered":"2024-03-12T04:13:46.100091+0000","last_clean":"2024-03-12T04:13:46.100091+0000","last_became_active":"2024-03-12T04:12:53.772759+0000","last_became_peered":"2024-03-12T04:12:53.772759+0000","last_unstale":"2024-03-12T04:13:46.100091+0000","last_undegraded":"2024-03-12T04:13:46.100091+0000","last_fullsized":"2024-03-12T04:13:46.100091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:03:17.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937169+0000","last_change":"2024-03-12T04:12:53.777035+0000","last_active":"2024-03-12T04:13:45.937169+0000","last_peered":"2024-03-12T04:13:45.937169+0000","last_clean":"2024-03-12T04:13:45.937169+0000","last_became_active":"2024-03-12T04:12:53.776587+0000","last_became_peered":"2024-03-12T04:12:53.776587+0000","last_unstale":"2024-03-12T04:13:45.937169+0000","last_undegraded":"2024-03-12T04:13:45.937169+0000","last_fullsized":"2024-03-12T04:13:45.937169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:21.390525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933689+0000","last_change":"2024-03-12T04:12:54.790835+0000","last_active":"2024-03-12T04:13:45.933689+0000","last_peered":"2024-03-12T04:13:45.933689+0000","last_clean":"2024-03-12T04:13:45.933689+0000","last_became_active":"2024-03-12T04:12:54.790512+0000","last_became_peered":"2024-03-12T04:12:54.790512+0000","last_unstale":"2024-03-12T04:13:45.933689+0000","last_undegraded":"2024-03-12T04:13:45.933689+0000","last_fullsized":"2024-03-12T04:13:45.933689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:37:58.029450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931022+0000","last_change":"2024-03-12T04:12:53.774604+0000","last_active":"2024-03-12T04:13:45.931022+0000","last_peered":"2024-03-12T04:13:45.931022+0000","last_clean":"2024-03-12T04:13:45.931022+0000","last_became_active":"2024-03-12T04:12:53.774270+0000","last_became_peered":"2024-03-12T04:12:53.774270+0000","last_unstale":"2024-03-12T04:13:45.931022+0000","last_undegraded":"2024-03-12T04:13:45.931022+0000","last_fullsized":"2024-03-12T04:13:45.931022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:23.635179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935988+0000","last_change":"2024-03-12T04:12:54.786259+0000","last_active":"2024-03-12T04:13:45.935988+0000","last_peered":"2024-03-12T04:13:45.935988+0000","last_clean":"2024-03-12T04:13:45.935988+0000","last_became_active":"2024-03-12T04:12:54.783114+0000","last_became_peered":"2024-03-12T04:12:54.783114+0000","last_unstale":"2024-03-12T04:13:45.935988+0000","last_undegraded":"2024-03-12T04:13:45.935988+0000","last_fullsized":"2024-03-12T04:13:45.935988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:53.036505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099781+0000","last_change":"2024-03-12T04:12:53.779618+0000","last_active":"2024-03-12T04:13:46.099781+0000","last_peered":"2024-03-12T04:13:46.099781+0000","last_clean":"2024-03-12T04:13:46.099781+0000","last_became_active":"2024-03-12T04:12:53.779264+0000","last_became_peered":"2024-03-12T04:12:53.779264+0000","last_unstale":"2024-03-12T04:13:46.099781+0000","last_undegraded":"2024-03-12T04:13:46.099781+0000","last_fullsized":"2024-03-12T04:13:46.099781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:57.633014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936675+0000","last_change":"2024-03-12T04:12:54.789012+0000","last_active":"2024-03-12T04:13:45.936675+0000","last_peered":"2024-03-12T04:13:45.936675+0000","last_clean":"2024-03-12T04:13:45.936675+0000","last_became_active":"2024-03-12T04:12:54.788441+0000","last_became_peered":"2024-03-12T04:12:54.788441+0000","last_unstale":"2024-03-12T04:13:45.936675+0000","last_undegraded":"2024-03-12T04:13:45.936675+0000","last_fullsized":"2024-03-12T04:13:45.936675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:55.858926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102816+0000","last_change":"2024-03-12T04:12:53.776621+0000","last_active":"2024-03-12T04:13:46.102816+0000","last_peered":"2024-03-12T04:13:46.102816+0000","last_clean":"2024-03-12T04:13:46.102816+0000","last_became_active":"2024-03-12T04:12:53.776313+0000","last_became_peered":"2024-03-12T04:12:53.776313+0000","last_unstale":"2024-03-12T04:13:46.102816+0000","last_undegraded":"2024-03-12T04:13:46.102816+0000","last_fullsized":"2024-03-12T04:13:46.102816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:43.536159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936189+0000","last_change":"2024-03-12T04:12:54.786929+0000","last_active":"2024-03-12T04:13:45.936189+0000","last_peered":"2024-03-12T04:13:45.936189+0000","last_clean":"2024-03-12T04:13:45.936189+0000","last_became_active":"2024-03-12T04:12:54.786580+0000","last_became_peered":"2024-03-12T04:12:54.786580+0000","last_unstale":"2024-03-12T04:13:45.936189+0000","last_undegraded":"2024-03-12T04:13:45.936189+0000","last_fullsized":"2024-03-12T04:13:45.936189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:58.955293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935801+0000","last_change":"2024-03-12T04:12:53.778577+0000","last_active":"2024-03-12T04:13:45.935801+0000","last_peered":"2024-03-12T04:13:45.935801+0000","last_clean":"2024-03-12T04:13:45.935801+0000","last_became_active":"2024-03-12T04:12:53.778033+0000","last_became_peered":"2024-03-12T04:12:53.778033+0000","last_unstale":"2024-03-12T04:13:45.935801+0000","last_undegraded":"2024-03-12T04:13:45.935801+0000","last_fullsized":"2024-03-12T04:13:45.935801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:02.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934684+0000","last_change":"2024-03-12T04:12:54.785861+0000","last_active":"2024-03-12T04:13:45.934684+0000","last_peered":"2024-03-12T04:13:45.934684+0000","last_clean":"2024-03-12T04:13:45.934684+0000","last_became_active":"2024-03-12T04:12:54.784682+0000","last_became_peered":"2024-03-12T04:12:54.784682+0000","last_unstale":"2024-03-12T04:13:45.934684+0000","last_undegraded":"2024-03-12T04:13:45.934684+0000","last_fullsized":"2024-03-12T04:13:45.934684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:35:12.288309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931757+0000","last_change":"2024-03-12T04:12:53.772865+0000","last_active":"2024-03-12T04:13:45.931757+0000","last_peered":"2024-03-12T04:13:45.931757+0000","last_clean":"2024-03-12T04:13:45.931757+0000","last_became_active":"2024-03-12T04:12:53.772502+0000","last_became_peered":"2024-03-12T04:12:53.772502+0000","last_unstale":"2024-03-12T04:13:45.931757+0000","last_undegraded":"2024-03-12T04:13:45.931757+0000","last_fullsized":"2024-03-12T04:13:45.931757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:40:24.112197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935763+0000","last_change":"2024-03-12T04:12:54.780722+0000","last_active":"2024-03-12T04:13:45.935763+0000","last_peered":"2024-03-12T04:13:45.935763+0000","last_clean":"2024-03-12T04:13:45.935763+0000","last_became_active":"2024-03-12T04:12:54.780404+0000","last_became_peered":"2024-03-12T04:12:54.780404+0000","last_unstale":"2024-03-12T04:13:45.935763+0000","last_undegraded":"2024-03-12T04:13:45.935763+0000","last_fullsized":"2024-03-12T04:13:45.935763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.068094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936882+0000","last_change":"2024-03-12T04:12:53.771471+0000","last_active":"2024-03-12T04:13:45.936882+0000","last_peered":"2024-03-12T04:13:45.936882+0000","last_clean":"2024-03-12T04:13:45.936882+0000","last_became_active":"2024-03-12T04:12:53.771147+0000","last_became_peered":"2024-03-12T04:12:53.771147+0000","last_unstale":"2024-03-12T04:13:45.936882+0000","last_undegraded":"2024-03-12T04:13:45.936882+0000","last_fullsized":"2024-03-12T04:13:45.936882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:46.159168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100077+0000","last_change":"2024-03-12T04:12:54.796253+0000","last_active":"2024-03-12T04:13:46.100077+0000","last_peered":"2024-03-12T04:13:46.100077+0000","last_clean":"2024-03-12T04:13:46.100077+0000","last_became_active":"2024-03-12T04:12:54.795656+0000","last_became_peered":"2024-03-12T04:12:54.795656+0000","last_unstale":"2024-03-12T04:13:46.100077+0000","last_undegraded":"2024-03-12T04:13:46.100077+0000","last_fullsized":"2024-03-12T04:13:46.100077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:39:32.788874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":101,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:13.136343+0000","last_change":"2024-03-12T04:12:53.780257+0000","last_active":"2024-03-12T04:14:13.136343+0000","last_peered":"2024-03-12T04:14:13.136343+0000","last_clean":"2024-03-12T04:14:13.136343+0000","last_became_active":"2024-03-12T04:12:53.779891+0000","last_became_peered":"2024-03-12T04:12:53.779891+0000","last_unstale":"2024-03-12T04:14:13.136343+0000","last_undegraded":"2024-03-12T04:14:13.136343+0000","last_fullsized":"2024-03-12T04:14:13.136343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:49:30.218372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100240+0000","last_change":"2024-03-12T04:12:54.792760+0000","last_active":"2024-03-12T04:13:46.100240+0000","last_peered":"2024-03-12T04:13:46.100240+0000","last_clean":"2024-03-12T04:13:46.100240+0000","last_became_active":"2024-03-12T04:12:54.792363+0000","last_became_peered":"2024-03-12T04:12:54.792363+0000","last_unstale":"2024-03-12T04:13:46.100240+0000","last_undegraded":"2024-03-12T04:13:46.100240+0000","last_fullsized":"2024-03-12T04:13:46.100240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:30:15.394318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935702+0000","last_change":"2024-03-12T04:12:53.775556+0000","last_active":"2024-03-12T04:13:45.935702+0000","last_peered":"2024-03-12T04:13:45.935702+0000","last_clean":"2024-03-12T04:13:45.935702+0000","last_became_active":"2024-03-12T04:12:53.775182+0000","last_became_peered":"2024-03-12T04:12:53.775182+0000","last_unstale":"2024-03-12T04:13:45.935702+0000","last_undegraded":"2024-03-12T04:13:45.935702+0000","last_fullsized":"2024-03-12T04:13:45.935702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:13:50.076672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100091+0000","last_change":"2024-03-12T04:12:54.795876+0000","last_active":"2024-03-12T04:13:46.100091+0000","last_peered":"2024-03-12T04:13:46.100091+0000","last_clean":"2024-03-12T04:13:46.100091+0000","last_became_active":"2024-03-12T04:12:54.793874+0000","last_became_peered":"2024-03-12T04:12:54.793874+0000","last_unstale":"2024-03-12T04:13:46.100091+0000","last_undegraded":"2024-03-12T04:13:46.100091+0000","last_fullsized":"2024-03-12T04:13:46.100091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:33:31.165974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933769+0000","last_change":"2024-03-12T04:12:53.774879+0000","last_active":"2024-03-12T04:13:45.933769+0000","last_peered":"2024-03-12T04:13:45.933769+0000","last_clean":"2024-03-12T04:13:45.933769+0000","last_became_active":"2024-03-12T04:12:53.774530+0000","last_became_peered":"2024-03-12T04:12:53.774530+0000","last_unstale":"2024-03-12T04:13:45.933769+0000","last_undegraded":"2024-03-12T04:13:45.933769+0000","last_fullsized":"2024-03-12T04:13:45.933769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:25.858165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100242+0000","last_change":"2024-03-12T04:12:54.790140+0000","last_active":"2024-03-12T04:13:46.100242+0000","last_peered":"2024-03-12T04:13:46.100242+0000","last_clean":"2024-03-12T04:13:46.100242+0000","last_became_active":"2024-03-12T04:12:54.789798+0000","last_became_peered":"2024-03-12T04:12:54.789798+0000","last_unstale":"2024-03-12T04:13:46.100242+0000","last_undegraded":"2024-03-12T04:13:46.100242+0000","last_fullsized":"2024-03-12T04:13:46.100242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:12:38.722638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934696+0000","last_change":"2024-03-12T04:12:54.785029+0000","last_active":"2024-03-12T04:13:45.934696+0000","last_peered":"2024-03-12T04:13:45.934696+0000","last_clean":"2024-03-12T04:13:45.934696+0000","last_became_active":"2024-03-12T04:12:54.784197+0000","last_became_peered":"2024-03-12T04:12:54.784197+0000","last_unstale":"2024-03-12T04:13:45.934696+0000","last_undegraded":"2024-03-12T04:13:45.934696+0000","last_fullsized":"2024-03-12T04:13:45.934696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:22:41.352093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931717+0000","last_change":"2024-03-12T04:12:54.793132+0000","last_active":"2024-03-12T04:13:45.931717+0000","last_peered":"2024-03-12T04:13:45.931717+0000","last_clean":"2024-03-12T04:13:45.931717+0000","last_became_active":"2024-03-12T04:12:54.792838+0000","last_became_peered":"2024-03-12T04:12:54.792838+0000","last_unstale":"2024-03-12T04:13:45.931717+0000","last_undegraded":"2024-03-12T04:13:45.931717+0000","last_fullsized":"2024-03-12T04:13:45.931717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:25:22.641963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.938126+0000","last_change":"2024-03-12T04:13:45.938126+0000","last_active":"2024-03-12T04:13:45.938126+0000","last_peered":"2024-03-12T04:13:45.938126+0000","last_clean":"2024-03-12T04:13:45.938126+0000","last_became_active":"2024-03-12T04:13:45.937698+0000","last_became_peered":"2024-03-12T04:13:45.937698+0000","last_unstale":"2024-03-12T04:13:45.938126+0000","last_undegraded":"2024-03-12T04:13:45.938126+0000","last_fullsized":"2024-03-12T04:13:45.938126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:10:25.378634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936928+0000","last_change":"2024-03-12T04:12:54.785830+0000","last_active":"2024-03-12T04:13:45.936928+0000","last_peered":"2024-03-12T04:13:45.936928+0000","last_clean":"2024-03-12T04:13:45.936928+0000","last_became_active":"2024-03-12T04:12:54.785087+0000","last_became_peered":"2024-03-12T04:12:54.785087+0000","last_unstale":"2024-03-12T04:13:45.936928+0000","last_undegraded":"2024-03-12T04:13:45.936928+0000","last_fullsized":"2024-03-12T04:13:45.936928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:08:09.860311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937883+0000","last_change":"2024-03-12T04:13:45.937883+0000","last_active":"2024-03-12T04:13:45.937883+0000","last_peered":"2024-03-12T04:13:45.937883+0000","last_clean":"2024-03-12T04:13:45.937883+0000","last_became_active":"2024-03-12T04:13:45.937471+0000","last_became_peered":"2024-03-12T04:13:45.937471+0000","last_unstale":"2024-03-12T04:13:45.937883+0000","last_undegraded":"2024-03-12T04:13:45.937883+0000","last_fullsized":"2024-03-12T04:13:45.937883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:28:21.997033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.938864+0000","last_change":"2024-03-12T04:13:45.938864+0000","last_active":"2024-03-12T04:13:45.938864+0000","last_peered":"2024-03-12T04:13:45.938864+0000","last_clean":"2024-03-12T04:13:45.938864+0000","last_became_active":"2024-03-12T04:13:45.938336+0000","last_became_peered":"2024-03-12T04:13:45.938336+0000","last_unstale":"2024-03-12T04:13:45.938864+0000","last_undegraded":"2024-03-12T04:13:45.938864+0000","last_fullsized":"2024-03-12T04:13:45.938864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:44.990282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932601+0000","last_change":"2024-03-12T04:12:54.796733+0000","last_active":"2024-03-12T04:13:45.932601+0000","last_peered":"2024-03-12T04:13:45.932601+0000","last_clean":"2024-03-12T04:13:45.932601+0000","last_became_active":"2024-03-12T04:12:54.796388+0000","last_became_peered":"2024-03-12T04:12:54.796388+0000","last_unstale":"2024-03-12T04:13:45.932601+0000","last_undegraded":"2024-03-12T04:13:45.932601+0000","last_fullsized":"2024-03-12T04:13:45.932601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:01.580286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935882+0000","last_change":"2024-03-12T04:12:54.787200+0000","last_active":"2024-03-12T04:13:45.935882+0000","last_peered":"2024-03-12T04:13:45.935882+0000","last_clean":"2024-03-12T04:13:45.935882+0000","last_became_active":"2024-03-12T04:12:54.786657+0000","last_became_peered":"2024-03-12T04:12:54.786657+0000","last_unstale":"2024-03-12T04:13:45.935882+0000","last_undegraded":"2024-03-12T04:13:45.935882+0000","last_fullsized":"2024-03-12T04:13:45.935882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:15:35.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934468+0000","last_change":"2024-03-12T04:12:54.785132+0000","last_active":"2024-03-12T04:13:45.934468+0000","last_peered":"2024-03-12T04:13:45.934468+0000","last_clean":"2024-03-12T04:13:45.934468+0000","last_became_active":"2024-03-12T04:12:54.782548+0000","last_became_peered":"2024-03-12T04:12:54.782548+0000","last_unstale":"2024-03-12T04:13:45.934468+0000","last_undegraded":"2024-03-12T04:13:45.934468+0000","last_fullsized":"2024-03-12T04:13:45.934468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:40:49.284778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935692+0000","last_change":"2024-03-12T04:12:54.790227+0000","last_active":"2024-03-12T04:13:45.935692+0000","last_peered":"2024-03-12T04:13:45.935692+0000","last_clean":"2024-03-12T04:13:45.935692+0000","last_became_active":"2024-03-12T04:12:54.789853+0000","last_became_peered":"2024-03-12T04:12:54.789853+0000","last_unstale":"2024-03-12T04:13:45.935692+0000","last_undegraded":"2024-03-12T04:13:45.935692+0000","last_fullsized":"2024-03-12T04:13:45.935692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:38:16.683661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102908+0000","last_change":"2024-03-12T04:12:54.796039+0000","last_active":"2024-03-12T04:13:46.102908+0000","last_peered":"2024-03-12T04:13:46.102908+0000","last_clean":"2024-03-12T04:13:46.102908+0000","last_became_active":"2024-03-12T04:12:54.795568+0000","last_became_peered":"2024-03-12T04:12:54.795568+0000","last_unstale":"2024-03-12T04:13:46.102908+0000","last_undegraded":"2024-03-12T04:13:46.102908+0000","last_fullsized":"2024-03-12T04:13:46.102908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:46.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933642+0000","last_change":"2024-03-12T04:12:54.785610+0000","last_active":"2024-03-12T04:13:45.933642+0000","last_peered":"2024-03-12T04:13:45.933642+0000","last_clean":"2024-03-12T04:13:45.933642+0000","last_became_active":"2024-03-12T04:12:54.785256+0000","last_became_peered":"2024-03-12T04:12:54.785256+0000","last_unstale":"2024-03-12T04:13:45.933642+0000","last_undegraded":"2024-03-12T04:13:45.933642+0000","last_fullsized":"2024-03-12T04:13:45.933642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:34.152361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933490+0000","last_change":"2024-03-12T04:12:54.787064+0000","last_active":"2024-03-12T04:13:45.933490+0000","last_peered":"2024-03-12T04:13:45.933490+0000","last_clean":"2024-03-12T04:13:45.933490+0000","last_became_active":"2024-03-12T04:12:54.786701+0000","last_became_peered":"2024-03-12T04:12:54.786701+0000","last_unstale":"2024-03-12T04:13:45.933490+0000","last_undegraded":"2024-03-12T04:13:45.933490+0000","last_fullsized":"2024-03-12T04:13:45.933490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:44:43.582171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250964,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65970,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":774,"num_read_kb":457,"num_write":237,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":197910,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":468048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1126400,"data_stored":1118054,"data_compressed":38179,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743465,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":467798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005102,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":468064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":1061085,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":467515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":527194,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":468854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":527737,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":467340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":467767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1060372,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60518,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61061,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60893,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T04:14:16.095 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-12T04:14:16.095 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-12T04:14:16.095 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-12T04:14:16.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph health --format=json 2024-03-12T04:14:16.338 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:16.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:16 smithi042 ceph-mon[32260]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:14:16.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:16 smithi119 ceph-mon[36119]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:14:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:16 smithi136 ceph-mon[35571]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:14:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:17 smithi042 ceph-mon[32260]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-12T04:14:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:17 smithi042 ceph-mon[32260]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-12T04:14:17.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:17 smithi119 ceph-mon[36119]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-12T04:14:17.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:17 smithi119 ceph-mon[36119]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-12T04:14:17.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:17 smithi136 ceph-mon[35571]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-12T04:14:17.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:17 smithi136 ceph-mon[35571]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-12T04:14:18.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 -- 172.21.15.42:0/3303630125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780107cc0 msgr2=0x7f57801080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:18.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 --2- 172.21.15.42:0/3303630125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780107cc0 0x7f57801080a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f57700099c0 tx=0x7f5770009cd0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 -- 172.21.15.42:0/3303630125 shutdown_connections 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 --2- 172.21.15.42:0/3303630125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f578006a2c0 0x7f578010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 --2- 172.21.15.42:0/3303630125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780107cc0 0x7f57801080a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 --2- 172.21.15.42:0/3303630125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780069920 0x7f5780069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 -- 172.21.15.42:0/3303630125 >> 172.21.15.42:0/3303630125 conn(0x7f5780077aa0 msgr2=0x7f5780077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 -- 172.21.15.42:0/3303630125 shutdown_connections 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.168+0000 7f5786a2d700 1 -- 172.21.15.42:0/3303630125 wait complete. 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 Processor -- start 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 -- start start 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780069920 0x7f57801980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578006a2c0 0x7f57801985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5780107cc0 0x7f578019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019cf80 con 0x7f5780069920 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019d0f0 con 0x7f578006a2c0 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f5786a2d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019d2b0 con 0x7f5780107cc0 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780069920 0x7f57801980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780069920 0x7f57801980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36126/0 (socket says 172.21.15.42:36126) 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 -- 172.21.15.42:0/2721784490 learned_addr learned my addr 172.21.15.42:0/2721784490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 -- 172.21.15.42:0/2721784490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5780107cc0 msgr2=0x7f578019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5780107cc0 0x7f578019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 -- 172.21.15.42:0/2721784490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578006a2c0 msgr2=0x7f57801985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578006a2c0 0x7f57801985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 -- 172.21.15.42:0/2721784490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700095f0 con 0x7f5780069920 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577ffff700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780069920 0x7f57801980a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f577000b580 tx=0x7f57700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.169+0000 7f577dffb700 1 -- 172.21.15.42:0/2721784490 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577001d070 con 0x7f5780069920 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.170+0000 7f577dffb700 1 -- 172.21.15.42:0/2721784490 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f577000fe40 con 0x7f5780069920 2024-03-12T04:14:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.170+0000 7f577dffb700 1 -- 172.21.15.42:0/2721784490 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770003790 con 0x7f5780069920 2024-03-12T04:14:18.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.170+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578019d530 con 0x7f5780069920 2024-03-12T04:14:18.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.170+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57801a4df0 con 0x7f5780069920 2024-03-12T04:14:18.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.171+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578010a1e0 con 0x7f5780069920 2024-03-12T04:14:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.172+0000 7f577dffb700 1 -- 172.21.15.42:0/2721784490 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5770022470 con 0x7f5780069920 2024-03-12T04:14:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.172+0000 7f577dffb700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f576c0a1ac0 0x7f576c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.172+0000 7f5777fff700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f576c0a1ac0 0x7f576c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.172+0000 7f577dffb700 1 -- 172.21.15.42:0/2721784490 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f57700ce710 con 0x7f5780069920 2024-03-12T04:14:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.173+0000 7f5777fff700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f576c0a1ac0 0x7f576c0a3f80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5768005fd0 tx=0x7f5768005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:18.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.177+0000 7f577dffb700 1 -- 172.21.15.42:0/2721784490 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f577005dd20 con 0x7f5780069920 2024-03-12T04:14:18.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.438+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5780103370 con 0x7f5780069920 2024-03-12T04:14:18.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.439+0000 7f577dffb700 1 -- 172.21.15.42:0/2721784490 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5770027040 con 0x7f5780069920 2024-03-12T04:14:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:18.442 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-12T04:14:18.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f576c0a1ac0 msgr2=0x7f576c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:18.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f576c0a1ac0 0x7f576c0a3f80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5768005fd0 tx=0x7f5768005d00 comp rx=0 tx=0).stop 2024-03-12T04:14:18.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780069920 msgr2=0x7f57801980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780069920 0x7f57801980a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f577000b580 tx=0x7f57700047c0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 shutdown_connections 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f576c0a1ac0 0x7f576c0a3f80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5780107cc0 0x7f578019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5780069920 0x7f57801980a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 --2- 172.21.15.42:0/2721784490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578006a2c0 0x7f57801985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 >> 172.21.15.42:0/2721784490 conn(0x7f5780077aa0 msgr2=0x7f57800ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 shutdown_connections 2024-03-12T04:14:18.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:18.441+0000 7f5786a2d700 1 -- 172.21.15.42:0/2721784490 wait complete. 2024-03-12T04:14:18.806 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:18 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2721784490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T04:14:18.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:18 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2721784490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T04:14:18.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:18 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2721784490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T04:14:19.107 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-12T04:14:19.108 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.215+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/1316830852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 msgr2=0x7f5bf4101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.215+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/1316830852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4101730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009900 tx=0x7f5be402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.215+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/1316830852 shutdown_connections 2024-03-12T04:14:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.215+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/1316830852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bf4101c70 0x7f5bf4108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.215+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/1316830852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4101730 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.215+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/1316830852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bf41000c0 0x7f5bf41004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.215+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/1316830852 >> 172.21.15.42:0/1316830852 conn(0x7f5bf40fb630 msgr2=0x7f5bf40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/1316830852 shutdown_connections 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/1316830852 wait complete. 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 Processor -- start 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 -- start start 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bf41000c0 0x7f5bf41970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bf4101c70 0x7f5bf419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf419ebc0 con 0x7f5bf41012b0 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf419ed30 con 0x7f5bf4101c70 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bfb6ae640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf419eef0 con 0x7f5bf41000c0 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bf8c49640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bf8c49640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36140/0 (socket says 172.21.15.42:36140) 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.216+0000 7f5bf8c49640 1 -- 172.21.15.42:0/3486108153 learned_addr learned my addr 172.21.15.42:0/3486108153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf944a640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bf41000c0 0x7f5bf41970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf9c4b640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bf4101c70 0x7f5bf419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf8c49640 1 -- 172.21.15.42:0/3486108153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bf41000c0 msgr2=0x7f5bf41970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf8c49640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bf41000c0 0x7f5bf41970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf8c49640 1 -- 172.21.15.42:0/3486108153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bf4101c70 msgr2=0x7f5bf419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf8c49640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bf4101c70 0x7f5bf419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf8c49640 1 -- 172.21.15.42:0/3486108153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4009560 con 0x7f5bf41012b0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf944a640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bf41000c0 0x7f5bf41970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bf8c49640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4197610 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5be4036040 tx=0x7f5be40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bea7fc640 1 -- 172.21.15.42:0/3486108153 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40044e0 con 0x7f5bf41012b0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bea7fc640 1 -- 172.21.15.42:0/3486108153 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5be402de90 con 0x7f5bf41012b0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bea7fc640 1 -- 172.21.15.42:0/3486108153 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4005470 con 0x7f5bf41012b0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf419f170 con 0x7f5bf41012b0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.217+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf419f680 con 0x7f5bf41012b0 2024-03-12T04:14:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.218+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf410b980 con 0x7f5bf41012b0 2024-03-12T04:14:19.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.219+0000 7f5bea7fc640 1 -- 172.21.15.42:0/3486108153 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5be4003800 con 0x7f5bf41012b0 2024-03-12T04:14:19.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.220+0000 7f5bea7fc640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5be00a1a30 0x7f5be00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.220+0000 7f5bf944a640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5be00a1a30 0x7f5be00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.220+0000 7f5bea7fc640 1 -- 172.21.15.42:0/3486108153 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5be40ef760 con 0x7f5bf41012b0 2024-03-12T04:14:19.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.220+0000 7f5bf944a640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5be00a1a30 0x7f5be00a3ef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4101110 tx=0x7f5bf000a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:19.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.224+0000 7f5bea7fc640 1 -- 172.21.15.42:0/3486108153 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5be40b8940 con 0x7f5bf41012b0 2024-03-12T04:14:19.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.408+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bf410bb10 con 0x7f5bf41012b0 2024-03-12T04:14:19.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.410+0000 7f5bea7fc640 1 -- 172.21.15.42:0/3486108153 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f5be40bc000 con 0x7f5bf41012b0 2024-03-12T04:14:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5be00a1a30 msgr2=0x7f5be00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5be00a1a30 0x7f5be00a3ef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4101110 tx=0x7f5bf000a5e0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 msgr2=0x7f5bf4197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4197610 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5be4036040 tx=0x7f5be40042f0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 shutdown_connections 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f5be00a1a30 0x7f5be00a3ef0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bf4101c70 0x7f5bf419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf41012b0 0x7f5bf4197610 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 --2- 172.21.15.42:0/3486108153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bf41000c0 0x7f5bf41970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 >> 172.21.15.42:0/3486108153 conn(0x7f5bf40fb630 msgr2=0x7f5bf40fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 shutdown_connections 2024-03-12T04:14:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.412+0000 7f5bfb6ae640 1 -- 172.21.15.42:0/3486108153 wait complete. 2024-03-12T04:14:19.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:14:19.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- 172.21.15.42:0/558583875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 msgr2=0x7f93381092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 --2- 172.21.15.42:0/558583875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f93381092d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9330009ca0 tx=0x7f933002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- 172.21.15.42:0/558583875 shutdown_connections 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 --2- 172.21.15.42:0/558583875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f93381092d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 --2- 172.21.15.42:0/558583875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9338104550 0x7f9338106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 --2- 172.21.15.42:0/558583875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9338101c20 0x7f9338104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- 172.21.15.42:0/558583875 >> 172.21.15.42:0/558583875 conn(0x7f93380fb630 msgr2=0x7f93380fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- 172.21.15.42:0/558583875 shutdown_connections 2024-03-12T04:14:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- 172.21.15.42:0/558583875 wait complete. 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 Processor -- start 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- start start 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9338101c20 0x7f93381971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9338104550 0x7f9338197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f933819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933819ece0 con 0x7f9338106e80 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933819ee50 con 0x7f9338104550 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.530+0000 7f933ed56640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933819f010 con 0x7f9338101c20 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f933819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f933819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36152/0 (socket says 172.21.15.42:36152) 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 -- 172.21.15.42:0/2762247975 learned_addr learned my addr 172.21.15.42:0/2762247975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933caf2640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9338101c20 0x7f93381971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 -- 172.21.15.42:0/2762247975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9338101c20 msgr2=0x7f93381971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9338101c20 0x7f93381971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 -- 172.21.15.42:0/2762247975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9338104550 msgr2=0x7f9338197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f9337fff640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9338104550 0x7f9338197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9338104550 0x7f9338197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 -- 172.21.15.42:0/2762247975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330009900 con 0x7f9338106e80 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f9337fff640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9338104550 0x7f9338197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933d2f3640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f933819e7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9330037040 tx=0x7f9330032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f9335ffb640 1 -- 172.21.15.42:0/2762247975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330034e70 con 0x7f9338106e80 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f9335ffb640 1 -- 172.21.15.42:0/2762247975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9330003e20 con 0x7f9338106e80 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f9335ffb640 1 -- 172.21.15.42:0/2762247975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933003d3f0 con 0x7f9338106e80 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933819f290 con 0x7f9338106e80 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.531+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933819f810 con 0x7f9338106e80 2024-03-12T04:14:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.532+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93380fd120 con 0x7f9338106e80 2024-03-12T04:14:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.533+0000 7f9335ffb640 1 -- 172.21.15.42:0/2762247975 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f933003b040 con 0x7f9338106e80 2024-03-12T04:14:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.534+0000 7f9335ffb640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f93280a1ab0 0x7f93280a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.534+0000 7f933caf2640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f93280a1ab0 0x7f93280a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.534+0000 7f9335ffb640 1 -- 172.21.15.42:0/2762247975 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93300efbb0 con 0x7f9338106e80 2024-03-12T04:14:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.534+0000 7f933caf2640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f93280a1ab0 0x7f93280a3f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f932c005fd0 tx=0x7f932c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.538+0000 7f9335ffb640 1 -- 172.21.15.42:0/2762247975 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f93300b8df0 con 0x7f9338106e80 2024-03-12T04:14:19.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.719+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f933802cda0 con 0x7f9338106e80 2024-03-12T04:14:19.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.720+0000 7f9335ffb640 1 -- 172.21.15.42:0/2762247975 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93300bc4b0 con 0x7f9338106e80 2024-03-12T04:14:19.722 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f93280a1ab0 msgr2=0x7f93280a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f93280a1ab0 0x7f93280a3f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f932c005fd0 tx=0x7f932c005850 comp rx=0 tx=0).stop 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 msgr2=0x7f933819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f933819e7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9330037040 tx=0x7f9330032980 comp rx=0 tx=0).stop 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 shutdown_connections 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9338106e80 0x7f933819e7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f93280a1ab0 0x7f93280a3f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9338104550 0x7f9338197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 --2- 172.21.15.42:0/2762247975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9338101c20 0x7f93381971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 >> 172.21.15.42:0/2762247975 conn(0x7f93380fb630 msgr2=0x7f9338102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 shutdown_connections 2024-03-12T04:14:19.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.722+0000 7f933ed56640 1 -- 172.21.15.42:0/2762247975 wait complete. 2024-03-12T04:14:19.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:19 smithi042 ceph-mon[32260]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:14:19.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:19 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3486108153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:19.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 -- 172.21.15.42:0/1202532618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101290 msgr2=0x7f3458101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 --2- 172.21.15.42:0/1202532618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101290 0x7f3458101710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3440009900 tx=0x7f344002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 -- 172.21.15.42:0/1202532618 shutdown_connections 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 --2- 172.21.15.42:0/1202532618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3458101c50 0x7f34581084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 --2- 172.21.15.42:0/1202532618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101290 0x7f3458101710 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 --2- 172.21.15.42:0/1202532618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34581000a0 0x7f34581004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 -- 172.21.15.42:0/1202532618 >> 172.21.15.42:0/1202532618 conn(0x7f34580fb630 msgr2=0x7f34580fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 -- 172.21.15.42:0/1202532618 shutdown_connections 2024-03-12T04:14:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 -- 172.21.15.42:0/1202532618 wait complete. 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 Processor -- start 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 -- start start 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.839+0000 7f345d809640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581000a0 0x7f345806a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345d809640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3458101290 0x7f345819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345d809640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101c50 0x7f345819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345d809640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345819ecb0 con 0x7f3458101c50 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345d809640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345819ee20 con 0x7f3458101290 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345d809640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345819efe0 con 0x7f34581000a0 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101c50 0x7f345819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101c50 0x7f345819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36170/0 (socket says 172.21.15.42:36170) 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 -- 172.21.15.42:0/1541621808 learned_addr learned my addr 172.21.15.42:0/1541621808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f3456ffd640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581000a0 0x7f345806a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 -- 172.21.15.42:0/1541621808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581000a0 msgr2=0x7f345806a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581000a0 0x7f345806a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34567fc640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3458101290 0x7f345819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 -- 172.21.15.42:0/1541621808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3458101290 msgr2=0x7f345819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3458101290 0x7f345819beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 -- 172.21.15.42:0/1541621808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3440009560 con 0x7f3458101c50 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f3456ffd640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581000a0 0x7f345806a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f34577fe640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101c50 0x7f345819e4d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f345000d750 tx=0x7f345000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345c807640 1 -- 172.21.15.42:0/1541621808 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450009bd0 con 0x7f3458101c50 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345819f260 con 0x7f3458101c50 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345c807640 1 -- 172.21.15.42:0/1541621808 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f345000de60 con 0x7f3458101c50 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345c807640 1 -- 172.21.15.42:0/1541621808 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450004e00 con 0x7f3458101c50 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.840+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345819f770 con 0x7f3458101c50 2024-03-12T04:14:19.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.841+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3458107480 con 0x7f3458101c50 2024-03-12T04:14:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.842+0000 7f345c807640 1 -- 172.21.15.42:0/1541621808 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3450004f60 con 0x7f3458101c50 2024-03-12T04:14:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.843+0000 7f345c807640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34440a1a30 0x7f34440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.843+0000 7f3456ffd640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34440a1a30 0x7f34440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.843+0000 7f345c807640 1 -- 172.21.15.42:0/1541621808 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f34500cd290 con 0x7f3458101c50 2024-03-12T04:14:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.843+0000 7f3456ffd640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34440a1a30 0x7f34440a3ef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f34581010f0 tx=0x7f344800a800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:19.846+0000 7f345c807640 1 -- 172.21.15.42:0/1541621808 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3450096470 con 0x7f3458101c50 2024-03-12T04:14:19.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:19 smithi119 ceph-mon[36119]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:14:19.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:19 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3486108153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:19.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:19 smithi136 ceph-mon[35571]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-12T04:14:19.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:19 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3486108153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:20.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.027+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f345810b960 con 0x7f3458101c50 2024-03-12T04:14:20.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.027+0000 7f345c807640 1 -- 172.21.15.42:0/1541621808 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3450020020 con 0x7f3458101c50 2024-03-12T04:14:20.030 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34440a1a30 msgr2=0x7f34440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34440a1a30 0x7f34440a3ef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f34581010f0 tx=0x7f344800a800 comp rx=0 tx=0).stop 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101c50 msgr2=0x7f345819e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101c50 0x7f345819e4d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f345000d750 tx=0x7f345000dc10 comp rx=0 tx=0).stop 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 shutdown_connections 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3458101c50 0x7f345819e4d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f34440a1a30 0x7f34440a3ef0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3458101290 0x7f345819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 --2- 172.21.15.42:0/1541621808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581000a0 0x7f345806a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 >> 172.21.15.42:0/1541621808 conn(0x7f34580fb630 msgr2=0x7f3458104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:20.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 shutdown_connections 2024-03-12T04:14:20.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.029+0000 7f345d809640 1 -- 172.21.15.42:0/1541621808 wait complete. 2024-03-12T04:14:20.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 -- 172.21.15.42:0/2649660825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04106e60 msgr2=0x7fae041092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 --2- 172.21.15.42:0/2649660825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04106e60 0x7fae041092b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009ca0 tx=0x7fadfc02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 -- 172.21.15.42:0/2649660825 shutdown_connections 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 --2- 172.21.15.42:0/2649660825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04106e60 0x7fae041092b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 --2- 172.21.15.42:0/2649660825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae04104530 0x7fae04106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 --2- 172.21.15.42:0/2649660825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04101c00 0x7fae04103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 -- 172.21.15.42:0/2649660825 >> 172.21.15.42:0/2649660825 conn(0x7fae040fb610 msgr2=0x7fae040fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 -- 172.21.15.42:0/2649660825 shutdown_connections 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.147+0000 7fae0b64e640 1 -- 172.21.15.42:0/2649660825 wait complete. 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 Processor -- start 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 -- start start 2024-03-12T04:14:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae04101c00 0x7fae04197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04104530 0x7fae04197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04106e60 0x7fae0419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419ec30 con 0x7fae04104530 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419eda0 con 0x7fae04101c00 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae0b64e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419ef60 con 0x7fae04106e60 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae08be9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04104530 0x7fae04197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae08be9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04104530 0x7fae04197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36198/0 (socket says 172.21.15.42:36198) 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae08be9640 1 -- 172.21.15.42:0/102226069 learned_addr learned my addr 172.21.15.42:0/102226069 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae093ea640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae04101c00 0x7fae04197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae09beb640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04106e60 0x7fae0419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae08be9640 1 -- 172.21.15.42:0/102226069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04106e60 msgr2=0x7fae0419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae08be9640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04106e60 0x7fae0419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae08be9640 1 -- 172.21.15.42:0/102226069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae04101c00 msgr2=0x7fae04197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.148+0000 7fae08be9640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae04101c00 0x7fae04197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fae08be9640 1 -- 172.21.15.42:0/102226069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc009900 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fae093ea640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae04101c00 0x7fae04197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fae09beb640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04106e60 0x7fae0419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fae08be9640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04104530 0x7fae04197680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fadf400e9c0 tx=0x7fadf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fadfa7fc640 1 -- 172.21.15.42:0/102226069 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4009700 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0419f240 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fadfa7fc640 1 -- 172.21.15.42:0/102226069 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fadf400cd80 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fadfa7fc640 1 -- 172.21.15.42:0/102226069 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4010430 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.149+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0419f7a0 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.150+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae040fd100 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.151+0000 7fadfa7fc640 1 -- 172.21.15.42:0/102226069 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fadf4004f30 con 0x7fae04104530 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.151+0000 7fadfa7fc640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.152+0000 7fae093ea640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.152+0000 7fadfa7fc640 1 -- 172.21.15.42:0/102226069 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fadf4014070 con 0x7fae04104530 2024-03-12T04:14:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.152+0000 7fae093ea640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fae00009b50 tx=0x7fae00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.156+0000 7fadfa7fc640 1 -- 172.21.15.42:0/102226069 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadf4096ad0 con 0x7fae04104530 2024-03-12T04:14:20.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.338+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae0410ca40 con 0x7fae04104530 2024-03-12T04:14:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.338+0000 7fadfa7fc640 1 -- 172.21.15.42:0/102226069 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadf409a190 con 0x7fae04104530 2024-03-12T04:14:20.341 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:14:20.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 msgr2=0x7fadf00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fae00009b50 tx=0x7fae00005f90 comp rx=0 tx=0).stop 2024-03-12T04:14:20.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04104530 msgr2=0x7fae04197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04104530 0x7fae04197680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fadf400e9c0 tx=0x7fadf400ee80 comp rx=0 tx=0).stop 2024-03-12T04:14:20.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 shutdown_connections 2024-03-12T04:14:20.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04106e60 0x7fae0419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae04104530 0x7fae04197680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 --2- 172.21.15.42:0/102226069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae04101c00 0x7fae04197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.340+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 >> 172.21.15.42:0/102226069 conn(0x7fae040fb610 msgr2=0x7fae04102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.341+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 shutdown_connections 2024-03-12T04:14:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.341+0000 7fae0b64e640 1 -- 172.21.15.42:0/102226069 wait complete. 2024-03-12T04:14:20.354 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 -- 172.21.15.42:0/2409239261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb680102ea0 msgr2=0x7fb680109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 --2- 172.21.15.42:0/2409239261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb680102ea0 0x7fb680109730 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb678009ca0 tx=0x7fb67802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 -- 172.21.15.42:0/2409239261 shutdown_connections 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 --2- 172.21.15.42:0/2409239261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb680102ea0 0x7fb680109730 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 --2- 172.21.15.42:0/2409239261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6801024e0 0x7fb680102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 --2- 172.21.15.42:0/2409239261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6800ff470 0x7fb6800ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 -- 172.21.15.42:0/2409239261 >> 172.21.15.42:0/2409239261 conn(0x7fb6800fada0 msgr2=0x7fb6800fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.454+0000 7fb6866eb640 1 -- 172.21.15.42:0/2409239261 shutdown_connections 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 -- 172.21.15.42:0/2409239261 wait complete. 2024-03-12T04:14:20.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 Processor -- start 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 -- start start 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6800ff470 0x7fb6801970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6801024e0 0x7fb680197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb680102ea0 0x7fb68019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68019ebe0 con 0x7fb6800ff470 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68019ed50 con 0x7fb6801024e0 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb6866eb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68019ef10 con 0x7fb680102ea0 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6800ff470 0x7fb6801970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6800ff470 0x7fb6801970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36222/0 (socket says 172.21.15.42:36222) 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.455+0000 7fb67ffff640 1 -- 172.21.15.42:0/3214300973 learned_addr learned my addr 172.21.15.42:0/3214300973 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:20.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67f7fe640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6801024e0 0x7fb680197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb684c88640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb680102ea0 0x7fb68019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67ffff640 1 -- 172.21.15.42:0/3214300973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb680102ea0 msgr2=0x7fb68019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67ffff640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb680102ea0 0x7fb68019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67ffff640 1 -- 172.21.15.42:0/3214300973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6801024e0 msgr2=0x7fb680197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67ffff640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6801024e0 0x7fb680197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67ffff640 1 -- 172.21.15.42:0/3214300973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678009900 con 0x7fb6800ff470 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67f7fe640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6801024e0 0x7fb680197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb684c88640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb680102ea0 0x7fb68019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67ffff640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6800ff470 0x7fb6801970f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb67000b450 tx=0x7fb67000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67d7fa640 1 -- 172.21.15.42:0/3214300973 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670011890 con 0x7fb6800ff470 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68019f1f0 con 0x7fb6800ff470 2024-03-12T04:14:20.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67d7fa640 1 -- 172.21.15.42:0/3214300973 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb6700103e0 con 0x7fb6800ff470 2024-03-12T04:14:20.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb67d7fa640 1 -- 172.21.15.42:0/3214300973 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670011ea0 con 0x7fb6800ff470 2024-03-12T04:14:20.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.456+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68019f730 con 0x7fb6800ff470 2024-03-12T04:14:20.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.457+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6801085b0 con 0x7fb6800ff470 2024-03-12T04:14:20.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.458+0000 7fb67d7fa640 1 -- 172.21.15.42:0/3214300973 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb670010550 con 0x7fb6800ff470 2024-03-12T04:14:20.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.459+0000 7fb67d7fa640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb66c0a1a30 0x7fb66c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.459+0000 7fb67f7fe640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb66c0a1a30 0x7fb66c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.459+0000 7fb67d7fa640 1 -- 172.21.15.42:0/3214300973 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6700cd240 con 0x7fb6800ff470 2024-03-12T04:14:20.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.459+0000 7fb67f7fe640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb66c0a1a30 0x7fb66c0a3ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb680198600 tx=0x7fb66800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:20.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.463+0000 7fb67d7fa640 1 -- 172.21.15.42:0/3214300973 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb670096480 con 0x7fb6800ff470 2024-03-12T04:14:20.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.684+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb68010cae0 con 0x7fb6800ff470 2024-03-12T04:14:20.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.685+0000 7fb67d7fa640 1 -- 172.21.15.42:0/3214300973 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fb68010cae0 con 0x7fb6800ff470 2024-03-12T04:14:20.687 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:13:01.029979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:14:20.688 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-03-12T04:14:20.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb66c0a1a30 msgr2=0x7fb66c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb66c0a1a30 0x7fb66c0a3ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb680198600 tx=0x7fb66800a5c0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6800ff470 msgr2=0x7fb6801970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6800ff470 0x7fb6801970f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb67000b450 tx=0x7fb67000b910 comp rx=0 tx=0).stop 2024-03-12T04:14:20.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 shutdown_connections 2024-03-12T04:14:20.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb680102ea0 0x7fb68019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fb66c0a1a30 0x7fb66c0a3ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6801024e0 0x7fb680197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 --2- 172.21.15.42:0/3214300973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6800ff470 0x7fb6801970f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 >> 172.21.15.42:0/3214300973 conn(0x7fb6800fada0 msgr2=0x7fb680105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:20.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 shutdown_connections 2024-03-12T04:14:20.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.687+0000 7fb6866eb640 1 -- 172.21.15.42:0/3214300973 wait complete. 2024-03-12T04:14:20.699 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:20 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2762247975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.699 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:20 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1541621808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.699 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:20 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/102226069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 -- 172.21.15.42:0/3642799178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb8102390 msgr2=0x7f9bb8102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/3642799178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb8102390 0x7f9bb8102810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009900 tx=0x7f9bac02bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 -- 172.21.15.42:0/3642799178 shutdown_connections 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/3642799178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bb8102d50 0x7f9bb81095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/3642799178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb8102390 0x7f9bb8102810 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/3642799178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb81011a0 0x7f9bb81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 -- 172.21.15.42:0/3642799178 >> 172.21.15.42:0/3642799178 conn(0x7f9bb80fc730 msgr2=0x7f9bb80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 -- 172.21.15.42:0/3642799178 shutdown_connections 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 -- 172.21.15.42:0/3642799178 wait complete. 2024-03-12T04:14:20.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.801+0000 7f9bbe10c640 1 Processor -- start 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 -- start start 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb81011a0 0x7f9bb8197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb8102390 0x7f9bb8197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bb8102d50 0x7f9bb819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819ec40 con 0x7f9bb81011a0 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819edb0 con 0x7f9bb8102390 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819ef70 con 0x7f9bb8102d50 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb81011a0 0x7f9bb8197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb81011a0 0x7f9bb8197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36236/0 (socket says 172.21.15.42:36236) 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 -- 172.21.15.42:0/1062590033 learned_addr learned my addr 172.21.15.42:0/1062590033 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb7fff640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bb8102d50 0x7f9bb819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 -- 172.21.15.42:0/1062590033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bb8102d50 msgr2=0x7f9bb819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bb8102d50 0x7f9bb819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 -- 172.21.15.42:0/1062590033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb8102390 msgr2=0x7f9bb8197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb8102390 0x7f9bb8197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 -- 172.21.15.42:0/1062590033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac009560 con 0x7f9bb81011a0 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb7fff640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bb8102d50 0x7f9bb819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb77fe640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb81011a0 0x7f9bb8197150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800ccb0 tx=0x7f9ba800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb4ff9640 1 -- 172.21.15.42:0/1062590033 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8003990 con 0x7f9bb81011a0 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bb4ff9640 1 -- 172.21.15.42:0/1062590033 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ba8004020 con 0x7f9bb81011a0 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.802+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb819f250 con 0x7f9bb81011a0 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.803+0000 7f9bb4ff9640 1 -- 172.21.15.42:0/1062590033 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800f6b0 con 0x7f9bb81011a0 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.803+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb819f760 con 0x7f9bb81011a0 2024-03-12T04:14:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.804+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb8108510 con 0x7f9bb81011a0 2024-03-12T04:14:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.805+0000 7f9bb4ff9640 1 -- 172.21.15.42:0/1062590033 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9ba8004190 con 0x7f9bb81011a0 2024-03-12T04:14:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.805+0000 7f9bb4ff9640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ba00a1a30 0x7f9ba00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.805+0000 7f9bb6ffd640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ba00a1a30 0x7f9ba00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.805+0000 7f9bb4ff9640 1 -- 172.21.15.42:0/1062590033 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9ba80cd860 con 0x7f9bb81011a0 2024-03-12T04:14:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.806+0000 7f9bb6ffd640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ba00a1a30 0x7f9ba00a3ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9bac036040 tx=0x7f9bac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:20.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.809+0000 7f9bb4ff9640 1 -- 172.21.15.42:0/1062590033 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ba8096aa0 con 0x7f9bb81011a0 2024-03-12T04:14:20.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:20 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2762247975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:20 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1541621808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:20 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/102226069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:20 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2762247975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:20 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1541621808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:20 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/102226069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:20.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.994+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bb810ca60 con 0x7f9bb81011a0 2024-03-12T04:14:20.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.995+0000 7f9bb4ff9640 1 -- 172.21.15.42:0/1062590033 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f9ba809a160 con 0x7f9bb81011a0 2024-03-12T04:14:20.998 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:20.998 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:21.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ba00a1a30 msgr2=0x7f9ba00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ba00a1a30 0x7f9ba00a3ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9bac036040 tx=0x7f9bac000bc0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb81011a0 msgr2=0x7f9bb8197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb81011a0 0x7f9bb8197150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800ccb0 tx=0x7f9ba800a620 comp rx=0 tx=0).stop 2024-03-12T04:14:21.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 shutdown_connections 2024-03-12T04:14:21.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bb8102d50 0x7f9bb819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f9ba00a1a30 0x7f9ba00a3ef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb8102390 0x7f9bb8197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 --2- 172.21.15.42:0/1062590033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb81011a0 0x7f9bb8197150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 >> 172.21.15.42:0/1062590033 conn(0x7f9bb80fc730 msgr2=0x7f9bb81056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:21.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 shutdown_connections 2024-03-12T04:14:21.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:20.997+0000 7f9bbe10c640 1 -- 172.21.15.42:0/1062590033 wait complete. 2024-03-12T04:14:21.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:14:21.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 -- 172.21.15.42:0/2583422392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60102380 msgr2=0x7fdb60102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 --2- 172.21.15.42:0/2583422392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60102380 0x7fdb60102800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 -- 172.21.15.42:0/2583422392 shutdown_connections 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 --2- 172.21.15.42:0/2583422392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb60102d40 0x7fdb601095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 --2- 172.21.15.42:0/2583422392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60102380 0x7fdb60102800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 --2- 172.21.15.42:0/2583422392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb60101190 0x7fdb601015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 -- 172.21.15.42:0/2583422392 >> 172.21.15.42:0/2583422392 conn(0x7fdb600fc720 msgr2=0x7fdb600feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 -- 172.21.15.42:0/2583422392 shutdown_connections 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 -- 172.21.15.42:0/2583422392 wait complete. 2024-03-12T04:14:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 Processor -- start 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 -- start start 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.115+0000 7fdb6840b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60101190 0x7fdb60197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb6840b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb60102380 0x7fdb601976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb6840b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb60102d40 0x7fdb6019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb6840b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6019ec70 con 0x7fdb60101190 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb6840b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6019ede0 con 0x7fdb60102d40 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb6840b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6019efa0 con 0x7fdb60102380 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60101190 0x7fdb60197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60101190 0x7fdb60197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36250/0 (socket says 172.21.15.42:36250) 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 -- 172.21.15.42:0/680688345 learned_addr learned my addr 172.21.15.42:0/680688345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 -- 172.21.15.42:0/680688345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb60102380 msgr2=0x7fdb601976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb60102380 0x7fdb601976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 -- 172.21.15.42:0/680688345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb60102d40 msgr2=0x7fdb6019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb60102d40 0x7fdb6019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 -- 172.21.15.42:0/680688345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50009560 con 0x7fdb60101190 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb661a7640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60101190 0x7fdb60197180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c970 tx=0x7fdb5c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb577fe640 1 -- 172.21.15.42:0/680688345 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c007c10 con 0x7fdb60101190 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb577fe640 1 -- 172.21.15.42:0/680688345 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb5c007d70 con 0x7fdb60101190 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6019f280 con 0x7fdb60101190 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb577fe640 1 -- 172.21.15.42:0/680688345 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c00f7d0 con 0x7fdb60101190 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.116+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6019f860 con 0x7fdb60101190 2024-03-12T04:14:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.117+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb601084b0 con 0x7fdb60101190 2024-03-12T04:14:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.118+0000 7fdb577fe640 1 -- 172.21.15.42:0/680688345 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdb5c004020 con 0x7fdb60101190 2024-03-12T04:14:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.119+0000 7fdb577fe640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fdb4c0a1ab0 0x7fdb4c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.119+0000 7fdb659a6640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fdb4c0a1ab0 0x7fdb4c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.119+0000 7fdb577fe640 1 -- 172.21.15.42:0/680688345 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdb5c013070 con 0x7fdb60101190 2024-03-12T04:14:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.119+0000 7fdb659a6640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fdb4c0a1ab0 0x7fdb4c0a3f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb50036040 tx=0x7fdb50000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:21.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.122+0000 7fdb577fe640 1 -- 172.21.15.42:0/680688345 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb5c096d60 con 0x7fdb60101190 2024-03-12T04:14:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.303+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb6010cae0 con 0x7fdb60101190 2024-03-12T04:14:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.304+0000 7fdb577fe640 1 -- 172.21.15.42:0/680688345 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb5c09a420 con 0x7fdb60101190 2024-03-12T04:14:21.307 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:14:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fdb4c0a1ab0 msgr2=0x7fdb4c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fdb4c0a1ab0 0x7fdb4c0a3f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb50036040 tx=0x7fdb50000bc0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60101190 msgr2=0x7fdb60197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60101190 0x7fdb60197180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c970 tx=0x7fdb5c00ce30 comp rx=0 tx=0).stop 2024-03-12T04:14:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 shutdown_connections 2024-03-12T04:14:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fdb4c0a1ab0 0x7fdb4c0a3f70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb60102d40 0x7fdb6019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb60102380 0x7fdb601976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 --2- 172.21.15.42:0/680688345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60101190 0x7fdb60197180 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 >> 172.21.15.42:0/680688345 conn(0x7fdb600fc720 msgr2=0x7fdb60105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 shutdown_connections 2024-03-12T04:14:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.306+0000 7fdb6840b640 1 -- 172.21.15.42:0/680688345 wait complete. 2024-03-12T04:14:21.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:14:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.422+0000 7f818c22e640 1 -- 172.21.15.42:0/3533939048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 msgr2=0x7f81840ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.422+0000 7f818c22e640 1 --2- 172.21.15.42:0/3533939048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81840ff880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8180009900 tx=0x7f818002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- 172.21.15.42:0/3533939048 shutdown_connections 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 --2- 172.21.15.42:0/3533939048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8184102df0 0x7f8184109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 --2- 172.21.15.42:0/3533939048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8184102430 0x7f81841028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 --2- 172.21.15.42:0/3533939048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81840ff880 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- 172.21.15.42:0/3533939048 >> 172.21.15.42:0/3533939048 conn(0x7f81840fadb0 msgr2=0x7f81840fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- 172.21.15.42:0/3533939048 shutdown_connections 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- 172.21.15.42:0/3533939048 wait complete. 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 Processor -- start 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- start start 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81841970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8184102430 0x7f8184197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8184102df0 0x7f818419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818419ebe0 con 0x7f81840ff460 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818419ed50 con 0x7f8184102430 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f818c22e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818419ef10 con 0x7f8184102df0 2024-03-12T04:14:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f8189fca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81841970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f8189fca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81841970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37524/0 (socket says 172.21.15.42:37524) 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.423+0000 7f8189fca640 1 -- 172.21.15.42:0/3538127836 learned_addr learned my addr 172.21.15.42:0/3538127836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f818a7cb640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8184102df0 0x7f818419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f81897c9640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8184102430 0x7f8184197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f8189fca640 1 -- 172.21.15.42:0/3538127836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8184102df0 msgr2=0x7f818419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f8189fca640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8184102df0 0x7f818419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f8189fca640 1 -- 172.21.15.42:0/3538127836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8184102430 msgr2=0x7f8184197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f8189fca640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8184102430 0x7f8184197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f8189fca640 1 -- 172.21.15.42:0/3538127836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009560 con 0x7f81840ff460 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f818a7cb640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8184102df0 0x7f818419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f8189fca640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81841970f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8180000c00 tx=0x7f8180004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f817affd640 1 -- 172.21.15.42:0/3538127836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180003800 con 0x7f81840ff460 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818419f190 con 0x7f81840ff460 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f817affd640 1 -- 172.21.15.42:0/3538127836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8180003e10 con 0x7f81840ff460 2024-03-12T04:14:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f817affd640 1 -- 172.21.15.42:0/3538127836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818003e450 con 0x7f81840ff460 2024-03-12T04:14:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.424+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818419f6f0 con 0x7f81840ff460 2024-03-12T04:14:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.425+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81841085b0 con 0x7f81840ff460 2024-03-12T04:14:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.426+0000 7f817affd640 1 -- 172.21.15.42:0/3538127836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8180004f50 con 0x7f81840ff460 2024-03-12T04:14:21.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.426+0000 7f817affd640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f81700a1a30 0x7f81700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.427+0000 7f81897c9640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f81700a1a30 0x7f81700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.427+0000 7f817affd640 1 -- 172.21.15.42:0/3538127836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f81800ef960 con 0x7f81840ff460 2024-03-12T04:14:21.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.427+0000 7f81897c9640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f81700a1a30 0x7f81700a3ef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8184198600 tx=0x7f8174005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:21.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.431+0000 7f817affd640 1 -- 172.21.15.42:0/3538127836 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f81800b8ba0 con 0x7f81840ff460 2024-03-12T04:14:21.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.612+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f818419f320 con 0x7f81840ff460 2024-03-12T04:14:21.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.613+0000 7f817affd640 1 -- 172.21.15.42:0/3538127836 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f818419f320 con 0x7f81840ff460 2024-03-12T04:14:21.615 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f81700a1a30 msgr2=0x7f81700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f81700a1a30 0x7f81700a3ef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8184198600 tx=0x7f8174005cc0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 msgr2=0x7f81841970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81841970f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8180000c00 tx=0x7f8180004360 comp rx=0 tx=0).stop 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 shutdown_connections 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8184102df0 0x7f818419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f81700a1a30 0x7f81700a3ef0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8184102430 0x7f8184197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 --2- 172.21.15.42:0/3538127836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81840ff460 0x7f81841970f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 >> 172.21.15.42:0/3538127836 conn(0x7f81840fadb0 msgr2=0x7f8184105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 shutdown_connections 2024-03-12T04:14:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.615+0000 7f818c22e640 1 -- 172.21.15.42:0/3538127836 wait complete. 2024-03-12T04:14:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:21 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3214300973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:21 smithi042 ceph-mon[32260]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:21 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1062590033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:21 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/680688345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:21.628 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 -- 172.21.15.42:0/2810016289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd44101290 msgr2=0x7fbd44101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 --2- 172.21.15.42:0/2810016289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd44101290 0x7fbd44101710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009900 tx=0x7fbd3402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 -- 172.21.15.42:0/2810016289 shutdown_connections 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 --2- 172.21.15.42:0/2810016289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd44101c50 0x7fbd441084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 --2- 172.21.15.42:0/2810016289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd44101290 0x7fbd44101710 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 --2- 172.21.15.42:0/2810016289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd441000a0 0x7fbd441004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 -- 172.21.15.42:0/2810016289 >> 172.21.15.42:0/2810016289 conn(0x7fbd440fb630 msgr2=0x7fbd440fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 -- 172.21.15.42:0/2810016289 shutdown_connections 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.731+0000 7fbd4b281640 1 -- 172.21.15.42:0/2810016289 wait complete. 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 Processor -- start 2024-03-12T04:14:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 -- start start 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd441000a0 0x7fbd44197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd44101290 0x7fbd441976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd44101c50 0x7fbd4419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4419ec70 con 0x7fbd441000a0 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4419ede0 con 0x7fbd44101290 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4419efa0 con 0x7fbd44101c50 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd441000a0 0x7fbd44197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd441000a0 0x7fbd44197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37546/0 (socket says 172.21.15.42:37546) 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 -- 172.21.15.42:0/3961381686 learned_addr learned my addr 172.21.15.42:0/3961381686 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4881c640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd44101290 0x7fbd441976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 -- 172.21.15.42:0/3961381686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd44101c50 msgr2=0x7fbd4419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd44101c50 0x7fbd4419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 -- 172.21.15.42:0/3961381686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd44101290 msgr2=0x7fbd441976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd44101290 0x7fbd441976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 -- 172.21.15.42:0/3961381686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009560 con 0x7fbd441000a0 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4901d640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd441000a0 0x7fbd44197180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000d950 tx=0x7fbd4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd3a7fc640 1 -- 172.21.15.42:0/3961381686 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40009c30 con 0x7fbd441000a0 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd3a7fc640 1 -- 172.21.15.42:0/3961381686 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd40009d90 con 0x7fbd441000a0 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4419f280 con 0x7fbd441000a0 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.732+0000 7fbd3a7fc640 1 -- 172.21.15.42:0/3961381686 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000f6b0 con 0x7fbd441000a0 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.733+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4419f740 con 0x7fbd441000a0 2024-03-12T04:14:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.733+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd441074b0 con 0x7fbd441000a0 2024-03-12T04:14:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.734+0000 7fbd3a7fc640 1 -- 172.21.15.42:0/3961381686 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fbd40010a30 con 0x7fbd441000a0 2024-03-12T04:14:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.735+0000 7fbd3a7fc640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbd300a1a30 0x7fbd300a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.735+0000 7fbd4881c640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbd300a1a30 0x7fbd300a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.735+0000 7fbd3a7fc640 1 -- 172.21.15.42:0/3961381686 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbd400cd210 con 0x7fbd441000a0 2024-03-12T04:14:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.735+0000 7fbd4881c640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbd300a1a30 0x7fbd300a3ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd44198690 tx=0x7fbd34000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.739+0000 7fbd3a7fc640 1 -- 172.21.15.42:0/3961381686 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd40096450 con 0x7fbd441000a0 2024-03-12T04:14:21.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.921+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd4402cee0 con 0x7fbd441000a0 2024-03-12T04:14:21.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.922+0000 7fbd3a7fc640 1 -- 172.21.15.42:0/3961381686 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd40099b10 con 0x7fbd441000a0 2024-03-12T04:14:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:14:21.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbd300a1a30 msgr2=0x7fbd300a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbd300a1a30 0x7fbd300a3ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd44198690 tx=0x7fbd34000bc0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd441000a0 msgr2=0x7fbd44197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd441000a0 0x7fbd44197180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000d950 tx=0x7fbd4000de10 comp rx=0 tx=0).stop 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 shutdown_connections 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd44101c50 0x7fbd4419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fbd300a1a30 0x7fbd300a3ef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd44101290 0x7fbd441976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 --2- 172.21.15.42:0/3961381686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd441000a0 0x7fbd44197180 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 >> 172.21.15.42:0/3961381686 conn(0x7fbd440fb630 msgr2=0x7fbd44104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 shutdown_connections 2024-03-12T04:14:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:21.924+0000 7fbd4b281640 1 -- 172.21.15.42:0/3961381686 wait complete. 2024-03-12T04:14:21.937 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:14:21.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:21 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3214300973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:21.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:21 smithi119 ceph-mon[36119]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:21.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:21 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1062590033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:21.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:21 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/680688345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:21 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3214300973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:21 smithi136 ceph-mon[35571]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:21 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1062590033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:21 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/680688345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:22.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 -- 172.21.15.42:0/1739342810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 msgr2=0x7ffbd81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/1739342810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd81067f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009900 tx=0x7ffbcc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 -- 172.21.15.42:0/1739342810 shutdown_connections 2024-03-12T04:14:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/1739342810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8106d30 0x7ffbd8109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/1739342810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd81067f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/1739342810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbd8101ad0 0x7ffbd8103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 -- 172.21.15.42:0/1739342810 >> 172.21.15.42:0/1739342810 conn(0x7ffbd80fb460 msgr2=0x7ffbd80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 -- 172.21.15.42:0/1739342810 shutdown_connections 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.037+0000 7ffbde5a0640 1 -- 172.21.15.42:0/1739342810 wait complete. 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 Processor -- start 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 -- start start 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbd8101ad0 0x7ffbd8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8106d30 0x7ffbd819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd819eaf0 con 0x7ffbd8104400 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd819ec60 con 0x7ffbd8106d30 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbde5a0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd819ee20 con 0x7ffbd8101ad0 2024-03-12T04:14:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd8197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37556/0 (socket says 172.21.15.42:37556) 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 -- 172.21.15.42:0/3202820900 learned_addr learned my addr 172.21.15.42:0/3202820900 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbddd9f640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8106d30 0x7ffbd819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdd59e640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbd8101ad0 0x7ffbd8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 -- 172.21.15.42:0/3202820900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbd8101ad0 msgr2=0x7ffbd8197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbd8101ad0 0x7ffbd8197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 -- 172.21.15.42:0/3202820900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8106d30 msgr2=0x7ffbd819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8106d30 0x7ffbd819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbdcd9d640 1 -- 172.21.15.42:0/3202820900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc009560 con 0x7ffbd8104400 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.038+0000 7ffbddd9f640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8106d30 0x7ffbd819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.039+0000 7ffbdcd9d640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd8197540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc000c00 tx=0x7ffbcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.039+0000 7ffbd67fc640 1 -- 172.21.15.42:0/3202820900 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0044e0 con 0x7ffbd8104400 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.039+0000 7ffbd67fc640 1 -- 172.21.15.42:0/3202820900 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffbcc02de90 con 0x7ffbd8104400 2024-03-12T04:14:22.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.039+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd819f0a0 con 0x7ffbd8104400 2024-03-12T04:14:22.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.039+0000 7ffbd67fc640 1 -- 172.21.15.42:0/3202820900 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc005410 con 0x7ffbd8104400 2024-03-12T04:14:22.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.039+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd819f680 con 0x7ffbd8104400 2024-03-12T04:14:22.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.040+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd80fcf50 con 0x7ffbd8104400 2024-03-12T04:14:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.041+0000 7ffbd67fc640 1 -- 172.21.15.42:0/3202820900 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ffbcc005570 con 0x7ffbd8104400 2024-03-12T04:14:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.041+0000 7ffbd67fc640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc00a1ab0 0x7ffbc00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.042+0000 7ffbdd59e640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc00a1ab0 0x7ffbc00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.042+0000 7ffbd67fc640 1 -- 172.21.15.42:0/3202820900 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffbcc0efde0 con 0x7ffbd8104400 2024-03-12T04:14:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.042+0000 7ffbdd59e640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc00a1ab0 0x7ffbc00a3f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009890 tx=0x7ffbd0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.045+0000 7ffbd67fc640 1 -- 172.21.15.42:0/3202820900 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffbcc0b8fc0 con 0x7ffbd8104400 2024-03-12T04:14:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.266+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbd810cb20 con 0x7ffbd8104400 2024-03-12T04:14:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.267+0000 7ffbd67fc640 1 -- 172.21.15.42:0/3202820900 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11388 (secure 0 0 0) 0x7ffbcc043090 con 0x7ffbd8104400 2024-03-12T04:14:22.270 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:22.270 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:13:01.029979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:14:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc00a1ab0 msgr2=0x7ffbc00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc00a1ab0 0x7ffbc00a3f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009890 tx=0x7ffbd0008040 comp rx=0 tx=0).stop 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 msgr2=0x7ffbd8197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd8197540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc000c00 tx=0x7ffbcc0042f0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 shutdown_connections 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffbc00a1ab0 0x7ffbc00a3f70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8106d30 0x7ffbd819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbd8104400 0x7ffbd8197540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 --2- 172.21.15.42:0/3202820900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbd8101ad0 0x7ffbd8197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 >> 172.21.15.42:0/3202820900 conn(0x7ffbd80fb460 msgr2=0x7ffbd81023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 shutdown_connections 2024-03-12T04:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.269+0000 7ffbde5a0640 1 -- 172.21.15.42:0/3202820900 wait complete. 2024-03-12T04:14:22.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- 172.21.15.42:0/1109161945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a301000a0 msgr2=0x7f7a301004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 --2- 172.21.15.42:0/1109161945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a301000a0 0x7f7a301004c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009900 tx=0x7f7a2c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- 172.21.15.42:0/1109161945 shutdown_connections 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 --2- 172.21.15.42:0/1109161945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a30101c50 0x7f7a301084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 --2- 172.21.15.42:0/1109161945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a30101290 0x7f7a30101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 --2- 172.21.15.42:0/1109161945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a301000a0 0x7f7a301004c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- 172.21.15.42:0/1109161945 >> 172.21.15.42:0/1109161945 conn(0x7f7a300fb630 msgr2=0x7f7a300fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- 172.21.15.42:0/1109161945 shutdown_connections 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- 172.21.15.42:0/1109161945 wait complete. 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 Processor -- start 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- start start 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301000a0 0x7f7a30197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a30101290 0x7f7a30197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a30101c50 0x7f7a3019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019ec80 con 0x7f7a30101290 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019edf0 con 0x7f7a30101c50 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.383+0000 7f7a37c81640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019efb0 con 0x7f7a301000a0 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a30101290 0x7f7a30197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a30101290 0x7f7a30197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37582/0 (socket says 172.21.15.42:37582) 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 -- 172.21.15.42:0/86260374 learned_addr learned my addr 172.21.15.42:0/86260374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3621e640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a30101c50 0x7f7a3019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a35a1d640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301000a0 0x7f7a30197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 -- 172.21.15.42:0/86260374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301000a0 msgr2=0x7f7a30197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301000a0 0x7f7a30197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 -- 172.21.15.42:0/86260374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a30101c50 msgr2=0x7f7a3019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a30101c50 0x7f7a3019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 -- 172.21.15.42:0/86260374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009560 con 0x7f7a30101290 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a35a1d640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301000a0 0x7f7a30197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3621e640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a30101c50 0x7f7a3019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a3521c640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a30101290 0x7f7a30197740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c920 tx=0x7f7a2000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a26ffd640 1 -- 172.21.15.42:0/86260374 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20007c70 con 0x7f7a30101290 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3019f290 con 0x7f7a30101290 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a26ffd640 1 -- 172.21.15.42:0/86260374 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a20007dd0 con 0x7f7a30101290 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a26ffd640 1 -- 172.21.15.42:0/86260374 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200187c0 con 0x7f7a30101290 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.384+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3019f820 con 0x7f7a30101290 2024-03-12T04:14:22.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.385+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3010b9a0 con 0x7f7a30101290 2024-03-12T04:14:22.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.386+0000 7f7a26ffd640 1 -- 172.21.15.42:0/86260374 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7a2000f450 con 0x7f7a30101290 2024-03-12T04:14:22.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.387+0000 7f7a26ffd640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7a1c0a1a30 0x7f7a1c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.387+0000 7f7a35a1d640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7a1c0a1a30 0x7f7a1c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.387+0000 7f7a26ffd640 1 -- 172.21.15.42:0/86260374 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7a200cd5f0 con 0x7f7a30101290 2024-03-12T04:14:22.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.387+0000 7f7a35a1d640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7a1c0a1a30 0x7f7a1c0a3ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c036040 tx=0x7f7a2c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:22.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.391+0000 7f7a26ffd640 1 -- 172.21.15.42:0/86260374 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a20096830 con 0x7f7a30101290 2024-03-12T04:14:22.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3538127836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:22.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3961381686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:22.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3202820900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.576+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a3019fc40 con 0x7f7a30101290 2024-03-12T04:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.577+0000 7f7a26ffd640 1 -- 172.21.15.42:0/86260374 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f7a20099ef0 con 0x7f7a30101290 2024-03-12T04:14:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:22.580 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7a1c0a1a30 msgr2=0x7f7a1c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7a1c0a1a30 0x7f7a1c0a3ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c036040 tx=0x7f7a2c000bc0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a30101290 msgr2=0x7f7a30197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a30101290 0x7f7a30197740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c920 tx=0x7f7a2000cde0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 shutdown_connections 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7a1c0a1a30 0x7f7a1c0a3ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a30101c50 0x7f7a3019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a30101290 0x7f7a30197740 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 --2- 172.21.15.42:0/86260374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301000a0 0x7f7a30197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 >> 172.21.15.42:0/86260374 conn(0x7f7a300fb630 msgr2=0x7f7a300fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 shutdown_connections 2024-03-12T04:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.579+0000 7f7a37c81640 1 -- 172.21.15.42:0/86260374 wait complete. 2024-03-12T04:14:22.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:14:22.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.692+0000 7f902358c640 1 -- 172.21.15.42:0/487942343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101c50 msgr2=0x7f901c1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.692+0000 7f902358c640 1 --2- 172.21.15.42:0/487942343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101c50 0x7f901c1084e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f900c009ca0 tx=0x7f900c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.692+0000 7f902358c640 1 -- 172.21.15.42:0/487942343 shutdown_connections 2024-03-12T04:14:22.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.692+0000 7f902358c640 1 --2- 172.21.15.42:0/487942343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101c50 0x7f901c1084e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.692+0000 7f902358c640 1 --2- 172.21.15.42:0/487942343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f901c101290 0x7f901c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.692+0000 7f902358c640 1 --2- 172.21.15.42:0/487942343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f901c1000a0 0x7f901c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.692+0000 7f902358c640 1 -- 172.21.15.42:0/487942343 >> 172.21.15.42:0/487942343 conn(0x7f901c0fb610 msgr2=0x7f901c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 -- 172.21.15.42:0/487942343 shutdown_connections 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 -- 172.21.15.42:0/487942343 wait complete. 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 Processor -- start 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 -- start start 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f901c1000a0 0x7f901c10a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101290 0x7f901c108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f901c101c50 0x7f901c1094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c109c40 con 0x7f901c101290 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c109db0 con 0x7f901c101c50 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f902358c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1a1160 con 0x7f901c1000a0 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f9020b27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101290 0x7f901c108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f9020b27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101290 0x7f901c108fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37596/0 (socket says 172.21.15.42:37596) 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f9020b27640 1 -- 172.21.15.42:0/1635271190 learned_addr learned my addr 172.21.15.42:0/1635271190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.693+0000 7f9021328640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f901c1000a0 0x7f901c10a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f9021b29640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f901c101c50 0x7f901c1094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f9020b27640 1 -- 172.21.15.42:0/1635271190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f901c1000a0 msgr2=0x7f901c10a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f9020b27640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f901c1000a0 0x7f901c10a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f9020b27640 1 -- 172.21.15.42:0/1635271190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f901c101c50 msgr2=0x7f901c1094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f9020b27640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f901c101c50 0x7f901c1094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f9020b27640 1 -- 172.21.15.42:0/1635271190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c009900 con 0x7f901c101290 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f9020b27640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101290 0x7f901c108fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f901000d7a0 tx=0x7f901000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f901a7fc640 1 -- 172.21.15.42:0/1635271190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010009c30 con 0x7f901c101290 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f901a7fc640 1 -- 172.21.15.42:0/1635271190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f901000deb0 con 0x7f901c101290 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c1a1410 con 0x7f901c101290 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f901a7fc640 1 -- 172.21.15.42:0/1635271190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010004dd0 con 0x7f901c101290 2024-03-12T04:14:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.694+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c1a18d0 con 0x7f901c101290 2024-03-12T04:14:22.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.695+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c107410 con 0x7f901c101290 2024-03-12T04:14:22.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.696+0000 7f901a7fc640 1 -- 172.21.15.42:0/1635271190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9010004f30 con 0x7f901c101290 2024-03-12T04:14:22.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.697+0000 7f901a7fc640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f90040a1a30 0x7f90040a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:22.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.697+0000 7f9021328640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f90040a1a30 0x7f90040a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:22.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.697+0000 7f901a7fc640 1 -- 172.21.15.42:0/1635271190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f90100cce50 con 0x7f901c101290 2024-03-12T04:14:22.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.697+0000 7f9021328640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f90040a1a30 0x7f90040a3ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f901c1010f0 tx=0x7f901400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:22.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.701+0000 7f901a7fc640 1 -- 172.21.15.42:0/1635271190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9010099bc0 con 0x7f901c101290 2024-03-12T04:14:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.924+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f901c02ce40 con 0x7f901c101290 2024-03-12T04:14:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.925+0000 7f901a7fc640 1 -- 172.21.15.42:0/1635271190 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f9010017090 con 0x7f901c101290 2024-03-12T04:14:22.928 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:22.928 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:13:01.029979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:14:22.928 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-03-12T04:14:22.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f90040a1a30 msgr2=0x7f90040a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f90040a1a30 0x7f90040a3ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f901c1010f0 tx=0x7f901400b390 comp rx=0 tx=0).stop 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101290 msgr2=0x7f901c108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101290 0x7f901c108fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f901000d7a0 tx=0x7f901000dc60 comp rx=0 tx=0).stop 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 shutdown_connections 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f90040a1a30 0x7f90040a3ef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f901c101c50 0x7f901c1094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f901c101290 0x7f901c108fa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 --2- 172.21.15.42:0/1635271190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f901c1000a0 0x7f901c10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 >> 172.21.15.42:0/1635271190 conn(0x7f901c0fb610 msgr2=0x7f901c1045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 shutdown_connections 2024-03-12T04:14:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:22.927+0000 7f902358c640 1 -- 172.21.15.42:0/1635271190 wait complete. 2024-03-12T04:14:22.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-12T04:12:54.116170+0000', 'modified': '2024-03-12T04:13:01.029979+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24515, 'mds_1': 24490, 'mds_2': 24485, 'mds_3': 24475, 'mds_4': 14634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.42:6839/1870620293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6838', 'nonce': 1870620293}, {'type': 'v1', 'addr': '172.21.15.42:6839', 'nonce': 1870620293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14634': {'gid': 14634, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.42:6841/799836757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6840', 'nonce': 799836757}, {'type': 'v1', 'addr': '172.21.15.42:6841', 'nonce': 799836757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/3418533459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 3418533459}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 3418533459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6833/2928023683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6832', 'nonce': 2928023683}, {'type': 'v1', 'addr': '172.21.15.119:6833', 'nonce': 2928023683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/2794337720', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 2794337720}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 2794337720}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/2718353674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 2718353674}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 2718353674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.136:6837/23397667', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 23397667}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 23397667}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.119:6837/3839383373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 3839383373}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 3839383373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.136:6839/3211189908', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 3211189908}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 3211189908}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6839/1451526204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1451526204}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1451526204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-12T04:14:22.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-12T04:14:22.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-12T04:14:22.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:22 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3538127836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:22.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:22 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3961381686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:22.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:22 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3202820900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:22.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:22 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3538127836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:22.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:22 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3961381686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:22.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:22 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3202820900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.041+0000 7efe640eb640 1 -- 172.21.15.42:0/2980506591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 msgr2=0x7efe5c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.041+0000 7efe640eb640 1 --2- 172.21.15.42:0/2980506591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c1004b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe58009900 tx=0x7efe5802bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- 172.21.15.42:0/2980506591 shutdown_connections 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 --2- 172.21.15.42:0/2980506591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c101c40 0x7efe5c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 --2- 172.21.15.42:0/2980506591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe5c101280 0x7efe5c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 --2- 172.21.15.42:0/2980506591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c1004b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- 172.21.15.42:0/2980506591 >> 172.21.15.42:0/2980506591 conn(0x7efe5c0fb620 msgr2=0x7efe5c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- 172.21.15.42:0/2980506591 shutdown_connections 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- 172.21.15.42:0/2980506591 wait complete. 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 Processor -- start 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- start start 2024-03-12T04:14:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe5c101280 0x7efe5c19be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c101c40 0x7efe5c19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c19ec50 con 0x7efe5c100090 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c19edc0 con 0x7efe5c101c40 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe640eb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c19ef80 con 0x7efe5c101280 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe61e87640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe61e87640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c06a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37622/0 (socket says 172.21.15.42:37622) 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.042+0000 7efe61e87640 1 -- 172.21.15.42:0/3571040452 learned_addr learned my addr 172.21.15.42:0/3571040452 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe61686640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe5c101280 0x7efe5c19be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe62688640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c101c40 0x7efe5c19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe61e87640 1 -- 172.21.15.42:0/3571040452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe5c101280 msgr2=0x7efe5c19be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe61e87640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe5c101280 0x7efe5c19be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe61e87640 1 -- 172.21.15.42:0/3571040452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c101c40 msgr2=0x7efe5c19e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe61e87640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c101c40 0x7efe5c19e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe61e87640 1 -- 172.21.15.42:0/3571040452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe58009560 con 0x7efe5c100090 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe61e87640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c06a320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe58000c00 tx=0x7efe580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe52ffd640 1 -- 172.21.15.42:0/3571040452 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe580044e0 con 0x7efe5c100090 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe52ffd640 1 -- 172.21.15.42:0/3571040452 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efe5802de90 con 0x7efe5c100090 2024-03-12T04:14:23.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c19f200 con 0x7efe5c100090 2024-03-12T04:14:23.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe52ffd640 1 -- 172.21.15.42:0/3571040452 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58005410 con 0x7efe5c100090 2024-03-12T04:14:23.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.043+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c19f760 con 0x7efe5c100090 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe52ffd640 1 -- 172.21.15.42:0/3571040452 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7efe58005570 con 0x7efe5c100090 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 Processor -- start 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 -- start start 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b1970 0x5642b14b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b146ccd0 con 0x5642b14b0f80 2024-03-12T04:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b141ff20 con 0x5642b14b6ae0 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.046+0000 7efe66d97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b141d450 con 0x5642b14b1970 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37624/0 (socket says 172.21.15.42:37624) 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 -- 172.21.15.42:0/2837875148 learned_addr learned my addr 172.21.15.42:0/2837875148 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe52ffd640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efe480a1ab0 0x7efe480a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe61686640 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efe480a1ab0 0x7efe480a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe52ffd640 1 -- 172.21.15.42:0/3571040452 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efe580ef590 con 0x7efe5c100090 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe61e87640 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b1970 0x5642b14b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 -- 172.21.15.42:0/2837875148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b1970 msgr2=0x5642b14b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b1970 0x5642b14b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 -- 172.21.15.42:0/2837875148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 msgr2=0x5642b14b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 -- 172.21.15.42:0/2837875148 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b13cdf20 con 0x5642b14b0f80 2024-03-12T04:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe61e87640 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b1970 0x5642b14b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe61686640 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14b13a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe540128f0 tx=0x7efe54012db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ac09478a7f2d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe47fff640 1 -- 172.21.15.42:0/2837875148 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54020070 con 0x5642b14b0f80 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe62688640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efe480a1ab0 0x7efe480a3f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe5401f000 tx=0x7efe5401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe47fff640 1 -- 172.21.15.42:0/2837875148 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efe54018070 con 0x5642b14b0f80 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.047+0000 7efe47fff640 1 -- 172.21.15.42:0/2837875148 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54024630 con 0x5642b14b0f80 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 -- 172.21.15.42:0/2837875148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 msgr2=0x5642b14b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14b13a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe540128f0 tx=0x7efe54012db0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 -- 172.21.15.42:0/2837875148 shutdown_connections 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b1970 0x5642b14b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 --2- 172.21.15.42:0/2837875148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14b13a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 -- 172.21.15.42:0/2837875148 >> 172.21.15.42:0/2837875148 conn(0x5642b14ac4f0 msgr2=0x5642b14ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 -- 172.21.15.42:0/2837875148 shutdown_connections 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 -- 172.21.15.42:0/2837875148 wait complete. 2024-03-12T04:14:23.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.048+0000 7efe66d97740 1 -- start start 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 1 Processor -- start 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b0f80 0x5642b14bb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b1418c30 con 0x5642b14b1970 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b13bdd30 con 0x5642b14b6ae0 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b10ec310 con 0x5642b14b0f80 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe66d97740 10 client.0 _ll_register_callbacks cb 0x5642b13c5d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37634/0 (socket says 172.21.15.42:37634) 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 -- 172.21.15.42:0/1141164908 learned_addr learned my addr 172.21.15.42:0/1141164908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61e87640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b0f80 0x5642b14bb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 -- 172.21.15.42:0/1141164908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b0f80 msgr2=0x5642b14bb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe62688640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b0f80 0x5642b14bb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 -- 172.21.15.42:0/1141164908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 msgr2=0x5642b14c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 -- 172.21.15.42:0/1141164908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe54012530 con 0x5642b14b1970 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe62688640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe61686640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bb6a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00b9c0 tx=0x7efe4c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdea657c77b36f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.049+0000 7efe457fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe457fa640 1 -- 172.21.15.42:0/1141164908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c01b440 con 0x5642b14b1970 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe457fa640 1 -- 172.21.15.42:0/1141164908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efe4c01b5a0 con 0x5642b14b1970 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642b11186c0 con 0x5642b14b1970 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe457fa640 1 -- 172.21.15.42:0/1141164908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe4c005020 con 0x5642b14b1970 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 10 client.15060 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5642b14172b0 con 0x5642b14b1970 2024-03-12T04:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe457fa640 1 -- 172.21.15.42:0/1141164908 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5642b11186c0 con 0x5642b14b1970 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe457fa640 1 -- 172.21.15.42:0/1141164908 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe4c01b760 con 0x5642b14b1970 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 10 client.15060 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 20 client.15060 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b14c7ae0 0x5642b14c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 4 client.15060 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642b1169270 con 0x5642b14c7ae0 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.050+0000 7efe61e87640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b14c7ae0 0x5642b14c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe457fa640 10 client.15060 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe61e87640 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b14c7ae0 0x5642b14c6d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2316d06f5141b460 server_cookie=f8a81f5168a39341 in_seq=0 out_seq=0 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe457fa640 1 -- 172.21.15.42:0/1141164908 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642b1169270 con 0x5642b14c7ae0 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe457fa640 10 client.15060 handle_command_reply: tid=0 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 client.15060 shutdown 2024-03-12T04:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 msgr2=0x5642b14bb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bb6a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00b9c0 tx=0x7efe4c00be80 comp rx=0 tx=0).stop 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 shutdown_connections 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b6ae0 0x5642b14c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b14c7ae0 0x5642b14c6d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bb6a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 --2- 172.21.15.42:0/1141164908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b0f80 0x5642b14bb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe457fa640 0 client.15060 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.051+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 >> 172.21.15.42:0/1141164908 conn(0x5642b14ac4f0 msgr2=0x5642b14ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.052+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 shutdown_connections 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.052+0000 7efe66d97740 1 -- 172.21.15.42:0/1141164908 wait complete. 2024-03-12T04:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.052+0000 7efe66d97740 20 client.15060 trim_cache size 0 max 16384 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 Processor -- start 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 -- start start 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b1970 0x5642b14c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b14ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b139e490 con 0x5642b14b0f80 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b14bc380 con 0x5642b14b1970 2024-03-12T04:14:23.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe66d97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b1169270 con 0x5642b14b6ae0 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe61e87640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe61e87640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37642/0 (socket says 172.21.15.42:37642) 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.067+0000 7efe61e87640 1 -- 172.21.15.42:0/3072946515 learned_addr learned my addr 172.21.15.42:0/3072946515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe62688640 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b14ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe61e87640 1 -- 172.21.15.42:0/3072946515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 msgr2=0x5642b14ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe61686640 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b1970 0x5642b14c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe61e87640 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b14ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe61e87640 1 -- 172.21.15.42:0/3072946515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b1970 msgr2=0x5642b14c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe61e87640 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b1970 0x5642b14c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe61e87640 1 -- 172.21.15.42:0/3072946515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b13b3a20 con 0x5642b14b0f80 2024-03-12T04:14:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe62688640 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b14ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe61e87640 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14c3130 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efe580b8990 tx=0x7efe580ecf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1aa9dda0a1be8a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe477fe640 1 -- 172.21.15.42:0/3072946515 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe580bcc30 con 0x5642b14b0f80 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe477fe640 1 -- 172.21.15.42:0/3072946515 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efe580bc530 con 0x5642b14b0f80 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.068+0000 7efe477fe640 1 -- 172.21.15.42:0/3072946515 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe580a70b0 con 0x5642b14b0f80 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- 172.21.15.42:0/3072946515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 msgr2=0x5642b14c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14c3130 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efe580b8990 tx=0x7efe580ecf90 comp rx=0 tx=0).stop 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- 172.21.15.42:0/3072946515 shutdown_connections 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b14ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b1970 0x5642b14c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 --2- 172.21.15.42:0/3072946515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b0f80 0x5642b14c3130 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- 172.21.15.42:0/3072946515 >> 172.21.15.42:0/3072946515 conn(0x5642b14ac4f0 msgr2=0x5642b14b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- 172.21.15.42:0/3072946515 shutdown_connections 2024-03-12T04:14:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- 172.21.15.42:0/3072946515 wait complete. 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- start start 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 Processor -- start 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b0f80 0x5642b14bf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b151d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b14bfeb0 con 0x5642b14b1970 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b14c0020 con 0x5642b14b0f80 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b151dc50 con 0x5642b14b6ae0 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe66d97740 10 client.0 _ll_register_callbacks cb 0x7efe54007680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe61686640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.069+0000 7efe61686640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37654/0 (socket says 172.21.15.42:37654) 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61686640 1 -- 172.21.15.42:0/3290499479 learned_addr learned my addr 172.21.15.42:0/3290499479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61e87640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b0f80 0x5642b14bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe62688640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b151d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61686640 1 -- 172.21.15.42:0/3290499479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 msgr2=0x5642b151d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61686640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b151d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61686640 1 -- 172.21.15.42:0/3290499479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b0f80 msgr2=0x5642b14bf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61686640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b0f80 0x5642b14bf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61686640 1 -- 172.21.15.42:0/3290499479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c19f760 con 0x5642b14b1970 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61e87640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b0f80 0x5642b14bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe62688640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b151d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe61686640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bf940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efe4c001eb0 tx=0x7efe4c01bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aee14cd9da4ab5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe457fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe457fa640 1 -- 172.21.15.42:0/3290499479 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0039c0 con 0x5642b14b1970 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642b14c77f0 con 0x5642b14b1970 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe457fa640 1 -- 172.21.15.42:0/3290499479 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efe4c000e40 con 0x5642b14b1970 2024-03-12T04:14:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe457fa640 1 -- 172.21.15.42:0/3290499479 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe4c005e50 con 0x5642b14b1970 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe66d97740 10 client.15066 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5642b13d9780 con 0x5642b14b1970 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.070+0000 7efe457fa640 1 -- 172.21.15.42:0/3290499479 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5642b14c77f0 con 0x5642b14b1970 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe457fa640 1 -- 172.21.15.42:0/3290499479 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe4c004bb0 con 0x5642b14b1970 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 10 client.15066 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 10 client.15066 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 20 client.15066 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b1522f70 0x5642b151f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 4 client.15066 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7efe54018070 con 0x5642b1522f70 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe61e87640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b1522f70 0x5642b151f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe457fa640 10 client.15066 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe61e87640 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b1522f70 0x5642b151f410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0fa457fd9ebabca server_cookie=fea4ec5e80a48627 in_seq=0 out_seq=0 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe457fa640 1 -- 172.21.15.42:0/3290499479 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7efe54018070 con 0x5642b1522f70 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe457fa640 10 client.15066 handle_command_reply: tid=0 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 client.15066 shutdown 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 msgr2=0x5642b14bf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bf940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efe4c001eb0 tx=0x7efe4c01bf50 comp rx=0 tx=0).stop 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 shutdown_connections 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642b14b6ae0 0x5642b151d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5642b1522f70 0x5642b151f410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642b14b1970 0x5642b14bf940 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 --2- 172.21.15.42:0/3290499479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b14b0f80 0x5642b14bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe457fa640 0 client.15066 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.071+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 >> 172.21.15.42:0/3290499479 conn(0x5642b14ac4f0 msgr2=0x5642b14ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 shutdown_connections 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe66d97740 1 -- 172.21.15.42:0/3290499479 wait complete. 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe66d97740 20 client.15066 trim_cache size 0 max 16384 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 4, 2024-03-12T04:14:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 14634, 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 83.047700628000001, 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 97.131251200999998 2024-03-12T04:14:23.084 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:23.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efe480a1ab0 msgr2=0x7efe480a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efe480a1ab0 0x7efe480a3f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe5401f000 tx=0x7efe5401e000 comp rx=0 tx=0).stop 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 msgr2=0x7efe5c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c06a320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe58000c00 tx=0x7efe580042f0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 shutdown_connections 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7efe480a1ab0 0x7efe480a3f70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c101c40 0x7efe5c19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe5c101280 0x7efe5c19be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 --2- 172.21.15.42:0/3571040452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe5c100090 0x7efe5c06a320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 >> 172.21.15.42:0/3571040452 conn(0x7efe5c0fb620 msgr2=0x7efe5c1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 shutdown_connections 2024-03-12T04:14:23.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.072+0000 7efe640eb640 1 -- 172.21.15.42:0/3571040452 wait complete. 2024-03-12T04:14:23.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-12T04:14:23.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2284715961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8100090 msgr2=0x7fc2e81004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2284715961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8100090 0x7fc2e81004b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009900 tx=0x7fc2e002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2284715961 shutdown_connections 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2284715961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8101c40 0x7fc2e81084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2284715961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e8101280 0x7fc2e8101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2284715961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8100090 0x7fc2e81004b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2284715961 >> 172.21.15.42:0/2284715961 conn(0x7fc2e80fb620 msgr2=0x7fc2e80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2284715961 shutdown_connections 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.187+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2284715961 wait complete. 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 Processor -- start 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 -- start start 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e8100090 0x7fc2e8196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8101280 0x7fc2e8196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8101c40 0x7fc2e819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e819dd30 con 0x7fc2e8101c40 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e819dea0 con 0x7fc2e8101280 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2ed0df640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e819e060 con 0x7fc2e8100090 2024-03-12T04:14:23.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8101c40 0x7fc2e819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8101c40 0x7fc2e819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37674/0 (socket says 172.21.15.42:37674) 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e759e640 1 -- 172.21.15.42:0/2328062754 learned_addr learned my addr 172.21.15.42:0/2328062754 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e8100090 0x7fc2e8196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e659c640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8101280 0x7fc2e8196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e759e640 1 -- 172.21.15.42:0/2328062754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e8100090 msgr2=0x7fc2e8196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e759e640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e8100090 0x7fc2e8196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.188+0000 7fc2e759e640 1 -- 172.21.15.42:0/2328062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8101280 msgr2=0x7fc2e8196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2e759e640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8101280 0x7fc2e8196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2e759e640 1 -- 172.21.15.42:0/2328062754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0009560 con 0x7fc2e8101c40 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2e659c640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8101280 0x7fc2e8196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2e759e640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8101c40 0x7fc2e819d7f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800b970 tx=0x7fc2d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2cffff640 1 -- 172.21.15.42:0/2328062754 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800cb40 con 0x7fc2e8101c40 2024-03-12T04:14:23.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e819e340 con 0x7fc2e8101c40 2024-03-12T04:14:23.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2cffff640 1 -- 172.21.15.42:0/2328062754 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2d800cca0 con 0x7fc2e8101c40 2024-03-12T04:14:23.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2cffff640 1 -- 172.21.15.42:0/2328062754 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80076e0 con 0x7fc2e8101c40 2024-03-12T04:14:23.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.189+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e819e8d0 con 0x7fc2e8101c40 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2cffff640 1 -- 172.21.15.42:0/2328062754 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc2d8014440 con 0x7fc2e8101c40 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 Processor -- start 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 -- start start 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97be680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2cffff640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2d00a1a30 0x7fc2d00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec9772cd0 con 0x55fec97b9190 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec9725f20 con 0x55fec97becf0 2024-03-12T04:14:23.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.191+0000 7fc2efd92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec9723450 con 0x55fec97b9b80 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97b95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37678/0 (socket says 172.21.15.42:37678) 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 -- 172.21.15.42:0/3759618168 learned_addr learned my addr 172.21.15.42:0/3759618168 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2cffff640 1 -- 172.21.15.42:0/2328062754 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc2d80cccb0 con 0x7fc2e8101c40 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2d00a1a30 0x7fc2d00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e759e640 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 -- 172.21.15.42:0/3759618168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 msgr2=0x55fec97be680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97be680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 -- 172.21.15.42:0/3759618168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 msgr2=0x55fec97c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 -- 172.21.15.42:0/3759618168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fec96d3f20 con 0x55fec97b9190 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97b95b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc015810 tx=0x7fc2dc015cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfc25ee5e4dc9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2e659c640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2d00a1a30 0x7fc2d00a3ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc013650 tx=0x7fc2dc019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2cd7fa640 1 -- 172.21.15.42:0/3759618168 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc018880 con 0x55fec97b9190 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2cd7fa640 1 -- 172.21.15.42:0/3759618168 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2dc018e90 con 0x55fec97b9190 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.192+0000 7fc2cd7fa640 1 -- 172.21.15.42:0/3759618168 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc01fa80 con 0x55fec97b9190 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 -- 172.21.15.42:0/3759618168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 msgr2=0x55fec97b95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97b95b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc015810 tx=0x7fc2dc015cd0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 -- 172.21.15.42:0/3759618168 shutdown_connections 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97be680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3759618168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97b95b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 -- 172.21.15.42:0/3759618168 >> 172.21.15.42:0/3759618168 conn(0x55fec97b24f0 msgr2=0x55fec97b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 -- 172.21.15.42:0/3759618168 shutdown_connections 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 -- 172.21.15.42:0/3759618168 wait complete. 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 -- start start 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.193+0000 7fc2efd92740 1 Processor -- start 2024-03-12T04:14:23.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec971ec30 con 0x55fec97becf0 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec96c3d30 con 0x55fec97b9190 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec93f2310 con 0x55fec97b9b80 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 10 client.0 _ll_register_callbacks cb 0x55fec96cbd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37686/0 (socket says 172.21.15.42:37686) 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 -- 172.21.15.42:0/1825966728 learned_addr learned my addr 172.21.15.42:0/1825966728 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e659c640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 -- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 msgr2=0x55fec97c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 -- 172.21.15.42:0/1825966728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 msgr2=0x55fec97c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 -- 172.21.15.42:0/1825966728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc0153c0 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2e759e640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97ca840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800b940 tx=0x7fc2d8097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276a2e433a5e5418 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2caff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2caff5640 1 -- 172.21.15.42:0/1825966728 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8097350 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2caff5640 1 -- 172.21.15.42:0/1825966728 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc2d8014a10 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fec941e6c0 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2caff5640 1 -- 172.21.15.42:0/1825966728 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d8099d00 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 10 client.15078 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.194+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fec971d2b0 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2caff5640 1 -- 172.21.15.42:0/1825966728 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55fec941e6c0 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2caff5640 1 -- 172.21.15.42:0/1825966728 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2d8097750 con 0x55fec97becf0 2024-03-12T04:14:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2efd92740 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2efd92740 10 client.15078 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2efd92740 20 client.15078 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2efd92740 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec97cfbe0 0x55fec97ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2efd92740 4 client.15078 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fec946f270 con 0x55fec97cfbe0 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec97cfbe0 0x55fec97ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2caff5640 10 client.15078 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.195+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec97cfbe0 0x55fec97ceec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c839aa816315a4c server_cookie=6e9d9f0c21f4d72d in_seq=0 out_seq=0 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2caff5640 1 -- 172.21.15.42:0/1825966728 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fec946f270 con 0x55fec97cfbe0 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2caff5640 10 client.15078 handle_command_reply: tid=0 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 client.15078 shutdown 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 msgr2=0x55fec97ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97ca840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800b940 tx=0x7fc2d8097f00 comp rx=0 tx=0).stop 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 shutdown_connections 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec97cfbe0 0x55fec97ceec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97ca840 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2caff5640 0 client.15078 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 --2- 172.21.15.42:0/1825966728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 >> 172.21.15.42:0/1825966728 conn(0x55fec97b24f0 msgr2=0x55fec97b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 shutdown_connections 2024-03-12T04:14:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 1 -- 172.21.15.42:0/1825966728 wait complete. 2024-03-12T04:14:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.196+0000 7fc2efd92740 20 client.15078 trim_cache size 0 max 16384 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 Processor -- start 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 -- start start 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97d8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec96a4490 con 0x55fec97becf0 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec97cf7b0 con 0x55fec97b9190 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2efd92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec97c2090 con 0x55fec97b9b80 2024-03-12T04:14:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97bb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37698/0 (socket says 172.21.15.42:37698) 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2e759e640 1 -- 172.21.15.42:0/3352296353 learned_addr learned my addr 172.21.15.42:0/3352296353 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97d8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2e759e640 1 -- 172.21.15.42:0/3352296353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 msgr2=0x55fec97ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2e759e640 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.212+0000 7fc2e759e640 1 -- 172.21.15.42:0/3352296353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 msgr2=0x55fec97d8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2e759e640 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97d8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2e759e640 1 -- 172.21.15.42:0/3352296353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fec96b9a20 con 0x55fec97becf0 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2e759e640 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97bb0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8099a00 tx=0x7fc2d8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49363913e91550a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2ccff9640 1 -- 172.21.15.42:0/3352296353 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8095a50 con 0x55fec97becf0 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2ccff9640 1 -- 172.21.15.42:0/3352296353 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2d8014a10 con 0x55fec97becf0 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2ccff9640 1 -- 172.21.15.42:0/3352296353 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8097670 con 0x55fec97becf0 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 -- 172.21.15.42:0/3352296353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 msgr2=0x55fec97bb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97bb0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8099a00 tx=0x7fc2d8004b50 comp rx=0 tx=0).stop 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 -- 172.21.15.42:0/3352296353 shutdown_connections 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97becf0 0x55fec97bb0d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 --2- 172.21.15.42:0/3352296353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97b9190 0x55fec97d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 -- 172.21.15.42:0/3352296353 >> 172.21.15.42:0/3352296353 conn(0x55fec97b24f0 msgr2=0x55fec97b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 -- 172.21.15.42:0/3352296353 shutdown_connections 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.213+0000 7fc2efd92740 1 -- 172.21.15.42:0/3352296353 wait complete. 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 -- start start 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 Processor -- start 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec97be8a0 con 0x55fec97b9190 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec97c1310 con 0x55fec97becf0 2024-03-12T04:14:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec97c93e0 con 0x55fec97b9b80 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2efd92740 10 client.0 _ll_register_callbacks cb 0x7fc2dc00c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37708/0 (socket says 172.21.15.42:37708) 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 -- 172.21.15.42:0/790634574 learned_addr learned my addr 172.21.15.42:0/790634574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e659c640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e759e640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 -- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 msgr2=0x55fec97c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 -- 172.21.15.42:0/790634574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 msgr2=0x55fec97c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 -- 172.21.15.42:0/790634574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fec971d2b0 con 0x55fec97b9190 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e759e640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2e6d9d640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97c7920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009230 tx=0x7fc2e002cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead479de9bcbd8d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.214+0000 7fc2caff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2caff5640 1 -- 172.21.15.42:0/790634574 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0004ad0 con 0x55fec97b9190 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2caff5640 1 -- 172.21.15.42:0/790634574 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc2e0044040 con 0x55fec97b9190 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fec97cf8f0 con 0x55fec97b9190 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2caff5640 1 -- 172.21.15.42:0/790634574 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e0047be0 con 0x55fec97b9190 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 10 client.15084 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fec96df780 con 0x55fec97b9190 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2caff5640 1 -- 172.21.15.42:0/790634574 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55fec97cf8f0 con 0x55fec97b9190 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2caff5640 1 -- 172.21.15.42:0/790634574 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2e003c040 con 0x55fec97b9190 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 10 client.15084 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 10 client.15084 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 20 client.15084 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec982b160 0x55fec98277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 4 client.15084 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.215+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fc2dc018e90 con 0x55fec982b160 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2e659c640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec982b160 0x55fec98277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2caff5640 10 client.15084 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2e659c640 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec982b160 0x55fec98277f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c43626126d4c084 server_cookie=216c034deeea427b in_seq=0 out_seq=0 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2caff5640 1 -- 172.21.15.42:0/790634574 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7fc2dc018e90 con 0x55fec982b160 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2caff5640 10 client.15084 handle_command_reply: tid=0 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 client.15084 shutdown 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 msgr2=0x55fec97c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97c7920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009230 tx=0x7fc2e002cb60 comp rx=0 tx=0).stop 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 shutdown_connections 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55fec982b160 0x55fec98277f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fec97becf0 0x55fec97c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2caff5640 0 client.15084 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fec97b9b80 0x55fec97c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 --2- 172.21.15.42:0/790634574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec97b9190 0x55fec97c7920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 >> 172.21.15.42:0/790634574 conn(0x55fec97b24f0 msgr2=0x55fec97d8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 shutdown_connections 2024-03-12T04:14:23.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.216+0000 7fc2efd92740 1 -- 172.21.15.42:0/790634574 wait complete. 2024-03-12T04:14:23.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2efd92740 20 client.15084 trim_cache size 0 max 16384 2024-03-12T04:14:23.229 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 3, 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24475, 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 84.167872453000001, 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 108.67166200200001 2024-03-12T04:14:23.230 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2d00a1a30 msgr2=0x7fc2d00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2d00a1a30 0x7fc2d00a3ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc013650 tx=0x7fc2dc019000 comp rx=0 tx=0).stop 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8101c40 msgr2=0x7fc2e819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8101c40 0x7fc2e819d7f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800b970 tx=0x7fc2d800be30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 shutdown_connections 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e8101c40 0x7fc2e819d7f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc2d00a1a30 0x7fc2d00a3ef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8101280 0x7fc2e8196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 --2- 172.21.15.42:0/2328062754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e8100090 0x7fc2e8196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 >> 172.21.15.42:0/2328062754 conn(0x7fc2e80fb620 msgr2=0x7fc2e8104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 shutdown_connections 2024-03-12T04:14:23.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.217+0000 7fc2ed0df640 1 -- 172.21.15.42:0/2328062754 wait complete. 2024-03-12T04:14:23.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-12T04:14:23.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- 172.21.15.42:0/1092446826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618101c80 msgr2=0x7f9618104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 --2- 172.21.15.42:0/1092446826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618101c80 0x7f9618104070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9610009900 tx=0x7f961002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- 172.21.15.42:0/1092446826 shutdown_connections 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 --2- 172.21.15.42:0/1092446826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9618106ee0 0x7f96181092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 --2- 172.21.15.42:0/1092446826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96181045b0 0x7f96181069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 --2- 172.21.15.42:0/1092446826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618101c80 0x7f9618104070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- 172.21.15.42:0/1092446826 >> 172.21.15.42:0/1092446826 conn(0x7f96180fb630 msgr2=0x7f96180fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- 172.21.15.42:0/1092446826 shutdown_connections 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- 172.21.15.42:0/1092446826 wait complete. 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 Processor -- start 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- start start 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9618101c80 0x7f9618197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96181045b0 0x7f9618197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618106ee0 0x7f961819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961819ed40 con 0x7f9618106ee0 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961819eeb0 con 0x7f9618101c80 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.334+0000 7f961fc76640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961819f070 con 0x7f96181045b0 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618106ee0 0x7f961819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618106ee0 0x7f961819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37726/0 (socket says 172.21.15.42:37726) 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 -- 172.21.15.42:0/1350794831 learned_addr learned my addr 172.21.15.42:0/1350794831 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961da12640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9618101c80 0x7f9618197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 -- 172.21.15.42:0/1350794831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96181045b0 msgr2=0x7f9618197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96181045b0 0x7f9618197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 -- 172.21.15.42:0/1350794831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9618101c80 msgr2=0x7f9618197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9618101c80 0x7f9618197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 -- 172.21.15.42:0/1350794831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610009560 con 0x7f9618106ee0 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961e213640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618106ee0 0x7f961819e800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f961400b970 tx=0x7f961400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f960effd640 1 -- 172.21.15.42:0/1350794831 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400cb40 con 0x7f9618106ee0 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f960effd640 1 -- 172.21.15.42:0/1350794831 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f961400cca0 con 0x7f9618106ee0 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f960effd640 1 -- 172.21.15.42:0/1350794831 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140076e0 con 0x7f9618106ee0 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961819f350 con 0x7f9618106ee0 2024-03-12T04:14:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.335+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961819f930 con 0x7f9618106ee0 2024-03-12T04:14:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.337+0000 7f960effd640 1 -- 172.21.15.42:0/1350794831 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9614014440 con 0x7f9618106ee0 2024-03-12T04:14:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 Processor -- start 2024-03-12T04:14:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 -- start start 2024-03-12T04:14:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc5ef80 0x55b69dc5f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5f970 0x55b69dc64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc64ae0 0x55b69dc66f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dc1acd0 con 0x55b69dc5ef80 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dbcdf20 con 0x55b69dc64ae0 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f9622846740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dbcb450 con 0x55b69dc5f970 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f960effd640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f96040a1ab0 0x7f96040a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc5ef80 0x55b69dc5f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc5ef80 0x55b69dc5f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37728/0 (socket says 172.21.15.42:37728) 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 -- 172.21.15.42:0/1073408271 learned_addr learned my addr 172.21.15.42:0/1073408271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961da12640 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5f970 0x55b69dc64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961e213640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f96040a1ab0 0x7f96040a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc64ae0 0x55b69dc66f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f960effd640 1 -- 172.21.15.42:0/1350794831 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f96140cccc0 con 0x7f9618106ee0 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 -- 172.21.15.42:0/1073408271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5f970 msgr2=0x55b69dc64470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5f970 0x55b69dc64470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 -- 172.21.15.42:0/1073408271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc64ae0 msgr2=0x55b69dc66f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc64ae0 0x55b69dc66f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.338+0000 7f961d211640 1 -- 172.21.15.42:0/1073408271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b69db7bf20 con 0x55b69dc5ef80 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.339+0000 7f961da12640 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5f970 0x55b69dc64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.339+0000 7f961d211640 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc5ef80 0x55b69dc5f3a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f960800bfd0 tx=0x7f960800b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3886e1fedca38dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.339+0000 7f961e213640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f96040a1ab0 0x7f96040a3f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f961400b940 tx=0x7f961406cf00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.339+0000 7f9603fff640 1 -- 172.21.15.42:0/1073408271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608019070 con 0x55b69dc5ef80 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.339+0000 7f9603fff640 1 -- 172.21.15.42:0/1073408271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f960800b6b0 con 0x55b69dc5ef80 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.339+0000 7f9603fff640 1 -- 172.21.15.42:0/1073408271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608004c30 con 0x55b69dc5ef80 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- 172.21.15.42:0/1073408271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc5ef80 msgr2=0x55b69dc5f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc5ef80 0x55b69dc5f3a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f960800bfd0 tx=0x7f960800b4d0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- 172.21.15.42:0/1073408271 shutdown_connections 2024-03-12T04:14:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc64ae0 0x55b69dc66f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5f970 0x55b69dc64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 --2- 172.21.15.42:0/1073408271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc5ef80 0x55b69dc5f3a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- 172.21.15.42:0/1073408271 >> 172.21.15.42:0/1073408271 conn(0x55b69dc5a4f0 msgr2=0x55b69dc5a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- 172.21.15.42:0/1073408271 shutdown_connections 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- 172.21.15.42:0/1073408271 wait complete. 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- start start 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 Processor -- start 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc69190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc70710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dbc6c30 con 0x55b69dc64ae0 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69db6bd30 con 0x55b69dc5f970 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69d89a310 con 0x55b69dc5ef80 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.340+0000 7f9622846740 10 client.0 _ll_register_callbacks cb 0x55b69db73d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc70710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc70710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37742/0 (socket says 172.21.15.42:37742) 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 -- 172.21.15.42:0/4127756948 learned_addr learned my addr 172.21.15.42:0/4127756948 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961da12640 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc69190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 -- 172.21.15.42:0/4127756948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 msgr2=0x55b69dc69190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc69190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 -- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 msgr2=0x55b69dc696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 -- 172.21.15.42:0/4127756948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960800bc00 con 0x55b69dc64ae0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f961d211640 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc70710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9608004a60 tx=0x7f9608005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71d73d080cafb62 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f96017fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f96017fa640 1 -- 172.21.15.42:0/4127756948 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608019040 con 0x55b69dc64ae0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f96017fa640 1 -- 172.21.15.42:0/4127756948 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9608004260 con 0x55b69dc64ae0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b69d917270 con 0x55b69dc64ae0 2024-03-12T04:14:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f96017fa640 1 -- 172.21.15.42:0/4127756948 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9608004420 con 0x55b69dc64ae0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f9622846740 10 client.15096 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b69dbc52b0 con 0x55b69dc64ae0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.341+0000 7f96017fa640 1 -- 172.21.15.42:0/4127756948 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b69d917270 con 0x55b69dc64ae0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f96017fa640 1 -- 172.21.15.42:0/4127756948 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f960801f070 con 0x55b69dc64ae0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f9622846740 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f9622846740 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f9622846740 20 client.15096 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f9622846740 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dc751b0 0x55b69dc79c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f9622846740 4 client.15096 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b69d903610 con 0x55b69dc751b0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f961da12640 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dc751b0 0x55b69dc79c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f96017fa640 10 client.15096 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.342+0000 7f961da12640 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dc751b0 0x55b69dc79c50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc150baf6313a178 server_cookie=5af4a40d7aabb84c in_seq=0 out_seq=0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f96017fa640 1 -- 172.21.15.42:0/4127756948 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b69d903610 con 0x55b69dc751b0 2024-03-12T04:14:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f96017fa640 10 client.15096 handle_command_reply: tid=0 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 client.15096 shutdown 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 msgr2=0x55b69dc70710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc70710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9608004a60 tx=0x7f9608005f30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 shutdown_connections 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dc751b0 0x55b69dc79c50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc70710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f96017fa640 0 client.15096 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 --2- 172.21.15.42:0/4127756948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc69190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 >> 172.21.15.42:0/4127756948 conn(0x55b69dc5a4f0 msgr2=0x55b69dc597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 shutdown_connections 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 1 -- 172.21.15.42:0/4127756948 wait complete. 2024-03-12T04:14:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.343+0000 7f9622846740 20 client.15096 trim_cache size 0 max 16384 2024-03-12T04:14:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 Processor -- start 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 -- start start 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc7e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc70630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc58f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69db4c490 con 0x55b69dc64ae0 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dc75a00 con 0x55b69dc5f970 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f9622846740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dc5af40 con 0x55b69dc5ef80 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc58f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc58f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37744/0 (socket says 172.21.15.42:37744) 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961d211640 1 -- 172.21.15.42:0/649070882 learned_addr learned my addr 172.21.15.42:0/649070882 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961da12640 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc70630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961d211640 1 -- 172.21.15.42:0/649070882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 msgr2=0x55b69dc7e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961d211640 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc7e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961d211640 1 -- 172.21.15.42:0/649070882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 msgr2=0x55b69dc70630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.359+0000 7f961d211640 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc70630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f961d211640 1 -- 172.21.15.42:0/649070882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b69db61a20 con 0x55b69dc64ae0 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f961da12640 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc70630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f961d211640 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc58f40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9608003e60 tx=0x7f960800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c6684570b778b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f96037fe640 1 -- 172.21.15.42:0/649070882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608019040 con 0x55b69dc64ae0 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f96037fe640 1 -- 172.21.15.42:0/649070882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f960800be20 con 0x55b69dc64ae0 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f96037fe640 1 -- 172.21.15.42:0/649070882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960802e820 con 0x55b69dc64ae0 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f9622846740 1 -- 172.21.15.42:0/649070882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 msgr2=0x55b69dc58f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f9622846740 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc58f40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9608003e60 tx=0x7f960800bb40 comp rx=0 tx=0).stop 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f9622846740 1 -- 172.21.15.42:0/649070882 shutdown_connections 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f9622846740 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dc58f40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f9622846740 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc70630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f9622846740 1 --2- 172.21.15.42:0/649070882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc7e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.360+0000 7f9622846740 1 -- 172.21.15.42:0/649070882 >> 172.21.15.42:0/649070882 conn(0x55b69dc5a4f0 msgr2=0x55b69dc7dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 -- 172.21.15.42:0/649070882 shutdown_connections 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 -- 172.21.15.42:0/649070882 wait complete. 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 -- start start 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 Processor -- start 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc6d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc6da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dccb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dc715c0 con 0x55b69dc64ae0 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dc67ee0 con 0x55b69dc5f970 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b69dc67100 con 0x55b69dc5ef80 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f9622846740 10 client.0 _ll_register_callbacks cb 0x7f960800b330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dccb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f961d211640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dccb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37760/0 (socket says 172.21.15.42:37760) 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f961d211640 1 -- 172.21.15.42:0/956916096 learned_addr learned my addr 172.21.15.42:0/956916096 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f961da12640 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc6da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.361+0000 7f961d211640 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc6d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f961d211640 1 -- 172.21.15.42:0/956916096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 msgr2=0x55b69dc6d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f961d211640 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc6d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f961d211640 1 -- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 msgr2=0x55b69dc6da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f961d211640 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc6da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f961d211640 1 -- 172.21.15.42:0/956916096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b69dbc52b0 con 0x55b69dc64ae0 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f961d211640 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dccb910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f960800b300 tx=0x7f960800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2451b4df4f2d0e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f96017fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f96017fa640 1 -- 172.21.15.42:0/956916096 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960802e9d0 con 0x55b69dc64ae0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f96017fa640 1 -- 172.21.15.42:0/956916096 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9608000e40 con 0x55b69dc64ae0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b69dc75b40 con 0x55b69dc64ae0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f96017fa640 1 -- 172.21.15.42:0/956916096 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9608004530 con 0x55b69dc64ae0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 10 client.15102 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b69db87780 con 0x55b69dc64ae0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f96017fa640 1 -- 172.21.15.42:0/956916096 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9608049080 con 0x55b69dc64ae0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f96017fa640 1 -- 172.21.15.42:0/956916096 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9608043030 con 0x55b69dc64ae0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 10 client.15102 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 20 client.15102 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dcd0970 0x55b69dccce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 4 client.15102 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.362+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b69d8c66c0 con 0x55b69dcd0970 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f961da12640 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dcd0970 0x55b69dccce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f96017fa640 10 client.15102 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f961da12640 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dcd0970 0x55b69dccce70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0d43db578f11da9 server_cookie=46ef2f91e67cc590 in_seq=0 out_seq=0 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f96017fa640 1 -- 172.21.15.42:0/956916096 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b69d8c66c0 con 0x55b69dcd0970 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f96017fa640 10 client.15102 handle_command_reply: tid=0 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 client.15102 shutdown 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 msgr2=0x55b69dccb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dccb910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f960800b300 tx=0x7f960800b990 comp rx=0 tx=0).stop 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 shutdown_connections 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b69dcd0970 0x55b69dccce70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b69dc64ae0 0x55b69dccb910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b69dc5f970 0x55b69dc6da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 --2- 172.21.15.42:0/956916096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b69dc5ef80 0x55b69dc6d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f96017fa640 0 client.15102 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 >> 172.21.15.42:0/956916096 conn(0x55b69dc5a4f0 msgr2=0x55b69dc7e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 shutdown_connections 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.363+0000 7f9622846740 1 -- 172.21.15.42:0/956916096 wait complete. 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f9622846740 20 client.15102 trim_cache size 0 max 16384 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24485, 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:23.374 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 85.325739869000003, 2024-03-12T04:14:23.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:23.375 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:23.375 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:23.375 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 105.916083461 2024-03-12T04:14:23.375 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f96040a1ab0 msgr2=0x7f96040a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f96040a1ab0 0x7f96040a3f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f961400b940 tx=0x7f961406cf00 comp rx=0 tx=0).stop 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618106ee0 msgr2=0x7f961819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618106ee0 0x7f961819e800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f961400b970 tx=0x7f961400be30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 shutdown_connections 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9618106ee0 0x7f961819e800 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96181045b0 0x7f9618197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f96040a1ab0 0x7f96040a3f70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 --2- 172.21.15.42:0/1350794831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9618101c80 0x7f9618197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 >> 172.21.15.42:0/1350794831 conn(0x7f96180fb630 msgr2=0x7f96181025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 shutdown_connections 2024-03-12T04:14:23.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.364+0000 7f961fc76640 1 -- 172.21.15.42:0/1350794831 wait complete. 2024-03-12T04:14:23.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-12T04:14:23.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 -- 172.21.15.42:0/1148278972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101280 msgr2=0x7f02c4101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 --2- 172.21.15.42:0/1148278972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101280 0x7f02c4101700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009900 tx=0x7f02b002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 -- 172.21.15.42:0/1148278972 shutdown_connections 2024-03-12T04:14:23.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 --2- 172.21.15.42:0/1148278972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c4101c40 0x7f02c41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 --2- 172.21.15.42:0/1148278972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101280 0x7f02c4101700 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 --2- 172.21.15.42:0/1148278972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02c4100090 0x7f02c41004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 -- 172.21.15.42:0/1148278972 >> 172.21.15.42:0/1148278972 conn(0x7f02c40fb620 msgr2=0x7f02c40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 -- 172.21.15.42:0/1148278972 shutdown_connections 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 -- 172.21.15.42:0/1148278972 wait complete. 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.478+0000 7f02cb23b640 1 Processor -- start 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 -- start start 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c4100090 0x7f02c4103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02c4101280 0x7f02c4102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101c40 0x7f02c4102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c4103330 con 0x7f02c4101c40 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41034a0 con 0x7f02c4100090 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a12c0 con 0x7f02c4101280 2024-03-12T04:14:23.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101c40 0x7f02c4102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101c40 0x7f02c4102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37772/0 (socket says 172.21.15.42:37772) 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 -- 172.21.15.42:0/3929497640 learned_addr learned my addr 172.21.15.42:0/3929497640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 -- 172.21.15.42:0/3929497640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02c4101280 msgr2=0x7f02c4102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02c4101280 0x7f02c4102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 -- 172.21.15.42:0/3929497640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c4100090 msgr2=0x7f02c4103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c4100090 0x7f02c4103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c4100090 0x7f02c4103fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 -- 172.21.15.42:0/3929497640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0009560 con 0x7f02c4101c40 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c4100090 0x7f02c4103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c97d8640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101c40 0x7f02c4102ba0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00cb00 tx=0x7f02bc00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c1ffb640 1 -- 172.21.15.42:0/3929497640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00ae90 con 0x7f02c4101c40 2024-03-12T04:14:23.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c41a1520 con 0x7f02c4101c40 2024-03-12T04:14:23.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c1ffb640 1 -- 172.21.15.42:0/3929497640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc005020 con 0x7f02c4101c40 2024-03-12T04:14:23.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.479+0000 7f02c1ffb640 1 -- 172.21.15.42:0/3929497640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00f690 con 0x7f02c4101c40 2024-03-12T04:14:23.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.480+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41a1a60 con 0x7f02c4101c40 2024-03-12T04:14:23.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.481+0000 7f02c1ffb640 1 -- 172.21.15.42:0/3929497640 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f02bc005190 con 0x7f02c4101c40 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 Processor -- start 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 -- start start 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02c1ffb640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f02b40a1a30 0x7f02b40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02c3fff640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f02b40a1a30 0x7f02b40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e1ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688dd3cd0 con 0x55a688e1dae0 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688d86f20 con 0x55a688e18970 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02cde05740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688d84450 con 0x55a688e17f80 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.482+0000 7f02c1ffb640 1 -- 172.21.15.42:0/3929497640 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f02bc0cd520 con 0x7f02c4101c40 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e1ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e1ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37778/0 (socket says 172.21.15.42:37778) 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 -- 172.21.15.42:0/2784595345 learned_addr learned my addr 172.21.15.42:0/2784595345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c3fff640 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 -- 172.21.15.42:0/2784595345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 msgr2=0x55a688e183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e183a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 -- 172.21.15.42:0/2784595345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 msgr2=0x55a688e1d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 -- 172.21.15.42:0/2784595345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a688d34f20 con 0x55a688e1dae0 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c3fff640 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c3fff640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f02b40a1a30 0x7f02b40a3ef0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02b0036040 tx=0x7f02b00047a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02c97d8640 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e1ff70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0a2710 tx=0x7f02bc09aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40461b54746e4cf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02aeffd640 1 -- 172.21.15.42:0/2784595345 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc09e5f0 con 0x55a688e1dae0 2024-03-12T04:14:23.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02aeffd640 1 -- 172.21.15.42:0/2784595345 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc0a2c40 con 0x55a688e1dae0 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.483+0000 7f02aeffd640 1 -- 172.21.15.42:0/2784595345 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc096b10 con 0x55a688e1dae0 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 -- 172.21.15.42:0/2784595345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 msgr2=0x55a688e1ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e1ff70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0a2710 tx=0x7f02bc09aba0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 -- 172.21.15.42:0/2784595345 shutdown_connections 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e1ff70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 --2- 172.21.15.42:0/2784595345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 -- 172.21.15.42:0/2784595345 >> 172.21.15.42:0/2784595345 conn(0x55a688e134f0 msgr2=0x55a688e138f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 -- 172.21.15.42:0/2784595345 shutdown_connections 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 -- 172.21.15.42:0/2784595345 wait complete. 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 -- start start 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.484+0000 7f02cde05740 1 Processor -- start 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e22100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e18970 0x55a688e22640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e1dae0 0x55a688e29680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688d7fc30 con 0x55a688e18970 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688d24d30 con 0x55a688e1dae0 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688a53310 con 0x55a688e17f80 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 10 client.0 _ll_register_callbacks cb 0x55a688d2cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e18970 0x55a688e22640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e18970 0x55a688e22640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37790/0 (socket says 172.21.15.42:37790) 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 -- 172.21.15.42:0/1137857451 learned_addr learned my addr 172.21.15.42:0/1137857451 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c97d8640 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e1dae0 0x55a688e29680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 -- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 msgr2=0x55a688e22100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e22100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 -- 172.21.15.42:0/1137857451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e1dae0 msgr2=0x55a688e29680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e1dae0 0x55a688e29680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 -- 172.21.15.42:0/1137857451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41a1a60 con 0x55a688e18970 2024-03-12T04:14:23.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02c3fff640 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e18970 0x55a688e22640 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f02b0035000 tx=0x7f02b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f89d6d106c76a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02a3fff640 1 -- 172.21.15.42:0/1137857451 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0003d70 con 0x55a688e18970 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a688a7f6c0 con 0x55a688e18970 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.485+0000 7f02a3fff640 1 -- 172.21.15.42:0/1137857451 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f02b0004dc0 con 0x55a688e18970 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02a3fff640 1 -- 172.21.15.42:0/1137857451 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b0004f80 con 0x55a688e18970 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 10 client.15114 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a688d7e2b0 con 0x55a688e18970 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02a3fff640 1 -- 172.21.15.42:0/1137857451 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a688a7f6c0 con 0x55a688e18970 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02a3fff640 1 -- 172.21.15.42:0/1137857451 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02b00438c0 con 0x55a688e18970 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 20 client.15114 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e2eaa0 0x55a688e2dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 4 client.15114 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a688ad0270 con 0x55a688e2eaa0 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.486+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e2eaa0 0x55a688e2dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02a3fff640 10 client.15114 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e2eaa0 0x55a688e2dd50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bea887635a01de05 server_cookie=5462c21eede6ee4 in_seq=0 out_seq=0 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02a3fff640 1 -- 172.21.15.42:0/1137857451 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a688ad0270 con 0x55a688e2eaa0 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02a3fff640 10 client.15114 handle_command_reply: tid=0 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 client.15114 shutdown 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e18970 msgr2=0x55a688e22640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e18970 0x55a688e22640 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f02b0035000 tx=0x7f02b0005fd0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 shutdown_connections 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e2eaa0 0x55a688e2dd50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e1dae0 0x55a688e29680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02a3fff640 0 client.15114 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e18970 0x55a688e22640 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 --2- 172.21.15.42:0/1137857451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e22100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.487+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 >> 172.21.15.42:0/1137857451 conn(0x55a688e134f0 msgr2=0x55a688e127d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.488+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 shutdown_connections 2024-03-12T04:14:23.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.488+0000 7f02cde05740 1 -- 172.21.15.42:0/1137857451 wait complete. 2024-03-12T04:14:23.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.488+0000 7f02cde05740 20 client.15114 trim_cache size 0 max 16384 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.503+0000 7f02cde05740 1 Processor -- start 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.503+0000 7f02cde05740 1 -- start start 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02cde05740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e19e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02cde05740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02cde05740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e29640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02cde05740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688d05490 con 0x55a688e1dae0 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02cde05740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688ad0270 con 0x55a688e18970 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02cde05740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688a7f6c0 con 0x55a688e17f80 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e29640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e29640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37796/0 (socket says 172.21.15.42:37796) 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 -- 172.21.15.42:0/1257645845 learned_addr learned my addr 172.21.15.42:0/1257645845 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e19e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c3fff640 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 -- 172.21.15.42:0/1257645845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 msgr2=0x55a688e19e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e19e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 -- 172.21.15.42:0/1257645845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 msgr2=0x55a688e1a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 -- 172.21.15.42:0/1257645845 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a688d1aa20 con 0x55a688e1dae0 2024-03-12T04:14:23.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.504+0000 7f02c97d8640 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e29640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f02bc096900 tx=0x7f02bc00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da67e92b4e5e1998 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02ae7fc640 1 -- 172.21.15.42:0/1257645845 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0ba0c0 con 0x55a688e1dae0 2024-03-12T04:14:23.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02ae7fc640 1 -- 172.21.15.42:0/1257645845 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc098540 con 0x55a688e1dae0 2024-03-12T04:14:23.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02ae7fc640 1 -- 172.21.15.42:0/1257645845 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0cd0d0 con 0x55a688e1dae0 2024-03-12T04:14:23.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02cde05740 1 -- 172.21.15.42:0/1257645845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 msgr2=0x55a688e29640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02cde05740 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e29640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f02bc096900 tx=0x7f02bc00a8f0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02cde05740 1 -- 172.21.15.42:0/1257645845 shutdown_connections 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02cde05740 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e29640 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02cde05740 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e1a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02cde05740 1 --2- 172.21.15.42:0/1257645845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e19e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.507+0000 7f02cde05740 1 -- 172.21.15.42:0/1257645845 >> 172.21.15.42:0/1257645845 conn(0x55a688e134f0 msgr2=0x55a688e36f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 -- 172.21.15.42:0/1257645845 shutdown_connections 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 -- 172.21.15.42:0/1257645845 wait complete. 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 -- start start 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 Processor -- start 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e26820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688e29840 con 0x55a688e1dae0 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688e299b0 con 0x55a688e18970 2024-03-12T04:14:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a688e84be0 con 0x55a688e17f80 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.508+0000 7f02cde05740 10 client.0 _ll_register_callbacks cb 0x7f02bc096100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e846a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37810/0 (socket says 172.21.15.42:37810) 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 -- 172.21.15.42:0/3344854146 learned_addr learned my addr 172.21.15.42:0/3344854146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c3fff640 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e26820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 -- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 msgr2=0x55a688e262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e262e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 -- 172.21.15.42:0/3344854146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 msgr2=0x55a688e26820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e26820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 -- 172.21.15.42:0/3344854146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a688d34f20 con 0x55a688e1dae0 2024-03-12T04:14:23.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02c97d8640 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e846a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00cad0 tx=0x7f02bc09e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5175af36c4b2bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.509+0000 7f02a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02a3fff640 1 -- 172.21.15.42:0/3344854146 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0cd0d0 con 0x55a688e1dae0 2024-03-12T04:14:23.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02a3fff640 1 -- 172.21.15.42:0/3344854146 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f02bc0cd230 con 0x55a688e1dae0 2024-03-12T04:14:23.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a688e2e7b0 con 0x55a688e1dae0 2024-03-12T04:14:23.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02a3fff640 1 -- 172.21.15.42:0/3344854146 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02bc09e120 con 0x55a688e1dae0 2024-03-12T04:14:23.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 10 client.15120 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a688d40780 con 0x55a688e1dae0 2024-03-12T04:14:23.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02a3fff640 1 -- 172.21.15.42:0/3344854146 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc0a2e90 con 0x55a688e1dae0 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02a3fff640 1 -- 172.21.15.42:0/3344854146 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02bc016030 con 0x55a688e1dae0 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 10 client.15120 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 10 client.15120 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 20 client.15120 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e89900 0x55a688e85da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 4 client.15120 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a688e2fad0 con 0x55a688e89900 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.511+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e89900 0x55a688e85da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02a3fff640 10 client.15120 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02c8fd7640 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e89900 0x55a688e85da0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ff9058a6bd23e74 server_cookie=dec849c3e3a813aa in_seq=0 out_seq=0 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02a3fff640 1 -- 172.21.15.42:0/3344854146 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a688e2fad0 con 0x55a688e89900 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02a3fff640 10 client.15120 handle_command_reply: tid=0 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 client.15120 shutdown 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 msgr2=0x55a688e846a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e846a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00cad0 tx=0x7f02bc09e6c0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 shutdown_connections 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55a688e89900 0x55a688e85da0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a688e1dae0 0x55a688e846a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a688e18970 0x55a688e26820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02a3fff640 0 client.15120 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 --2- 172.21.15.42:0/3344854146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a688e17f80 0x55a688e262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 >> 172.21.15.42:0/3344854146 conn(0x55a688e134f0 msgr2=0x55a688e19e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 shutdown_connections 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.512+0000 7f02cde05740 1 -- 172.21.15.42:0/3344854146 wait complete. 2024-03-12T04:14:23.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cde05740 20 client.15120 trim_cache size 0 max 16384 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24490, 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 86.481567565999995, 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 103.249891799 2024-03-12T04:14:23.519 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:23.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f02b40a1a30 msgr2=0x7f02b40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f02b40a1a30 0x7f02b40a3ef0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02b0036040 tx=0x7f02b00047a0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101c40 msgr2=0x7f02c4102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101c40 0x7f02c4102ba0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00cb00 tx=0x7f02bc00a250 comp rx=0 tx=0).stop 2024-03-12T04:14:23.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 shutdown_connections 2024-03-12T04:14:23.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02c4101c40 0x7f02c4102ba0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02c4101280 0x7f02c4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f02b40a1a30 0x7f02b40a3ef0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 --2- 172.21.15.42:0/3929497640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c4100090 0x7f02c4103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 >> 172.21.15.42:0/3929497640 conn(0x7f02c40fb620 msgr2=0x7f02c4069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 shutdown_connections 2024-03-12T04:14:23.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.513+0000 7f02cb23b640 1 -- 172.21.15.42:0/3929497640 wait complete. 2024-03-12T04:14:23.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-12T04:14:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 -- 172.21.15.42:0/2966733821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8101c20 msgr2=0x7face8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 --2- 172.21.15.42:0/2966733821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8101c20 0x7face8104010 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7face4009900 tx=0x7face402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 -- 172.21.15.42:0/2966733821 shutdown_connections 2024-03-12T04:14:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 --2- 172.21.15.42:0/2966733821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8106e80 0x7face81092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 --2- 172.21.15.42:0/2966733821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face8104550 0x7face8106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 --2- 172.21.15.42:0/2966733821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8101c20 0x7face8104010 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 -- 172.21.15.42:0/2966733821 >> 172.21.15.42:0/2966733821 conn(0x7face80fb630 msgr2=0x7face80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 -- 172.21.15.42:0/2966733821 shutdown_connections 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.627+0000 7facf0081640 1 -- 172.21.15.42:0/2966733821 wait complete. 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 Processor -- start 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 -- start start 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8101c20 0x7face81994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face8104550 0x7face81999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8106e80 0x7face81a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face81a0f90 con 0x7face8106e80 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face81a1100 con 0x7face8101c20 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facf0081640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face81a12c0 con 0x7face8104550 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8106e80 0x7face81a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8106e80 0x7face81a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37828/0 (socket says 172.21.15.42:37828) 2024-03-12T04:14:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 -- 172.21.15.42:0/846902655 learned_addr learned my addr 172.21.15.42:0/846902655 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facede1d640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8101c20 0x7face81994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7faced61c640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face8104550 0x7face81999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 -- 172.21.15.42:0/846902655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face8104550 msgr2=0x7face81999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face8104550 0x7face81999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 -- 172.21.15.42:0/846902655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8101c20 msgr2=0x7face81994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8101c20 0x7face81994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 -- 172.21.15.42:0/846902655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009560 con 0x7face8106e80 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.628+0000 7facee61e640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8106e80 0x7face81a0a50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7face000b970 tx=0x7face000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.629+0000 7facdeffd640 1 -- 172.21.15.42:0/846902655 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face000cb40 con 0x7face8106e80 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.629+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face81a15a0 con 0x7face8106e80 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.629+0000 7facdeffd640 1 -- 172.21.15.42:0/846902655 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7face000cca0 con 0x7face8106e80 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.629+0000 7facdeffd640 1 -- 172.21.15.42:0/846902655 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face00076e0 con 0x7face8106e80 2024-03-12T04:14:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.629+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face81a1ae0 con 0x7face8106e80 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facdeffd640 1 -- 172.21.15.42:0/846902655 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7face0014440 con 0x7face8106e80 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 Processor -- start 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 -- start start 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f09470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f09ae0 0x5600b3f0bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3ebfcd0 con 0x5600b3f03f80 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3e72f20 con 0x5600b3f09ae0 2024-03-12T04:14:23.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facf2c50740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3e70450 con 0x5600b3f04970 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7faced61c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.631+0000 7facdeffd640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7facd40a1a30 0x7facd40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37842/0 (socket says 172.21.15.42:37842) 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 -- 172.21.15.42:0/1892844889 learned_addr learned my addr 172.21.15.42:0/1892844889 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7facee61e640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7facd40a1a30 0x7facd40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f09ae0 0x5600b3f0bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7facede1d640 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f09470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7facdeffd640 1 -- 172.21.15.42:0/846902655 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7face00ccd40 con 0x7face8106e80 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 -- 172.21.15.42:0/1892844889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 msgr2=0x5600b3f09470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f09470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 -- 172.21.15.42:0/1892844889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f09ae0 msgr2=0x5600b3f0bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f09ae0 0x5600b3f0bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 -- 172.21.15.42:0/1892844889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600b3e20f20 con 0x5600b3f03f80 2024-03-12T04:14:23.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7facee61e640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7facd40a1a30 0x7facd40a3ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7face00966b0 tx=0x7face0069110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7faced61c640 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f043a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7facd800cf10 tx=0x7facd800c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d38d2cda673c43 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7facd3fff640 1 -- 172.21.15.42:0/1892844889 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8013700 con 0x5600b3f03f80 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7facd3fff640 1 -- 172.21.15.42:0/1892844889 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7facd8013d10 con 0x5600b3f03f80 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.632+0000 7facd3fff640 1 -- 172.21.15.42:0/1892844889 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8018a70 con 0x5600b3f03f80 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 -- 172.21.15.42:0/1892844889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 msgr2=0x5600b3f043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f043a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7facd800cf10 tx=0x7facd800c170 comp rx=0 tx=0).stop 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 -- 172.21.15.42:0/1892844889 shutdown_connections 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f09ae0 0x5600b3f0bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f09470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 --2- 172.21.15.42:0/1892844889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f043a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 -- 172.21.15.42:0/1892844889 >> 172.21.15.42:0/1892844889 conn(0x5600b3eff4f0 msgr2=0x5600b3eff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 -- 172.21.15.42:0/1892844889 shutdown_connections 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 -- 172.21.15.42:0/1892844889 wait complete. 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 -- start start 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.633+0000 7facf2c50740 1 Processor -- start 2024-03-12T04:14:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f03f80 0x5600b3f0e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f04970 0x5600b3f0e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f09ae0 0x5600b3f156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3e6bc30 con 0x5600b3f04970 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3e10d30 con 0x5600b3f03f80 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3b3f310 con 0x5600b3f09ae0 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 10 client.0 _ll_register_callbacks cb 0x5600b3e18d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f04970 0x5600b3f0e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f04970 0x5600b3f0e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37854/0 (socket says 172.21.15.42:37854) 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 -- 172.21.15.42:0/1373514222 learned_addr learned my addr 172.21.15.42:0/1373514222 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7faced61c640 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f09ae0 0x5600b3f156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 -- 172.21.15.42:0/1373514222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f09ae0 msgr2=0x5600b3f156f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f09ae0 0x5600b3f156f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 -- 172.21.15.42:0/1373514222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f03f80 msgr2=0x5600b3f0e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f03f80 0x5600b3f0e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 -- 172.21.15.42:0/1373514222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd8015300 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7faced61c640 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f09ae0 0x5600b3f156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facede1d640 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f04970 0x5600b3f0e6b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7face4009900 tx=0x7face40049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d313f20a38c8f53 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facd17fa640 1 -- 172.21.15.42:0/1373514222 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4004250 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facd17fa640 1 -- 172.21.15.42:0/1373514222 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7face40043b0 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.634+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600b3b6b6c0 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facd17fa640 1 -- 172.21.15.42:0/1373514222 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7face40035f0 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 10 client.15132 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5600b3e6a2b0 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facd17fa640 1 -- 172.21.15.42:0/1373514222 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5600b3b6b6c0 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facd17fa640 1 -- 172.21.15.42:0/1373514222 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7face403c040 con 0x5600b3f04970 2024-03-12T04:14:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 10 client.15132 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 20 client.15132 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f1ab60 0x5600b3f19dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 4 client.15132 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600b3bbc270 con 0x5600b3f1ab60 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.635+0000 7faced61c640 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f1ab60 0x5600b3f19dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.636+0000 7facd17fa640 10 client.15132 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.636+0000 7faced61c640 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f1ab60 0x5600b3f19dc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd1dc1f4e4ed254 server_cookie=a7e7d470aabbb370 in_seq=0 out_seq=0 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.636+0000 7facd17fa640 1 -- 172.21.15.42:0/1373514222 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600b3bbc270 con 0x5600b3f1ab60 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.636+0000 7facd17fa640 10 client.15132 handle_command_reply: tid=0 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 client.15132 shutdown 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f04970 msgr2=0x5600b3f0e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f04970 0x5600b3f0e6b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7face4009900 tx=0x7face40049e0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 shutdown_connections 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f1ab60 0x5600b3f19dc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f09ae0 0x5600b3f156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f04970 0x5600b3f0e6b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facd17fa640 0 client.15132 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 --2- 172.21.15.42:0/1373514222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f03f80 0x5600b3f0e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 >> 172.21.15.42:0/1373514222 conn(0x5600b3eff4f0 msgr2=0x5600b3efe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 shutdown_connections 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 1 -- 172.21.15.42:0/1373514222 wait complete. 2024-03-12T04:14:23.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.637+0000 7facf2c50740 20 client.15132 trim_cache size 0 max 16384 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.652+0000 7facf2c50740 1 Processor -- start 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.652+0000 7facf2c50740 1 -- start start 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facf2c50740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3eff4f0 0x5600b3f0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facf2c50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f03f80 0x5600b3f156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facf2c50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f04970 0x5600b3f0e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facf2c50740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3df1490 con 0x5600b3eff4f0 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facf2c50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3f03b40 con 0x5600b3f04970 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facf2c50740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3f004f0 con 0x5600b3f03f80 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3eff4f0 0x5600b3f0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3eff4f0 0x5600b3f0bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37868/0 (socket says 172.21.15.42:37868) 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 -- 172.21.15.42:0/1450186870 learned_addr learned my addr 172.21.15.42:0/1450186870 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 -- 172.21.15.42:0/1450186870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f03f80 msgr2=0x5600b3f156b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 --2- 172.21.15.42:0/1450186870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f03f80 0x5600b3f156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 -- 172.21.15.42:0/1450186870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f04970 msgr2=0x5600b3f0e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 --2- 172.21.15.42:0/1450186870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f04970 0x5600b3f0e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 -- 172.21.15.42:0/1450186870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600b3e06a20 con 0x5600b3eff4f0 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7faced61c640 1 --2- 172.21.15.42:0/1450186870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3eff4f0 0x5600b3f0bed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5600b3f051d0 tx=0x7facd80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c80b5b53a42881 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facd37fe640 1 -- 172.21.15.42:0/1450186870 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8028ea0 con 0x5600b3eff4f0 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facd37fe640 1 -- 172.21.15.42:0/1450186870 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7facd80139b0 con 0x5600b3eff4f0 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.653+0000 7facd37fe640 1 -- 172.21.15.42:0/1450186870 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8028860 con 0x5600b3eff4f0 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 -- 172.21.15.42:0/1450186870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3eff4f0 msgr2=0x5600b3f0bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 --2- 172.21.15.42:0/1450186870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3eff4f0 0x5600b3f0bed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5600b3f051d0 tx=0x7facd80047d0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 -- 172.21.15.42:0/1450186870 shutdown_connections 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 --2- 172.21.15.42:0/1450186870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3f04970 0x5600b3f0e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 --2- 172.21.15.42:0/1450186870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f03f80 0x5600b3f156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 --2- 172.21.15.42:0/1450186870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3eff4f0 0x5600b3f0bed0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 -- 172.21.15.42:0/1450186870 >> 172.21.15.42:0/1450186870 conn(0x5600b3f05750 msgr2=0x5600b3eff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 -- 172.21.15.42:0/1450186870 shutdown_connections 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 -- 172.21.15.42:0/1450186870 wait complete. 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 -- start start 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.654+0000 7facf2c50740 1 Processor -- start 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3eff4f0 0x5600b3f12670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f12bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3f70ca0 con 0x5600b3f03f80 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3f70de0 con 0x5600b3eff4f0 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b3f70f50 con 0x5600b3f04970 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 10 client.0 _ll_register_callbacks cb 0x7facd800c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f12bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f12bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37882/0 (socket says 172.21.15.42:37882) 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 -- 172.21.15.42:0/845574791 learned_addr learned my addr 172.21.15.42:0/845574791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7faced61c640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3eff4f0 0x5600b3f12670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7faced61c640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 -- 172.21.15.42:0/845574791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 msgr2=0x5600b3f70760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f70760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 -- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3eff4f0 msgr2=0x5600b3f12670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3eff4f0 0x5600b3f12670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 -- 172.21.15.42:0/845574791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600b3e20f20 con 0x5600b3f03f80 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7faced61c640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facede1d640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f12bb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7face4034440 tx=0x7face40056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99849a4bbb4be6fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facd17fa640 1 -- 172.21.15.42:0/845574791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4004fd0 con 0x5600b3f03f80 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facd17fa640 1 -- 172.21.15.42:0/845574791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7face4004250 con 0x5600b3f03f80 2024-03-12T04:14:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facd8013d10 con 0x5600b3f03f80 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facd17fa640 1 -- 172.21.15.42:0/845574791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7face4004410 con 0x5600b3f03f80 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 10 client.15138 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.655+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5600b3e2c780 con 0x5600b3f03f80 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facd17fa640 1 -- 172.21.15.42:0/845574791 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7face4061020 con 0x5600b3f03f80 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facd17fa640 1 -- 172.21.15.42:0/845574791 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7face4034500 con 0x5600b3f03f80 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facf2c50740 10 client.15138 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facf2c50740 10 client.15138 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facf2c50740 20 client.15138 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facf2c50740 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f09ae0 0x5600b3f72470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facf2c50740 4 client.15138 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5600b3ba8610 con 0x5600b3f09ae0 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.656+0000 7faced61c640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f09ae0 0x5600b3f72470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7faced61c640 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f09ae0 0x5600b3f72470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2ab7ed8079184a server_cookie=6467ac439320053f in_seq=0 out_seq=0 2024-03-12T04:14:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facd17fa640 10 client.15138 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facd17fa640 1 -- 172.21.15.42:0/845574791 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5600b3ba8610 con 0x5600b3f09ae0 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facd17fa640 10 client.15138 handle_command_reply: tid=0 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 client.15138 shutdown 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 msgr2=0x5600b3f12bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f12bb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7face4034440 tx=0x7face40056b0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 shutdown_connections 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600b3f09ae0 0x5600b3f72470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b3f04970 0x5600b3f70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600b3f03f80 0x5600b3f12bb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facd17fa640 0 client.15138 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 --2- 172.21.15.42:0/845574791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600b3eff4f0 0x5600b3f12670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 >> 172.21.15.42:0/845574791 conn(0x5600b3f05750 msgr2=0x5600b3f05e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 shutdown_connections 2024-03-12T04:14:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 1 -- 172.21.15.42:0/845574791 wait complete. 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.657+0000 7facf2c50740 20 client.15138 trim_cache size 0 max 16384 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24515, 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 88.896723473999998, 2024-03-12T04:14:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 94.703109792999996 2024-03-12T04:14:23.668 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7facd40a1a30 msgr2=0x7facd40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7facd40a1a30 0x7facd40a3ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7face00966b0 tx=0x7face0069110 comp rx=0 tx=0).stop 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8106e80 msgr2=0x7face81a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8106e80 0x7face81a0a50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7face000b970 tx=0x7face000be30 comp rx=0 tx=0).stop 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 shutdown_connections 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face8106e80 0x7face81a0a50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face8104550 0x7face81999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7facd40a1a30 0x7facd40a3ef0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 --2- 172.21.15.42:0/846902655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8101c20 0x7face81994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 >> 172.21.15.42:0/846902655 conn(0x7face80fb630 msgr2=0x7face8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:23.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 shutdown_connections 2024-03-12T04:14:23.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:23.658+0000 7facf0081640 1 -- 172.21.15.42:0/846902655 wait complete. 2024-03-12T04:14:23.671 INFO:tasks.cephadm:Setup complete, yielding 2024-03-12T04:14:23.671 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-12T04:14:23.683 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T04:14:23.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- bash -c 'ceph orch status' 2024-03-12T04:14:23.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:23 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/86260374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:23.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:23 smithi042 ceph-mon[32260]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:23.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:23 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1635271190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:23.924 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:23.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:23 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/86260374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:23.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:23 smithi119 ceph-mon[36119]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:23.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:23 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1635271190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:23.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:23 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/86260374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:23.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:23 smithi136 ceph-mon[35571]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:23.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:23 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1635271190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:25.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:24 smithi042 ceph-mon[32260]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:14:25.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:24 smithi119 ceph-mon[36119]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:14:25.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:24 smithi136 ceph-mon[35571]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:14:25.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.720+0000 7fe76aab4700 1 -- 172.21.15.42:0/2461746434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640ffd20 msgr2=0x7fe764100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2461746434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640ffd20 0x7fe764100180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe750009990 tx=0x7fe750009ca0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 -- 172.21.15.42:0/2461746434 shutdown_connections 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2461746434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7641006c0 0x7fe76410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2461746434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7641032a0 0x7fe7641036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2461746434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640ffd20 0x7fe764100180 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 -- 172.21.15.42:0/2461746434 >> 172.21.15.42:0/2461746434 conn(0x7fe7640fb8c0 msgr2=0x7fe7640fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 -- 172.21.15.42:0/2461746434 shutdown_connections 2024-03-12T04:14:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 -- 172.21.15.42:0/2461746434 wait complete. 2024-03-12T04:14:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 Processor -- start 2024-03-12T04:14:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.721+0000 7fe76aab4700 1 -- start start 2024-03-12T04:14:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe76aab4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640ffd20 0x7fe764197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe76aab4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7641006c0 0x7fe7641980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe76aab4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7641032a0 0x7fe76419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe76aab4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76419f710 con 0x7fe7641032a0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe76aab4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76419f850 con 0x7fe7640ffd20 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe76aab4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76419f9e0 con 0x7fe7641006c0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7641032a0 0x7fe76419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7641032a0 0x7fe76419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37896/0 (socket says 172.21.15.42:37896) 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 -- 172.21.15.42:0/2128154916 learned_addr learned my addr 172.21.15.42:0/2128154916 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe763fff700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7641006c0 0x7fe7641980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe768850700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640ffd20 0x7fe764197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 -- 172.21.15.42:0/2128154916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7641006c0 msgr2=0x7fe7641980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7641006c0 0x7fe7641980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 -- 172.21.15.42:0/2128154916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640ffd20 msgr2=0x7fe764197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640ffd20 0x7fe764197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 -- 172.21.15.42:0/2128154916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7500095f0 con 0x7fe7641032a0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe769051700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7641032a0 0x7fe76419f140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007da0 tx=0x7fe75c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.722+0000 7fe761ffb700 1 -- 172.21.15.42:0/2128154916 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00ac50 con 0x7fe7641032a0 2024-03-12T04:14:25.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.723+0000 7fe761ffb700 1 -- 172.21.15.42:0/2128154916 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe75c00adb0 con 0x7fe7641032a0 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.723+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76419fcc0 con 0x7fe7641032a0 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.723+0000 7fe761ffb700 1 -- 172.21.15.42:0/2128154916 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0115e0 con 0x7fe7641032a0 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.723+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7641a01c0 con 0x7fe7641032a0 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.724+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe764109280 con 0x7fe7641032a0 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.725+0000 7fe761ffb700 1 -- 172.21.15.42:0/2128154916 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe75c012420 con 0x7fe7641032a0 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.725+0000 7fe761ffb700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7540a1b40 0x7fe7540a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.725+0000 7fe768850700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7540a1b40 0x7fe7540a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:25.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.725+0000 7fe761ffb700 1 -- 172.21.15.42:0/2128154916 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe75c0cd580 con 0x7fe7641032a0 2024-03-12T04:14:25.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.726+0000 7fe768850700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7540a1b40 0x7fe7540a4000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe76406a110 tx=0x7fe758009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:25.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.730+0000 7fe761ffb700 1 -- 172.21.15.42:0/2128154916 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe75c0967c0 con 0x7fe7641032a0 2024-03-12T04:14:25.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.951+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe76402d8e0 con 0x7fe7540a1b40 2024-03-12T04:14:25.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.953+0000 7fe761ffb700 1 -- 172.21.15.42:0/2128154916 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe76402d8e0 con 0x7fe7540a1b40 2024-03-12T04:14:25.955 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2024-03-12T04:14:25.955 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2024-03-12T04:14:25.955 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7540a1b40 msgr2=0x7fe7540a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7540a1b40 0x7fe7540a4000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe76406a110 tx=0x7fe758009190 comp rx=0 tx=0).stop 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7641032a0 msgr2=0x7fe76419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7641032a0 0x7fe76419f140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007da0 tx=0x7fe75c00a340 comp rx=0 tx=0).stop 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 shutdown_connections 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe7540a1b40 0x7fe7540a4000 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7641032a0 0x7fe76419f140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640ffd20 0x7fe764197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 --2- 172.21.15.42:0/2128154916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7641006c0 0x7fe7641980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 >> 172.21.15.42:0/2128154916 conn(0x7fe7640fb8c0 msgr2=0x7fe7641063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 shutdown_connections 2024-03-12T04:14:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:25.955+0000 7fe76aab4700 1 -- 172.21.15.42:0/2128154916 wait complete. 2024-03-12T04:14:26.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-12T04:14:27.002 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:27.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:26 smithi042 ceph-mon[32260]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:27.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:26 smithi042 ceph-mon[32260]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:27.248 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:26 smithi119 ceph-mon[36119]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:27.248 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:26 smithi119 ceph-mon[36119]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:27.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:26 smithi136 ceph-mon[35571]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:27.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:26 smithi136 ceph-mon[35571]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-12T04:14:28.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.797+0000 7f318bece700 1 -- 172.21.15.42:0/1761843915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3184101e10 msgr2=0x7f3184102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:28.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.797+0000 7f318bece700 1 --2- 172.21.15.42:0/1761843915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3184101e10 0x7f3184102270 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f31740099c0 tx=0x7f3174009cd0 comp rx=0 tx=0).stop 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.797+0000 7f318bece700 1 -- 172.21.15.42:0/1761843915 shutdown_connections 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.797+0000 7f318bece700 1 --2- 172.21.15.42:0/1761843915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31841027b0 0x7f3184109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.797+0000 7f318bece700 1 --2- 172.21.15.42:0/1761843915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3184100c10 0x7f3184101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.797+0000 7f318bece700 1 --2- 172.21.15.42:0/1761843915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3184101e10 0x7f3184102270 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.797+0000 7f318bece700 1 -- 172.21.15.42:0/1761843915 >> 172.21.15.42:0/1761843915 conn(0x7f31840fc1d0 msgr2=0x7f31840fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 -- 172.21.15.42:0/1761843915 shutdown_connections 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 -- 172.21.15.42:0/1761843915 wait complete. 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 Processor -- start 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 -- start start 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3184100c10 0x7f318406aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3184101e10 0x7f318419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31841027b0 0x7f318419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318419f790 con 0x7f31841027b0 2024-03-12T04:14:28.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318419f900 con 0x7f3184101e10 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318bece700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318419fac0 con 0x7f3184100c10 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.798+0000 7f318a46b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31841027b0 0x7f318419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31841027b0 0x7f318419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37918/0 (socket says 172.21.15.42:37918) 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 -- 172.21.15.42:0/3783214384 learned_addr learned my addr 172.21.15.42:0/3783214384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f3189c6a700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3184100c10 0x7f318406aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f3189469700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3184101e10 0x7f318419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 -- 172.21.15.42:0/3783214384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3184100c10 msgr2=0x7f318406aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3184100c10 0x7f318406aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 -- 172.21.15.42:0/3783214384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3184101e10 msgr2=0x7f318419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3184101e10 0x7f318419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 -- 172.21.15.42:0/3783214384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740095f0 con 0x7f31841027b0 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f3189469700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3184101e10 0x7f318419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f3189c6a700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3184100c10 0x7f318406aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:28.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318a46b700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31841027b0 0x7f318419eff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b970 tx=0x7f317c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:28.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f317affd700 1 -- 172.21.15.42:0/3783214384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00c9b0 con 0x7f31841027b0 2024-03-12T04:14:28.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f317affd700 1 -- 172.21.15.42:0/3783214384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f317c014440 con 0x7f31841027b0 2024-03-12T04:14:28.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f317affd700 1 -- 172.21.15.42:0/3783214384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0075d0 con 0x7f31841027b0 2024-03-12T04:14:28.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318419fda0 con 0x7f31841027b0 2024-03-12T04:14:28.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.799+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31841a0320 con 0x7f31841027b0 2024-03-12T04:14:28.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.800+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31841080a0 con 0x7f31841027b0 2024-03-12T04:14:28.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.801+0000 7f317affd700 1 -- 172.21.15.42:0/3783214384 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f317c0145b0 con 0x7f31841027b0 2024-03-12T04:14:28.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.802+0000 7f317affd700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f31700a1ac0 0x7f31700a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:28.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.802+0000 7f317affd700 1 -- 172.21.15.42:0/3783214384 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f317c006e40 con 0x7f31841027b0 2024-03-12T04:14:28.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.802+0000 7f3189c6a700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f31700a1ac0 0x7f31700a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:28.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.802+0000 7f3189c6a700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f31700a1ac0 0x7f31700a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3184101c70 tx=0x7f3180008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:28.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:28.806+0000 7f317affd700 1 -- 172.21.15.42:0/3783214384 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f317c095d20 con 0x7f31841027b0 2024-03-12T04:14:29.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.021+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f318402d870 con 0x7f31700a1ac0 2024-03-12T04:14:29.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.035+0000 7f317affd700 1 -- 172.21.15.42:0/3783214384 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f318402d870 con 0x7f31700a1ac0 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.a smithi042 running (103s) 16s ago 103s 18.5M - 19.0.0-1607-g94a53970 a6477a474d71 b8e97fa10157 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.b smithi119 running (100s) 33s ago 99s 28.6M - 19.0.0-1607-g94a53970 a6477a474d71 c6dd84ddb419 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.c smithi136 running (114s) 34s ago 114s 19.7M - 19.0.0-1607-g94a53970 a6477a474d71 8fde661ac291 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.d smithi042 running (2m) 16s ago 2m 13.9M - 19.0.0-1607-g94a53970 a6477a474d71 ca93b1335576 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.e smithi119 running (2m) 33s ago 2m 19.0M - 19.0.0-1607-g94a53970 a6477a474d71 41cc2f8e28ed 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.f smithi136 running (117s) 34s ago 117s 20.3M - 19.0.0-1607-g94a53970 a6477a474d71 055b1c6b7828 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.g smithi042 running (106s) 16s ago 105s 16.7M - 19.0.0-1607-g94a53970 a6477a474d71 01229492569c 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.h smithi119 running (111s) 33s ago 111s 19.2M - 19.0.0-1607-g94a53970 a6477a474d71 06d5e2c6cabb 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.i smithi136 running (109s) 34s ago 108s 17.9M - 19.0.0-1607-g94a53970 a6477a474d71 e92a959ee9ec 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.j smithi042 running (2m) 16s ago 2m 16.0M - 19.0.0-1607-g94a53970 a6477a474d71 2e78f253e894 2024-03-12T04:14:29.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.k smithi119 running (2m) 33s ago 2m 16.9M - 19.0.0-1607-g94a53970 a6477a474d71 a47debd23377 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:mds.l smithi136 running (2m) 34s ago 2m 16.7M - 19.0.0-1607-g94a53970 a6477a474d71 d76170f81d9f 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.x smithi042 *:9283,8765 running (9m) 16s ago 9m 512M - 19.0.0-1607-g94a53970 a6477a474d71 58dfb605d35e 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.y smithi119 *:8443,8765 running (7m) 33s ago 7m 451M - 19.0.0-1607-g94a53970 a6477a474d71 35294ab596e7 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.z smithi136 *:8443,8765 running (7m) 34s ago 7m 453M - 19.0.0-1607-g94a53970 a6477a474d71 3682a4875a16 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a smithi042 running (9m) 16s ago 9m 50.6M 2048M 19.0.0-1607-g94a53970 a6477a474d71 67e30e1d77b0 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b smithi119 running (7m) 33s ago 7m 38.9M 2048M 19.0.0-1607-g94a53970 a6477a474d71 e687522f7f21 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c smithi136 running (7m) 34s ago 7m 39.0M 2048M 19.0.0-1607-g94a53970 a6477a474d71 0b107daf45d6 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 16s ago 7m 45.7M 4273M 19.0.0-1607-g94a53970 a6477a474d71 bd36700dc754 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi119 running (6m) 33s ago 6m 49.1M 4272M 19.0.0-1607-g94a53970 a6477a474d71 ed958a788e4b 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi136 running (6m) 34s ago 6m 49.8M 4272M 19.0.0-1607-g94a53970 a6477a474d71 391789aa2a68 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi042 running (6m) 16s ago 6m 53.1M 4273M 19.0.0-1607-g94a53970 a6477a474d71 85b54009a77e 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi119 running (5m) 33s ago 5m 42.1M 4272M 19.0.0-1607-g94a53970 a6477a474d71 ca88ac741f7b 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi136 running (5m) 34s ago 5m 55.5M 4272M 19.0.0-1607-g94a53970 a6477a474d71 93e0d787ef97 2024-03-12T04:14:29.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 smithi042 running (4m) 16s ago 4m 51.3M 4273M 19.0.0-1607-g94a53970 a6477a474d71 c20940336b32 2024-03-12T04:14:29.040 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 smithi119 running (4m) 33s ago 4m 57.1M 4272M 19.0.0-1607-g94a53970 a6477a474d71 e6db9a800223 2024-03-12T04:14:29.040 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 smithi136 running (3m) 34s ago 3m 54.1M 4272M 19.0.0-1607-g94a53970 a6477a474d71 1d38b98e2522 2024-03-12T04:14:29.040 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 smithi042 running (3m) 16s ago 3m 42.6M 4273M 19.0.0-1607-g94a53970 a6477a474d71 b5ad832cafb1 2024-03-12T04:14:29.040 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 smithi119 running (3m) 33s ago 3m 55.0M 4272M 19.0.0-1607-g94a53970 a6477a474d71 f33854c069cd 2024-03-12T04:14:29.040 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 smithi136 running (2m) 34s ago 2m 51.4M 4272M 19.0.0-1607-g94a53970 a6477a474d71 b6843c35c6ec 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f31700a1ac0 msgr2=0x7f31700a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f31700a1ac0 0x7f31700a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3184101c70 tx=0x7f3180008040 comp rx=0 tx=0).stop 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31841027b0 msgr2=0x7f318419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31841027b0 0x7f318419eff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b970 tx=0x7f317c00bd30 comp rx=0 tx=0).stop 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 shutdown_connections 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f31700a1ac0 0x7f31700a3f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31841027b0 0x7f318419eff0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3184100c10 0x7f318406aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 --2- 172.21.15.42:0/3783214384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3184101e10 0x7f318419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:29.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 >> 172.21.15.42:0/3783214384 conn(0x7f31840fc1d0 msgr2=0x7f31841051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:29.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 shutdown_connections 2024-03-12T04:14:29.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:29.037+0000 7f318bece700 1 -- 172.21.15.42:0/3783214384 wait complete. 2024-03-12T04:14:29.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:28 smithi042 ceph-mon[32260]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-12T04:14:29.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:28 smithi119 ceph-mon[36119]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-12T04:14:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:28 smithi136 ceph-mon[35571]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-12T04:14:29.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-12T04:14:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:30.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:29 smithi042 ceph-mon[32260]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:30.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:29 smithi119 ceph-mon[36119]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:30.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:29 smithi136 ceph-mon[35571]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:31.151 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:30 smithi042 ceph-mon[32260]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:14:31.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:30 smithi119 ceph-mon[36119]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:14:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:30 smithi136 ceph-mon[35571]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-12T04:14:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 -- 172.21.15.42:0/2701739104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150100b90 msgr2=0x7f5150100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 --2- 172.21.15.42:0/2701739104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150100b90 0x7f5150100fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f51400099c0 tx=0x7f5140009cd0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 -- 172.21.15.42:0/2701739104 shutdown_connections 2024-03-12T04:14:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 --2- 172.21.15.42:0/2701739104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5150102730 0x7f5150108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 --2- 172.21.15.42:0/2701739104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150100b90 0x7f5150100fb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 --2- 172.21.15.42:0/2701739104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5150101d90 0x7f51501021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 -- 172.21.15.42:0/2701739104 >> 172.21.15.42:0/2701739104 conn(0x7f51500fc130 msgr2=0x7f51500fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 -- 172.21.15.42:0/2701739104 shutdown_connections 2024-03-12T04:14:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.757+0000 7f51568c1700 1 -- 172.21.15.42:0/2701739104 wait complete. 2024-03-12T04:14:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 Processor -- start 2024-03-12T04:14:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 -- start start 2024-03-12T04:14:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5150100b90 0x7f5150197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150101d90 0x7f51501980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5150102730 0x7f515019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019f720 con 0x7f5150101d90 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019f860 con 0x7f5150100b90 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f51568c1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019f9f0 con 0x7f5150102730 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150101d90 0x7f51501980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150101d90 0x7f51501980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54204/0 (socket says 172.21.15.42:54204) 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f514f7fe700 1 -- 172.21.15.42:0/3113649274 learned_addr learned my addr 172.21.15.42:0/3113649274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f5154e5e700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5150102730 0x7f515019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f514ffff700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5150100b90 0x7f5150197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.758+0000 7f514f7fe700 1 -- 172.21.15.42:0/3113649274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5150102730 msgr2=0x7f515019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514f7fe700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5150102730 0x7f515019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514f7fe700 1 -- 172.21.15.42:0/3113649274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5150100b90 msgr2=0x7f5150197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514f7fe700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5150100b90 0x7f5150197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514f7fe700 1 -- 172.21.15.42:0/3113649274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51400095f0 con 0x7f5150101d90 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514ffff700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5150100b90 0x7f5150197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514f7fe700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150101d90 0x7f51501980d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f513800d980 tx=0x7f513800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514d7fa700 1 -- 172.21.15.42:0/3113649274 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138009b40 con 0x7f5150101d90 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515019fcd0 con 0x7f5150101d90 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514d7fa700 1 -- 172.21.15.42:0/3113649274 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f513800b6e0 con 0x7f5150101d90 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f514d7fa700 1 -- 172.21.15.42:0/3113649274 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800f620 con 0x7f5150101d90 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.759+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51501a0170 con 0x7f5150101d90 2024-03-12T04:14:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.760+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5150068960 con 0x7f5150101d90 2024-03-12T04:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.761+0000 7f514d7fa700 1 -- 172.21.15.42:0/3113649274 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5138010460 con 0x7f5150101d90 2024-03-12T04:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.762+0000 7f514d7fa700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f513c0a1ac0 0x7f513c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.762+0000 7f514ffff700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f513c0a1ac0 0x7f513c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.762+0000 7f514d7fa700 1 -- 172.21.15.42:0/3113649274 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f51380cd280 con 0x7f5150101d90 2024-03-12T04:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.762+0000 7f514ffff700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f513c0a1ac0 0x7f513c0a3f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5140000c00 tx=0x7f5140005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:31.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.765+0000 7f514d7fa700 1 -- 172.21.15.42:0/3113649274 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5138096460 con 0x7f5150101d90 2024-03-12T04:14:31.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.978+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f51500fe350 con 0x7f513c0a1ac0 2024-03-12T04:14:31.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.984+0000 7f514d7fa700 1 -- 172.21.15.42:0/3113649274 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f51500fe350 con 0x7f513c0a1ac0 2024-03-12T04:14:31.987 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-12T04:14:31.987 INFO:teuthology.orchestra.run.smithi042.stdout:mds.all 12/12 37s ago 2m smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-03-12T04:14:31.987 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 3/3 37s ago 7m smithi042=x;smithi119=y;smithi136=z;count:3 2024-03-12T04:14:31.987 INFO:teuthology.orchestra.run.smithi042.stdout:mon 3/3 37s ago 8m smithi042:172.21.15.42=a;smithi119:172.21.15.119=b;smithi136:172.21.15.136=c;count:3 2024-03-12T04:14:31.987 INFO:teuthology.orchestra.run.smithi042.stdout:osd 12 37s ago - 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f513c0a1ac0 msgr2=0x7f513c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f513c0a1ac0 0x7f513c0a3f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5140000c00 tx=0x7f5140005fd0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150101d90 msgr2=0x7f51501980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150101d90 0x7f51501980d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f513800d980 tx=0x7f513800dc90 comp rx=0 tx=0).stop 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 shutdown_connections 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f513c0a1ac0 0x7f513c0a3f80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5150102730 0x7f515019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5150100b90 0x7f5150197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 --2- 172.21.15.42:0/3113649274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5150101d90 0x7f51501980d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:31.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 >> 172.21.15.42:0/3113649274 conn(0x7f51500fc130 msgr2=0x7f51500fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:31.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 shutdown_connections 2024-03-12T04:14:31.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:31.986+0000 7f51568c1700 1 -- 172.21.15.42:0/3113649274 wait complete. 2024-03-12T04:14:32.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-12T04:14:33.009 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:33.292 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:32 smithi042 ceph-mon[32260]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:33.292 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:32 smithi042 ceph-mon[32260]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:33.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:32 smithi119 ceph-mon[36119]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:33.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:32 smithi119 ceph-mon[36119]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:32 smithi136 ceph-mon[35571]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:32 smithi136 ceph-mon[35571]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:34.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 -- 172.21.15.42:0/1265500323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501050c0 msgr2=0x7fe9501074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:34.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 --2- 172.21.15.42:0/1265500323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501050c0 0x7fe9501074b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe940009990 tx=0x7fe940009ca0 comp rx=0 tx=0).stop 2024-03-12T04:14:34.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 -- 172.21.15.42:0/1265500323 shutdown_connections 2024-03-12T04:14:34.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 --2- 172.21.15.42:0/1265500323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9501079f0 0x7fe950109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:34.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 --2- 172.21.15.42:0/1265500323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe950102790 0x7fe950104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:34.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 --2- 172.21.15.42:0/1265500323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501050c0 0x7fe9501074b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:34.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 -- 172.21.15.42:0/1265500323 >> 172.21.15.42:0/1265500323 conn(0x7fe9500fc140 msgr2=0x7fe9500fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 -- 172.21.15.42:0/1265500323 shutdown_connections 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.984+0000 7fe95868e700 1 -- 172.21.15.42:0/1265500323 wait complete. 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 Processor -- start 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 -- start start 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe950102790 0x7fe950197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9501050c0 0x7fe950198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501079f0 0x7fe95019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019f760 con 0x7fe9501079f0 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019f8a0 con 0x7fe950102790 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95868e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019fa60 con 0x7fe9501050c0 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe956c2b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501079f0 0x7fe95019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe955c29700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9501050c0 0x7fe950198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe956c2b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501079f0 0x7fe95019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54214/0 (socket says 172.21.15.42:54214) 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe956c2b700 1 -- 172.21.15.42:0/4225815346 learned_addr learned my addr 172.21.15.42:0/4225815346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:34.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe95642a700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe950102790 0x7fe950197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:34.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe956c2b700 1 -- 172.21.15.42:0/4225815346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9501050c0 msgr2=0x7fe950198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:34.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.985+0000 7fe956c2b700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9501050c0 0x7fe950198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:34.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe956c2b700 1 -- 172.21.15.42:0/4225815346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe950102790 msgr2=0x7fe950197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:34.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe956c2b700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe950102790 0x7fe950197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:34.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe956c2b700 1 -- 172.21.15.42:0/4225815346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400095f0 con 0x7fe9501079f0 2024-03-12T04:14:34.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe956c2b700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501079f0 0x7fe95019f190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe9480094f0 tx=0x7fe94800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe9477fe700 1 -- 172.21.15.42:0/4225815346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948004db0 con 0x7fe9501079f0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe9477fe700 1 -- 172.21.15.42:0/4225815346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe94800f040 con 0x7fe9501079f0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95019fd40 con 0x7fe9501079f0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe9477fe700 1 -- 172.21.15.42:0/4225815346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948005970 con 0x7fe9501079f0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.986+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9501a0190 con 0x7fe9501079f0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.987+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9500fdcc0 con 0x7fe9501079f0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.988+0000 7fe9477fe700 1 -- 172.21.15.42:0/4225815346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe94800db20 con 0x7fe9501079f0 2024-03-12T04:14:34.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.989+0000 7fe9477fe700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe93c0a1ac0 0x7fe93c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:34.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.989+0000 7fe95642a700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe93c0a1ac0 0x7fe93c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:34.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.989+0000 7fe9477fe700 1 -- 172.21.15.42:0/4225815346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe9480ccd20 con 0x7fe9501079f0 2024-03-12T04:14:34.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.989+0000 7fe95642a700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe93c0a1ac0 0x7fe93c0a3f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9501990f0 tx=0x7fe94c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:34.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:34.993+0000 7fe9477fe700 1 -- 172.21.15.42:0/4225815346 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe948095f00 con 0x7fe9501079f0 2024-03-12T04:14:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:34 smithi042 ceph-mon[32260]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.203+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe95002d990 con 0x7fe93c0a1ac0 2024-03-12T04:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.205+0000 7fe9477fe700 1 -- 172.21.15.42:0/4225815346 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fe95002d990 con 0x7fe93c0a1ac0 2024-03-12T04:14:35.207 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2024-03-12T04:14:35.207 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2024-03-12T04:14:35.207 INFO:teuthology.orchestra.run.smithi042.stdout:smithi119 172.21.15.119 2024-03-12T04:14:35.207 INFO:teuthology.orchestra.run.smithi042.stdout:smithi136 172.21.15.136 2024-03-12T04:14:35.208 INFO:teuthology.orchestra.run.smithi042.stdout:3 hosts in cluster 2024-03-12T04:14:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe93c0a1ac0 msgr2=0x7fe93c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe93c0a1ac0 0x7fe93c0a3f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9501990f0 tx=0x7fe94c00b410 comp rx=0 tx=0).stop 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501079f0 msgr2=0x7fe95019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501079f0 0x7fe95019f190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe9480094f0 tx=0x7fe94800d3b0 comp rx=0 tx=0).stop 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 shutdown_connections 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe93c0a1ac0 0x7fe93c0a3f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9501079f0 0x7fe95019f190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe950102790 0x7fe950197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 --2- 172.21.15.42:0/4225815346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9501050c0 0x7fe950198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 >> 172.21.15.42:0/4225815346 conn(0x7fe9500fc140 msgr2=0x7fe950104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 shutdown_connections 2024-03-12T04:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:35.207+0000 7fe95868e700 1 -- 172.21.15.42:0/4225815346 wait complete. 2024-03-12T04:14:35.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:34 smithi119 ceph-mon[36119]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:35.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:34 smithi136 ceph-mon[35571]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-12T04:14:36.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-12T04:14:36.264 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:36.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:35 smithi042 ceph-mon[32260]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:36.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:35 smithi119 ceph-mon[36119]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:36.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:35 smithi136 ceph-mon[35571]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:37.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:36 smithi042 ceph-mon[32260]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:37.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:36 smithi119 ceph-mon[36119]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:37.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:36 smithi136 ceph-mon[35571]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:38.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 -- 172.21.15.42:0/1720329917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 msgr2=0x7f3a4c10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/1720329917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c10a190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b0a0 tx=0x7f3a4400b3b0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 -- 172.21.15.42:0/1720329917 shutdown_connections 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/1720329917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c10a190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/1720329917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c0fff60 0x7f3a4c100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/1720329917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4c102f40 0x7f3a4c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 -- 172.21.15.42:0/1720329917 >> 172.21.15.42:0/1720329917 conn(0x7f3a4c0fb8e0 msgr2=0x7f3a4c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 -- 172.21.15.42:0/1720329917 shutdown_connections 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.091+0000 7f3a51ad4700 1 -- 172.21.15.42:0/1720329917 wait complete. 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 Processor -- start 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 -- start start 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c0fff60 0x7f3a4c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4c102f40 0x7f3a4c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c19f770 con 0x7f3a4c1038e0 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c19f8b0 con 0x7f3a4c102f40 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a51ad4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c19fa70 con 0x7f3a4c0fff60 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c19f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54232/0 (socket says 172.21.15.42:54232) 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a4bfff700 1 -- 172.21.15.42:0/3363882651 learned_addr learned my addr 172.21.15.42:0/3363882651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a4b7fe700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c0fff60 0x7f3a4c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.092+0000 7f3a4affd700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4c102f40 0x7f3a4c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a4bfff700 1 -- 172.21.15.42:0/3363882651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c0fff60 msgr2=0x7f3a4c197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a4bfff700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c0fff60 0x7f3a4c197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a4bfff700 1 -- 172.21.15.42:0/3363882651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4c102f40 msgr2=0x7f3a4c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a4bfff700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4c102f40 0x7f3a4c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a4bfff700 1 -- 172.21.15.42:0/3363882651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44009d00 con 0x7f3a4c1038e0 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a4bfff700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c19f1a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3a44004d00 tx=0x7f3a44004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a48ff9700 1 -- 172.21.15.42:0/3363882651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400e040 con 0x7f3a4c1038e0 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a48ff9700 1 -- 172.21.15.42:0/3363882651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3a4400cea0 con 0x7f3a4c1038e0 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c19fcf0 con 0x7f3a4c1038e0 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a48ff9700 1 -- 172.21.15.42:0/3363882651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4401b9c0 con 0x7f3a4c1038e0 2024-03-12T04:14:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.093+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c1a0160 con 0x7f3a4c1038e0 2024-03-12T04:14:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.094+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c109120 con 0x7f3a4c1038e0 2024-03-12T04:14:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.095+0000 7f3a48ff9700 1 -- 172.21.15.42:0/3363882651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3a4401dcd0 con 0x7f3a4c1038e0 2024-03-12T04:14:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.096+0000 7f3a48ff9700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3a340a1ac0 0x7f3a340a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.096+0000 7f3a4b7fe700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3a340a1ac0 0x7f3a340a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.096+0000 7f3a48ff9700 1 -- 172.21.15.42:0/3363882651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3a440cef80 con 0x7f3a4c1038e0 2024-03-12T04:14:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.096+0000 7f3a4b7fe700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3a340a1ac0 0x7f3a340a3f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c199100 tx=0x7f3a3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:38.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.100+0000 7f3a48ff9700 1 -- 172.21.15.42:0/3363882651 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a440982f0 con 0x7f3a4c1038e0 2024-03-12T04:14:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.311+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a4c02d8d0 con 0x7f3a340a1ac0 2024-03-12T04:14:38.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.313+0000 7f3a48ff9700 1 -- 172.21.15.42:0/3363882651 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f3a4c02d8d0 con 0x7f3a340a1ac0 2024-03-12T04:14:38.316 INFO:teuthology.orchestra.run.smithi042.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-12T04:14:38.316 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130021480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-12T04:14:38.316 INFO:teuthology.orchestra.run.smithi042.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-12T04:14:38.316 INFO:teuthology.orchestra.run.smithi042.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 2m ago Has a FileSystem, LVM detected 2024-03-12T04:14:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3a340a1ac0 msgr2=0x7f3a340a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3a340a1ac0 0x7f3a340a3f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c199100 tx=0x7f3a3c00b410 comp rx=0 tx=0).stop 2024-03-12T04:14:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 msgr2=0x7f3a4c19f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c19f1a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3a44004d00 tx=0x7f3a44004d30 comp rx=0 tx=0).stop 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 shutdown_connections 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f3a340a1ac0 0x7f3a340a3f80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a4c1038e0 0x7f3a4c19f1a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c0fff60 0x7f3a4c197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 --2- 172.21.15.42:0/3363882651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4c102f40 0x7f3a4c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.315+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 >> 172.21.15.42:0/3363882651 conn(0x7f3a4c0fb8e0 msgr2=0x7f3a4c106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.316+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 shutdown_connections 2024-03-12T04:14:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:38.316+0000 7f3a51ad4700 1 -- 172.21.15.42:0/3363882651 wait complete. 2024-03-12T04:14:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:38 smithi042 ceph-mon[32260]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:38 smithi042 ceph-mon[32260]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:14:39.120 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-12T04:14:39.130 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T04:14:39.130 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-12T04:14:39.376 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:39.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:38 smithi119 ceph-mon[36119]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:39.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:38 smithi119 ceph-mon[36119]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:14:39.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:38 smithi136 ceph-mon[35571]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:14:39.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:38 smithi136 ceph-mon[35571]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-12T04:14:41.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2974986112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 msgr2=0x7fe4d0100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:41.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2974986112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0100fb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00099c0 tx=0x7fe4c0009cd0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.188 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:41 smithi042 ceph-mon[32260]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:41.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2974986112 shutdown_connections 2024-03-12T04:14:41.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2974986112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d0102730 0x7fe4d0108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2974986112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0100fb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2974986112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d0101d90 0x7fe4d01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2974986112 >> 172.21.15.42:0/2974986112 conn(0x7fe4d00fc130 msgr2=0x7fe4d00fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.184+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2974986112 shutdown_connections 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.185+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2974986112 wait complete. 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.185+0000 7fe4d4f66700 1 Processor -- start 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.185+0000 7fe4d4f66700 1 -- start start 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4d4f66700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4d4f66700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d0101d90 0x7fe4d0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4d4f66700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d0102730 0x7fe4d019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4d4f66700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d019f6a0 con 0x7fe4d0100b90 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4d4f66700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d019f7e0 con 0x7fe4d0101d90 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4d4f66700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d019f970 con 0x7fe4d0102730 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:41.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54242/0 (socket says 172.21.15.42:54242) 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 -- 172.21.15.42:0/2946026874 learned_addr learned my addr 172.21.15.42:0/2946026874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 -- 172.21.15.42:0/2946026874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d0102730 msgr2=0x7fe4d019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ced9d700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d0102730 0x7fe4d019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4cdd9b700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d0101d90 0x7fe4d0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d0102730 0x7fe4d019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 -- 172.21.15.42:0/2946026874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d0101d90 msgr2=0x7fe4d0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d0101d90 0x7fe4d0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 -- 172.21.15.42:0/2946026874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00095f0 con 0x7fe4d0100b90 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4cdd9b700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d0101d90 0x7fe4d0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ced9d700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d0102730 0x7fe4d019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4ce59c700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0197b10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000b580 tx=0x7fe4c0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4bf7fe700 1 -- 172.21.15.42:0/2946026874 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c001d070 con 0x7fe4d0100b90 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.186+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d019fbf0 con 0x7fe4d0100b90 2024-03-12T04:14:41.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.187+0000 7fe4bf7fe700 1 -- 172.21.15.42:0/2946026874 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4c0004180 con 0x7fe4d0100b90 2024-03-12T04:14:41.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.187+0000 7fe4bf7fe700 1 -- 172.21.15.42:0/2946026874 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0022e30 con 0x7fe4d0100b90 2024-03-12T04:14:41.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.187+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d01a00e0 con 0x7fe4d0100b90 2024-03-12T04:14:41.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.188+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d0068930 con 0x7fe4d0100b90 2024-03-12T04:14:41.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.189+0000 7fe4bf7fe700 1 -- 172.21.15.42:0/2946026874 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe4c0022600 con 0x7fe4d0100b90 2024-03-12T04:14:41.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.189+0000 7fe4bf7fe700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4b80a1ac0 0x7fe4b80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:41.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.189+0000 7fe4cdd9b700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4b80a1ac0 0x7fe4b80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:41.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.189+0000 7fe4bf7fe700 1 -- 172.21.15.42:0/2946026874 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe4c00ce7c0 con 0x7fe4d0100b90 2024-03-12T04:14:41.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.190+0000 7fe4cdd9b700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4b80a1ac0 0x7fe4b80a3f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4005fd0 tx=0x7fe4c4005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:41.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.194+0000 7fe4bf7fe700 1 -- 172.21.15.42:0/2946026874 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4c0097a90 con 0x7fe4d0100b90 2024-03-12T04:14:41.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.439+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe4d01a0450 con 0x7fe4d0100b90 2024-03-12T04:14:41.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.439+0000 7fe4bf7fe700 1 -- 172.21.15.42:0/2946026874 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2967 (secure 0 0 0) 0x7fe4c009b150 con 0x7fe4d0100b90 2024-03-12T04:14:41.442 INFO:teuthology.orchestra.run.smithi042.stdout:e26 2024-03-12T04:14:41.442 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-12T04:14:41.442 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-12T04:14:41.442 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 26 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-03-12T04:12:54.116170+0000 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-03-12T04:13:01.029979+0000 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 300 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-03-12T04:14:41.443 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 5 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1,2,3,4 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24515,1=24490,2=24485,3=24475,4=14634} 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-03-12T04:14:41.444 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b{0:24515} state up:active seq 3 addr [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f{0:24457} state up:standby-replay seq 1 addr [v2:172.21.15.136:6834/2718353674,v1:172.21.15.136:6835/2718353674] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.i{1:24490} state up:active seq 6 addr [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e{1:24452} state up:standby-replay seq 1 addr [v2:172.21.15.119:6834/2794337720,v1:172.21.15.119:6835/2794337720] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.h{2:24485} state up:active seq 7 addr [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.k{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.119:6832/2928023683,v1:172.21.15.119:6833/2928023683] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c{3:24475} state up:active seq 8 addr [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.l{3:24415} state up:standby-replay seq 1 addr [v2:172.21.15.136:6832/3418533459,v1:172.21.15.136:6833/3418533459] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a{4:14634} state up:active seq 5 addr [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.g{4:14622} state up:standby-replay seq 1 addr [v2:172.21.15.42:6838/1870620293,v1:172.21.15.42:6839/1870620293] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:41.446 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-03-12T04:14:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:41.446 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.42:6834/43691567,v1:172.21.15.42:6835/43691567] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.446 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.42:6836/3380811405,v1:172.21.15.42:6837/3380811405] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T04:14:41.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:41 smithi119 ceph-mon[36119]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4b80a1ac0 msgr2=0x7fe4b80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4b80a1ac0 0x7fe4b80a3f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4005fd0 tx=0x7fe4c4005dc0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 msgr2=0x7fe4d0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0197b10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000b580 tx=0x7fe4c0005ef0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 shutdown_connections 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4b80a1ac0 0x7fe4b80a3f80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d0102730 0x7fe4d019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d0100b90 0x7fe4d0197b10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 --2- 172.21.15.42:0/2946026874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d0101d90 0x7fe4d0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:41.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 >> 172.21.15.42:0/2946026874 conn(0x7fe4d00fc130 msgr2=0x7fe4d00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:41.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 shutdown_connections 2024-03-12T04:14:41.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:41.442+0000 7fe4d4f66700 1 -- 172.21.15.42:0/2946026874 wait complete. 2024-03-12T04:14:41.449 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-03-12T04:14:41.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:41 smithi136 ceph-mon[35571]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-12T04:14:42.243 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-12T04:14:42.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:42 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2946026874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-12T04:14:42.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:42 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2946026874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-12T04:14:42.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:42 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2946026874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-12T04:14:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:43.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:43 smithi042 ceph-mon[32260]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:43.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:43 smithi119 ceph-mon[36119]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:43.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:43 smithi136 ceph-mon[35571]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1152539032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0102740 msgr2=0x7f70c0108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1152539032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0102740 0x7f70c0108ff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f70b800b0d0 tx=0x7f70b800b3e0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1152539032 shutdown_connections 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1152539032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0102740 0x7f70c0108ff0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1152539032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c0100ba0 0x7f70c0100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1152539032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70c0101da0 0x7f70c0102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1152539032 >> 172.21.15.42:0/1152539032 conn(0x7f70c00fc160 msgr2=0x7f70c00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1152539032 shutdown_connections 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1152539032 wait complete. 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 Processor -- start 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- start start 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70c0100ba0 0x7f70c0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:44.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0101da0 0x7f70c01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c0102740 0x7f70c019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c019f730 con 0x7f70c0101da0 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c019f870 con 0x7f70c0100ba0 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.299+0000 7f70c5c7d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c019fa00 con 0x7f70c0102740 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0101da0 0x7f70c01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0101da0 0x7f70c01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52022/0 (socket says 172.21.15.42:52022) 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 -- 172.21.15.42:0/1752913200 learned_addr learned my addr 172.21.15.42:0/1752913200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70bf7fe700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70c0100ba0 0x7f70c0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70bffff700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c0102740 0x7f70c019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 -- 172.21.15.42:0/1752913200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c0102740 msgr2=0x7f70c019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c0102740 0x7f70c019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 -- 172.21.15.42:0/1752913200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70c0100ba0 msgr2=0x7f70c0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70c0100ba0 0x7f70c0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 -- 172.21.15.42:0/1752913200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b8009d00 con 0x7f70c0101da0 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70b7fff700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0101da0 0x7f70c01980e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c9e0 tx=0x7f70a800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70bd7fa700 1 -- 172.21.15.42:0/1752913200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8007ac0 con 0x7f70c0101da0 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70bd7fa700 1 -- 172.21.15.42:0/1752913200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70a8007c20 con 0x7f70c0101da0 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c019fce0 con 0x7f70c0101da0 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70bd7fa700 1 -- 172.21.15.42:0/1752913200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8018660 con 0x7f70c0101da0 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.300+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c01a0190 con 0x7f70c0101da0 2024-03-12T04:14:44.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.301+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c0068930 con 0x7f70c0101da0 2024-03-12T04:14:44.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.303+0000 7f70bd7fa700 1 -- 172.21.15.42:0/1752913200 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f70a800f450 con 0x7f70c0101da0 2024-03-12T04:14:44.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.303+0000 7f70bd7fa700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f70ac0a1ac0 0x7f70ac0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:44.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.303+0000 7f70bd7fa700 1 -- 172.21.15.42:0/1752913200 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f70a80cd620 con 0x7f70c0101da0 2024-03-12T04:14:44.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.303+0000 7f70bf7fe700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f70ac0a1ac0 0x7f70ac0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:44.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.304+0000 7f70bf7fe700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f70ac0a1ac0 0x7f70ac0a3f80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70c0101c00 tx=0x7f70b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:44.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.307+0000 7f70bd7fa700 1 -- 172.21.15.42:0/1752913200 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70a80968f0 con 0x7f70c0101da0 2024-03-12T04:14:44.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.511+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f70c010c5c0 con 0x7f70c0101da0 2024-03-12T04:14:44.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.512+0000 7f70bd7fa700 1 -- 172.21.15.42:0/1752913200 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) v1 ==== 56+0+5260 (secure 0 0 0) 0x7f70a800f700 con 0x7f70c0101da0 2024-03-12T04:14:44.514 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 89 2024-03-12T04:14:44.514 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-03-12T04:04:45.008136+0000 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-03-12T04:13:45.923160+0000 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 26 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client luminous 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release squid 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-12T04:14:44.515 INFO:teuthology.orchestra.run.smithi042.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 12 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6802/1137485788,v1:172.21.15.42:6803/1137485788] [v2:172.21.15.42:6804/1137485788,v1:172.21.15.42:6805/1137485788] exists,up 66e5b58b-4170-4df0-a9b2-5abe5a61884c 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/3462838566,v1:172.21.15.119:6801/3462838566] [v2:172.21.15.119:6802/3462838566,v1:172.21.15.119:6803/3462838566] exists,up 7e1a8d54-070c-49ef-a2a6-f2fe66c292ce 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/2837276479,v1:172.21.15.136:6801/2837276479] [v2:172.21.15.136:6802/2837276479,v1:172.21.15.136:6803/2837276479] exists,up fb3ebf6a-d807-4ead-89b0-bd81dcf650e5 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6810/3878628201,v1:172.21.15.42:6811/3878628201] [v2:172.21.15.42:6812/3878628201,v1:172.21.15.42:6813/3878628201] exists,up bcfd2c44-146b-4495-92e4-ad2e3c85d16d 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/239664950,v1:172.21.15.119:6809/239664950] [v2:172.21.15.119:6810/239664950,v1:172.21.15.119:6811/239664950] exists,up 97a77fed-dfbe-45a5-8048-e1424bfee89f 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 38 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/2791487312,v1:172.21.15.136:6809/2791487312] [v2:172.21.15.136:6810/2791487312,v1:172.21.15.136:6811/2791487312] exists,up 68dc50d1-ecd2-4abc-af46-3ad102d339be 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 46 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6818/515757893,v1:172.21.15.42:6819/515757893] [v2:172.21.15.42:6820/515757893,v1:172.21.15.42:6821/515757893] exists,up 99e7959d-3257-4c03-b302-c98052f009e0 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 52 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/1789109628,v1:172.21.15.119:6817/1789109628] [v2:172.21.15.119:6818/1789109628,v1:172.21.15.119:6819/1789109628] exists,up c430bbfa-80fb-485a-a186-ee581a9a8196 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 up in weight 1 up_from 61 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/509882015,v1:172.21.15.136:6817/509882015] [v2:172.21.15.136:6818/509882015,v1:172.21.15.136:6819/509882015] exists,up c1b72976-753c-4cee-9be5-867ebd0479ef 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 up in weight 1 up_from 69 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6826/1422185869,v1:172.21.15.42:6827/1422185869] [v2:172.21.15.42:6828/1422185869,v1:172.21.15.42:6829/1422185869] exists,up 4d19ae72-048c-4680-bf7d-2b2f2bf353f7 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 up in weight 1 up_from 74 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/3868326698,v1:172.21.15.119:6825/3868326698] [v2:172.21.15.119:6826/3868326698,v1:172.21.15.119:6827/3868326698] exists,up f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4 2024-03-12T04:14:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/2421468523,v1:172.21.15.136:6825/2421468523] [v2:172.21.15.136:6826/2421468523,v1:172.21.15.136:6827/2421468523] exists,up bf7c32db-7617-4fe4-ab70-4bf8efcb0869 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.10 [8,11] 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.13 [2,11] 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.1d [4,8] 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.22 [3,9] 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.24 [4,9] 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.25 [5,9] 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.3a [4,6] 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1914532345 expires 2024-03-13T04:05:44.816230+0000 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/956219817 expires 2024-03-13T04:05:44.816230+0000 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/2670488995 expires 2024-03-13T04:05:44.816230+0000 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2257992536 expires 2024-03-13T04:05:44.816230+0000 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1532115047 expires 2024-03-13T04:05:17.928065+0000 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/3492564527 expires 2024-03-13T04:05:17.928065+0000 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/2755197547 expires 2024-03-13T04:05:17.928065+0000 2024-03-12T04:14:44.517 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/331646616 expires 2024-03-13T04:05:17.928065+0000 2024-03-12T04:14:44.518 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/2670488995 expires 2024-03-13T04:05:44.816230+0000 2024-03-12T04:14:44.518 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/2755197547 expires 2024-03-13T04:05:17.928065+0000 2024-03-12T04:14:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f70ac0a1ac0 msgr2=0x7f70ac0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f70ac0a1ac0 0x7f70ac0a3f80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70c0101c00 tx=0x7f70b000b410 comp rx=0 tx=0).stop 2024-03-12T04:14:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0101da0 msgr2=0x7f70c01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0101da0 0x7f70c01980e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c9e0 tx=0x7f70a800ccf0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 shutdown_connections 2024-03-12T04:14:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f70ac0a1ac0 0x7f70ac0a3f80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c0102740 0x7f70c019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70c0100ba0 0x7f70c0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 --2- 172.21.15.42:0/1752913200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c0101da0 0x7f70c01980e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 >> 172.21.15.42:0/1752913200 conn(0x7f70c00fc160 msgr2=0x7f70c00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 shutdown_connections 2024-03-12T04:14:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:44.514+0000 7f70c5c7d700 1 -- 172.21.15.42:0/1752913200 wait complete. 2024-03-12T04:14:44.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:44 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1752913200' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-12T04:14:44.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:44 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1752913200' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-12T04:14:44.982 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:44 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1752913200' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-12T04:14:45.413 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-12T04:14:45.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 -- 172.21.15.42:0/3255540501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 msgr2=0x7f4178106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 --2- 172.21.15.42:0/3255540501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178106a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f416c009900 tx=0x7f416c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 -- 172.21.15.42:0/3255540501 shutdown_connections 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 --2- 172.21.15.42:0/3255540501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178106f60 0x7f4178109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 --2- 172.21.15.42:0/3255540501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178106a20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 --2- 172.21.15.42:0/3255540501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4178101d00 0x7f41781040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 -- 172.21.15.42:0/3255540501 >> 172.21.15.42:0/3255540501 conn(0x7f41780fb6b0 msgr2=0x7f41780fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 -- 172.21.15.42:0/3255540501 shutdown_connections 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 -- 172.21.15.42:0/3255540501 wait complete. 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.532+0000 7f417d948640 1 Processor -- start 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f417d948640 1 -- start start 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f417d948640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178101d00 0x7f4178197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f417d948640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f417d948640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4178106f60 0x7f417819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f417d948640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417819ec30 con 0x7f4178104630 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f417d948640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417819eda0 con 0x7f4178101d00 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f417d948640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417819ef60 con 0x7f4178106f60 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52032/0 (socket says 172.21.15.42:52032) 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 -- 172.21.15.42:0/2383322506 learned_addr learned my addr 172.21.15.42:0/2383322506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41777fe640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4178106f60 0x7f417819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f4176ffd640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178101d00 0x7f4178197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 -- 172.21.15.42:0/2383322506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4178106f60 msgr2=0x7f417819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4178106f60 0x7f417819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 -- 172.21.15.42:0/2383322506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178101d00 msgr2=0x7f4178197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178101d00 0x7f4178197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 -- 172.21.15.42:0/2383322506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009560 con 0x7f4178104630 2024-03-12T04:14:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41777fe640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4178106f60 0x7f417819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f4176ffd640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178101d00 0x7f4178197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.533+0000 7f41767fc640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178197680 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f416c000c00 tx=0x7f416c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.534+0000 7f417c946640 1 -- 172.21.15.42:0/2383322506 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0044e0 con 0x7f4178104630 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.534+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417819f1e0 con 0x7f4178104630 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.534+0000 7f417c946640 1 -- 172.21.15.42:0/2383322506 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f416c02de90 con 0x7f4178104630 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.534+0000 7f417c946640 1 -- 172.21.15.42:0/2383322506 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c005410 con 0x7f4178104630 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.534+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417819f6f0 con 0x7f4178104630 2024-03-12T04:14:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.535+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41780fd1a0 con 0x7f4178104630 2024-03-12T04:14:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.536+0000 7f417c946640 1 -- 172.21.15.42:0/2383322506 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f416c005570 con 0x7f4178104630 2024-03-12T04:14:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.536+0000 7f417c946640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f41600a1a30 0x7f41600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.536+0000 7f4176ffd640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f41600a1a30 0x7f41600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.536+0000 7f417c946640 1 -- 172.21.15.42:0/2383322506 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f416c0efe00 con 0x7f4178104630 2024-03-12T04:14:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.537+0000 7f4176ffd640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f41600a1a30 0x7f41600a3ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41680058c0 tx=0x7f4168005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.540+0000 7f417c946640 1 -- 172.21.15.42:0/2383322506 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f416c0b9040 con 0x7f4178104630 2024-03-12T04:14:45.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.723+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f417810ccc0 con 0x7f4178104630 2024-03-12T04:14:45.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.724+0000 7f417c946640 1 -- 172.21.15.42:0/2383322506 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f416c0bc700 con 0x7f4178104630 2024-03-12T04:14:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:45.727 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f41600a1a30 msgr2=0x7f41600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f41600a1a30 0x7f41600a3ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41680058c0 tx=0x7f4168005ec0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 msgr2=0x7f4178197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178197680 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f416c000c00 tx=0x7f416c0042f0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 shutdown_connections 2024-03-12T04:14:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4178106f60 0x7f417819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4178104630 0x7f4178197680 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f41600a1a30 0x7f41600a3ef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 --2- 172.21.15.42:0/2383322506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178101d00 0x7f4178197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.726+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 >> 172.21.15.42:0/2383322506 conn(0x7f41780fb6b0 msgr2=0x7f4178102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.727+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 shutdown_connections 2024-03-12T04:14:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.727+0000 7f417d948640 1 -- 172.21.15.42:0/2383322506 wait complete. 2024-03-12T04:14:45.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:45 smithi042 ceph-mon[32260]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-12T04:14:45.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:14:45.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:14:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 -- 172.21.15.42:0/935085861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 msgr2=0x7fae04107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 --2- 172.21.15.42:0/935085861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae04107c60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009ca0 tx=0x7fadf402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 -- 172.21.15.42:0/935085861 shutdown_connections 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 --2- 172.21.15.42:0/935085861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae04107c60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 --2- 172.21.15.42:0/935085861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04100a10 0x7fae04100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 --2- 172.21.15.42:0/935085861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae040ff820 0x7fae040ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 -- 172.21.15.42:0/935085861 >> 172.21.15.42:0/935085861 conn(0x7fae040fadb0 msgr2=0x7fae040fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 -- 172.21.15.42:0/935085861 shutdown_connections 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.844+0000 7fae096d4640 1 -- 172.21.15.42:0/935085861 wait complete. 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 Processor -- start 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 -- start start 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae040ff820 0x7fae041955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04100a10 0x7fae04195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae0419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419ec10 con 0x7fae041013d0 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419ed80 con 0x7fae040ff820 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae096d4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419ef40 con 0x7fae04100a10 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae037fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae0419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae037fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae0419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52038/0 (socket says 172.21.15.42:52038) 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae037fe640 1 -- 172.21.15.42:0/3227449839 learned_addr learned my addr 172.21.15.42:0/3227449839 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae02ffd640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae040ff820 0x7fae041955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae027fc640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04100a10 0x7fae04195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.845+0000 7fae037fe640 1 -- 172.21.15.42:0/3227449839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04100a10 msgr2=0x7fae04195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae037fe640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04100a10 0x7fae04195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae037fe640 1 -- 172.21.15.42:0/3227449839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae040ff820 msgr2=0x7fae041955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae037fe640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae040ff820 0x7fae041955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:45.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae037fe640 1 -- 172.21.15.42:0/3227449839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4009900 con 0x7fae041013d0 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae02ffd640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae040ff820 0x7fae041955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae037fe640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae0419e6d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fadf4037040 tx=0x7fadf4032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fadfbfff640 1 -- 172.21.15.42:0/3227449839 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4003970 con 0x7fae041013d0 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fadfbfff640 1 -- 172.21.15.42:0/3227449839 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fadf4004040 con 0x7fae041013d0 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0419f1c0 con 0x7fae041013d0 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fadfbfff640 1 -- 172.21.15.42:0/3227449839 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf403d630 con 0x7fae041013d0 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.846+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0419f6f0 con 0x7fae041013d0 2024-03-12T04:14:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.847+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae04106ae0 con 0x7fae041013d0 2024-03-12T04:14:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.848+0000 7fadfbfff640 1 -- 172.21.15.42:0/3227449839 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fadf403b040 con 0x7fae041013d0 2024-03-12T04:14:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.849+0000 7fadfbfff640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.849+0000 7fae02ffd640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.849+0000 7fadfbfff640 1 -- 172.21.15.42:0/3227449839 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fadf40f0180 con 0x7fae041013d0 2024-03-12T04:14:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.849+0000 7fae02ffd640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0058c0 tx=0x7fadfc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:45.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:45.853+0000 7fadfbfff640 1 -- 172.21.15.42:0/3227449839 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadf40b93c0 con 0x7fae041013d0 2024-03-12T04:14:45.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:45 smithi119 ceph-mon[36119]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-12T04:14:45.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:45 smithi136 ceph-mon[35571]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-12T04:14:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.033+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae0402ce40 con 0x7fae041013d0 2024-03-12T04:14:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.034+0000 7fadfbfff640 1 -- 172.21.15.42:0/3227449839 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadf40bca80 con 0x7fae041013d0 2024-03-12T04:14:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 msgr2=0x7fadf00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0058c0 tx=0x7fadfc005850 comp rx=0 tx=0).stop 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 msgr2=0x7fae0419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae0419e6d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fadf4037040 tx=0x7fadf4032980 comp rx=0 tx=0).stop 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 shutdown_connections 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae041013d0 0x7fae0419e6d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae04100a10 0x7fae04195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fadf00a1a30 0x7fadf00a3ef0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 --2- 172.21.15.42:0/3227449839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae040ff820 0x7fae041955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 >> 172.21.15.42:0/3227449839 conn(0x7fae040fadb0 msgr2=0x7fae04103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 shutdown_connections 2024-03-12T04:14:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.036+0000 7fae096d4640 1 -- 172.21.15.42:0/3227449839 wait complete. 2024-03-12T04:14:46.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 -- 172.21.15.42:0/3434504442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c101290 msgr2=0x7fd55c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 --2- 172.21.15.42:0/3434504442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c101290 0x7fd55c101710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd550009900 tx=0x7fd55002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 -- 172.21.15.42:0/3434504442 shutdown_connections 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 --2- 172.21.15.42:0/3434504442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c101c50 0x7fd55c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 --2- 172.21.15.42:0/3434504442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c101290 0x7fd55c101710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 --2- 172.21.15.42:0/3434504442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c1000a0 0x7fd55c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 -- 172.21.15.42:0/3434504442 >> 172.21.15.42:0/3434504442 conn(0x7fd55c0fb630 msgr2=0x7fd55c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 -- 172.21.15.42:0/3434504442 shutdown_connections 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.153+0000 7fd562839640 1 -- 172.21.15.42:0/3434504442 wait complete. 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 Processor -- start 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 -- start start 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c1000a0 0x7fd55c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c101290 0x7fd55c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c101c50 0x7fd55c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c19ec70 con 0x7fd55c1000a0 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c19ede0 con 0x7fd55c101290 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd562839640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c19efa0 con 0x7fd55c101c50 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c1000a0 0x7fd55c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c1000a0 0x7fd55c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52052/0 (socket says 172.21.15.42:52052) 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 -- 172.21.15.42:0/1710046456 learned_addr learned my addr 172.21.15.42:0/1710046456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55b7fe640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c101290 0x7fd55c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd560dd6640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c101c50 0x7fd55c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 -- 172.21.15.42:0/1710046456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c101c50 msgr2=0x7fd55c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c101c50 0x7fd55c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 -- 172.21.15.42:0/1710046456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c101290 msgr2=0x7fd55c1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c101290 0x7fd55c1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 -- 172.21.15.42:0/1710046456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550009560 con 0x7fd55c1000a0 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55b7fe640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c101290 0x7fd55c1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.154+0000 7fd55bfff640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c1000a0 0x7fd55c197180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00d950 tx=0x7fd54c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.155+0000 7fd5597fa640 1 -- 172.21.15.42:0/1710046456 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c009c30 con 0x7fd55c1000a0 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.155+0000 7fd5597fa640 1 -- 172.21.15.42:0/1710046456 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd54c009d90 con 0x7fd55c1000a0 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.155+0000 7fd5597fa640 1 -- 172.21.15.42:0/1710046456 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00f6b0 con 0x7fd55c1000a0 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.155+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c19f280 con 0x7fd55c1000a0 2024-03-12T04:14:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.155+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c19f740 con 0x7fd55c1000a0 2024-03-12T04:14:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.156+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c107530 con 0x7fd55c1000a0 2024-03-12T04:14:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.157+0000 7fd5597fa640 1 -- 172.21.15.42:0/1710046456 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd54c010a30 con 0x7fd55c1000a0 2024-03-12T04:14:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.157+0000 7fd5597fa640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd5440a1ab0 0x7fd5440a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.157+0000 7fd55b7fe640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd5440a1ab0 0x7fd5440a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.157+0000 7fd5597fa640 1 -- 172.21.15.42:0/1710046456 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd54c0cd1a0 con 0x7fd55c1000a0 2024-03-12T04:14:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.158+0000 7fd55b7fe640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd5440a1ab0 0x7fd5440a3f70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd55c198690 tx=0x7fd550005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.162+0000 7fd5597fa640 1 -- 172.21.15.42:0/1710046456 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd54c0963e0 con 0x7fd55c1000a0 2024-03-12T04:14:46.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.341+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd55c02ce40 con 0x7fd55c1000a0 2024-03-12T04:14:46.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.342+0000 7fd5597fa640 1 -- 172.21.15.42:0/1710046456 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd54c099aa0 con 0x7fd55c1000a0 2024-03-12T04:14:46.345 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd5440a1ab0 msgr2=0x7fd5440a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd5440a1ab0 0x7fd5440a3f70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd55c198690 tx=0x7fd550005c00 comp rx=0 tx=0).stop 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c1000a0 msgr2=0x7fd55c197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c1000a0 0x7fd55c197180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00d950 tx=0x7fd54c00de10 comp rx=0 tx=0).stop 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 shutdown_connections 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c101c50 0x7fd55c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fd5440a1ab0 0x7fd5440a3f70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c101290 0x7fd55c1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 --2- 172.21.15.42:0/1710046456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd55c1000a0 0x7fd55c197180 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 >> 172.21.15.42:0/1710046456 conn(0x7fd55c0fb630 msgr2=0x7fd55c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 shutdown_connections 2024-03-12T04:14:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.344+0000 7fd562839640 1 -- 172.21.15.42:0/1710046456 wait complete. 2024-03-12T04:14:46.358 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:14:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.463+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4133512619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 msgr2=0x7f959c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4133512619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c1092d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9594009ca0 tx=0x7f959402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4133512619 shutdown_connections 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4133512619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c1092d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4133512619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c104550 0x7f959c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4133512619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f959c101c20 0x7f959c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4133512619 >> 172.21.15.42:0/4133512619 conn(0x7f959c0fb630 msgr2=0x7f959c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4133512619 shutdown_connections 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4133512619 wait complete. 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 Processor -- start 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- start start 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c101c20 0x7f959c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f959c104550 0x7f959c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c19ebf0 con 0x7f959c106e80 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c19ed60 con 0x7f959c104550 2024-03-12T04:14:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a2ef1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c19ef20 con 0x7f959c101c20 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a148e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a148e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52068/0 (socket says 172.21.15.42:52068) 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.464+0000 7f95a148e640 1 -- 172.21.15.42:0/4171810808 learned_addr learned my addr 172.21.15.42:0/4171810808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f959bfff640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f959c104550 0x7f959c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a0c8d640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c101c20 0x7f959c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a148e640 1 -- 172.21.15.42:0/4171810808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c101c20 msgr2=0x7f959c197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a148e640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c101c20 0x7f959c197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a148e640 1 -- 172.21.15.42:0/4171810808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f959c104550 msgr2=0x7f959c197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a148e640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f959c104550 0x7f959c197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a148e640 1 -- 172.21.15.42:0/4171810808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9594009900 con 0x7f959c106e80 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a148e640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c19e6b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9594037040 tx=0x7f9594032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f9599ffb640 1 -- 172.21.15.42:0/4171810808 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594034e70 con 0x7f959c106e80 2024-03-12T04:14:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f9599ffb640 1 -- 172.21.15.42:0/4171810808 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9594003e20 con 0x7f959c106e80 2024-03-12T04:14:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c19f1a0 con 0x7f959c106e80 2024-03-12T04:14:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f9599ffb640 1 -- 172.21.15.42:0/4171810808 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594004800 con 0x7f959c106e80 2024-03-12T04:14:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.465+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c19f650 con 0x7f959c106e80 2024-03-12T04:14:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.466+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c0fd120 con 0x7f959c106e80 2024-03-12T04:14:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.467+0000 7f9599ffb640 1 -- 172.21.15.42:0/4171810808 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f959403b040 con 0x7f959c106e80 2024-03-12T04:14:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.468+0000 7f9599ffb640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f958c0a1a30 0x7f958c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.468+0000 7f95a0c8d640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f958c0a1a30 0x7f958c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.468+0000 7f9599ffb640 1 -- 172.21.15.42:0/4171810808 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f95940efb90 con 0x7f959c106e80 2024-03-12T04:14:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.468+0000 7f95a0c8d640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f958c0a1a30 0x7f958c0a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9590009b60 tx=0x7f9590005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.472+0000 7f9599ffb640 1 -- 172.21.15.42:0/4171810808 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95940b8d70 con 0x7f959c106e80 2024-03-12T04:14:46.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.654+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f959c02cdd0 con 0x7f959c106e80 2024-03-12T04:14:46.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.655+0000 7f9599ffb640 1 -- 172.21.15.42:0/4171810808 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95940bc430 con 0x7f959c106e80 2024-03-12T04:14:46.658 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f958c0a1a30 msgr2=0x7f958c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f958c0a1a30 0x7f958c0a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9590009b60 tx=0x7f9590005f90 comp rx=0 tx=0).stop 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 msgr2=0x7f959c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c19e6b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9594037040 tx=0x7f9594032980 comp rx=0 tx=0).stop 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 shutdown_connections 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f959c106e80 0x7f959c19e6b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f958c0a1a30 0x7f958c0a3ef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f959c104550 0x7f959c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 --2- 172.21.15.42:0/4171810808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c101c20 0x7f959c197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 >> 172.21.15.42:0/4171810808 conn(0x7f959c0fb630 msgr2=0x7f959c102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 shutdown_connections 2024-03-12T04:14:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.657+0000 7f95a2ef1640 1 -- 172.21.15.42:0/4171810808 wait complete. 2024-03-12T04:14:46.669 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:46 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2383322506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:46.669 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:46 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3227449839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:46.669 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:46 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1710046456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:46.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:14:46.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 -- 172.21.15.42:0/2592883947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 msgr2=0x7f569c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 --2- 172.21.15.42:0/2592883947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c101790 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5690009900 tx=0x7f569002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 -- 172.21.15.42:0/2592883947 shutdown_connections 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 --2- 172.21.15.42:0/2592883947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f569c101cd0 0x7f569c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 --2- 172.21.15.42:0/2592883947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c101790 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 --2- 172.21.15.42:0/2592883947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f569c100120 0x7f569c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 -- 172.21.15.42:0/2592883947 >> 172.21.15.42:0/2592883947 conn(0x7f569c0fb6b0 msgr2=0x7f569c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 -- 172.21.15.42:0/2592883947 shutdown_connections 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 -- 172.21.15.42:0/2592883947 wait complete. 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 Processor -- start 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.771+0000 7f56a258a640 1 -- start start 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a258a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f569c100120 0x7f569c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a258a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a258a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f569c101cd0 0x7f569c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a258a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c19edd0 con 0x7f569c101310 2024-03-12T04:14:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a258a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c19ef40 con 0x7f569c100120 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a258a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c19f100 con 0x7f569c101cd0 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52100/0 (socket says 172.21.15.42:52100) 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 -- 172.21.15.42:0/1451814515 learned_addr learned my addr 172.21.15.42:0/1451814515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a0b27640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f569c101cd0 0x7f569c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569bfff640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f569c100120 0x7f569c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 -- 172.21.15.42:0/1451814515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f569c101cd0 msgr2=0x7f569c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f569c101cd0 0x7f569c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 -- 172.21.15.42:0/1451814515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f569c100120 msgr2=0x7f569c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f569c100120 0x7f569c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 -- 172.21.15.42:0/1451814515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5690009560 con 0x7f569c101310 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56a0b27640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f569c101cd0 0x7f569c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f569b7fe640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c197820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5690036040 tx=0x7f5690004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:46.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.772+0000 7f56997fa640 1 -- 172.21.15.42:0/1451814515 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690003800 con 0x7f569c101310 2024-03-12T04:14:46.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.773+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c19f380 con 0x7f569c101310 2024-03-12T04:14:46.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.773+0000 7f56997fa640 1 -- 172.21.15.42:0/1451814515 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569002de90 con 0x7f569c101310 2024-03-12T04:14:46.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.773+0000 7f56997fa640 1 -- 172.21.15.42:0/1451814515 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690005600 con 0x7f569c101310 2024-03-12T04:14:46.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.773+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c19f910 con 0x7f569c101310 2024-03-12T04:14:46.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.773+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c1075b0 con 0x7f569c101310 2024-03-12T04:14:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.774+0000 7f56997fa640 1 -- 172.21.15.42:0/1451814515 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5690004dc0 con 0x7f569c101310 2024-03-12T04:14:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.775+0000 7f56997fa640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f56840a1a30 0x7f56840a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.775+0000 7f569bfff640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f56840a1a30 0x7f56840a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.775+0000 7f56997fa640 1 -- 172.21.15.42:0/1451814515 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f56900ef850 con 0x7f569c101310 2024-03-12T04:14:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.776+0000 7f569bfff640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f56840a1a30 0x7f56840a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f569c101170 tx=0x7f568c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:46.780+0000 7f56997fa640 1 -- 172.21.15.42:0/1451814515 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56900b8a30 con 0x7f569c101310 2024-03-12T04:14:46.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:46 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2383322506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:46.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:46 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3227449839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:46.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:46 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1710046456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:46 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2383322506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:46 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3227449839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:46 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1710046456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:47.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.001+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f569c02ce40 con 0x7f569c101310 2024-03-12T04:14:47.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.002+0000 7f56997fa640 1 -- 172.21.15.42:0/1451814515 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f56900bc0f0 con 0x7f569c101310 2024-03-12T04:14:47.004 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:47.005 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:13:01.029979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:14:47.005 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f56840a1a30 msgr2=0x7f56840a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f56840a1a30 0x7f56840a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f569c101170 tx=0x7f568c008040 comp rx=0 tx=0).stop 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 msgr2=0x7f569c197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c197820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5690036040 tx=0x7f5690004370 comp rx=0 tx=0).stop 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 shutdown_connections 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f569c101cd0 0x7f569c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c101310 0x7f569c197820 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f56840a1a30 0x7f56840a3ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 --2- 172.21.15.42:0/1451814515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f569c100120 0x7f569c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 >> 172.21.15.42:0/1451814515 conn(0x7f569c0fb6b0 msgr2=0x7f569c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 shutdown_connections 2024-03-12T04:14:47.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.004+0000 7f56a258a640 1 -- 172.21.15.42:0/1451814515 wait complete. 2024-03-12T04:14:47.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 -- 172.21.15.42:0/3276579569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b8102d60 msgr2=0x7f82b81095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 --2- 172.21.15.42:0/3276579569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b8102d60 0x7f82b81095f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009ca0 tx=0x7f82b002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 -- 172.21.15.42:0/3276579569 shutdown_connections 2024-03-12T04:14:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 --2- 172.21.15.42:0/3276579569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b8102d60 0x7f82b81095f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 --2- 172.21.15.42:0/3276579569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b81023a0 0x7f82b8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 --2- 172.21.15.42:0/3276579569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81011b0 0x7f82b81015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 -- 172.21.15.42:0/3276579569 >> 172.21.15.42:0/3276579569 conn(0x7f82b80fc760 msgr2=0x7f82b80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 -- 172.21.15.42:0/3276579569 shutdown_connections 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.118+0000 7f82bf70c640 1 -- 172.21.15.42:0/3276579569 wait complete. 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 Processor -- start 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 -- start start 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81011b0 0x7f82b8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b81023a0 0x7f82b8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b8102d60 0x7f82b819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b819ebf0 con 0x7f82b81023a0 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b819ed60 con 0x7f82b81011b0 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bf70c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b819ef20 con 0x7f82b8102d60 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b81023a0 0x7f82b8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b81023a0 0x7f82b8197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52116/0 (socket says 172.21.15.42:52116) 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 -- 172.21.15.42:0/1904958729 learned_addr learned my addr 172.21.15.42:0/1904958729 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bd4a8640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81011b0 0x7f82b8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bdca9640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b8102d60 0x7f82b819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 -- 172.21.15.42:0/1904958729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b8102d60 msgr2=0x7f82b819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b8102d60 0x7f82b819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 -- 172.21.15.42:0/1904958729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81011b0 msgr2=0x7f82b8197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81011b0 0x7f82b8197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 -- 172.21.15.42:0/1904958729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b0009900 con 0x7f82b81023a0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bd4a8640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81011b0 0x7f82b8197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82bcca7640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b81023a0 0x7f82b8197640 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82a800d950 tx=0x7f82a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.119+0000 7f82ae7fc640 1 -- 172.21.15.42:0/1904958729 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8009c30 con 0x7f82b81023a0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.120+0000 7f82ae7fc640 1 -- 172.21.15.42:0/1904958729 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82a8009d90 con 0x7f82b81023a0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.120+0000 7f82ae7fc640 1 -- 172.21.15.42:0/1904958729 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800f6b0 con 0x7f82b81023a0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.120+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b819f200 con 0x7f82b81023a0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.120+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b819f710 con 0x7f82b81023a0 2024-03-12T04:14:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.121+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b81085a0 con 0x7f82b81023a0 2024-03-12T04:14:47.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.122+0000 7f82ae7fc640 1 -- 172.21.15.42:0/1904958729 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f82a8010aa0 con 0x7f82b81023a0 2024-03-12T04:14:47.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.122+0000 7f82ae7fc640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82a40a1a30 0x7f82a40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.122+0000 7f82ae7fc640 1 -- 172.21.15.42:0/1904958729 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82a80cd1a0 con 0x7f82b81023a0 2024-03-12T04:14:47.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.122+0000 7f82bd4a8640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82a40a1a30 0x7f82a40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.123+0000 7f82bd4a8640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82a40a1a30 0x7f82a40a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82b8102200 tx=0x7f82b400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:47.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.127+0000 7f82ae7fc640 1 -- 172.21.15.42:0/1904958729 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82a80963e0 con 0x7f82b81023a0 2024-03-12T04:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.309+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82b802ce40 con 0x7f82b81023a0 2024-03-12T04:14:47.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.310+0000 7f82ae7fc640 1 -- 172.21.15.42:0/1904958729 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f82a8099aa0 con 0x7f82b81023a0 2024-03-12T04:14:47.313 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:47.313 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.312+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82a40a1a30 msgr2=0x7f82a40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82a40a1a30 0x7f82a40a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82b8102200 tx=0x7f82b400a560 comp rx=0 tx=0).stop 2024-03-12T04:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b81023a0 msgr2=0x7f82b8197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b81023a0 0x7f82b8197640 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82a800d950 tx=0x7f82a800de10 comp rx=0 tx=0).stop 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 shutdown_connections 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b8102d60 0x7f82b819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b81023a0 0x7f82b8197640 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f82a40a1a30 0x7f82a40a3ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 --2- 172.21.15.42:0/1904958729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81011b0 0x7f82b8197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 >> 172.21.15.42:0/1904958729 conn(0x7f82b80fc760 msgr2=0x7f82b8105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 shutdown_connections 2024-03-12T04:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.313+0000 7f82bf70c640 1 -- 172.21.15.42:0/1904958729 wait complete. 2024-03-12T04:14:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:14:47.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:14:47.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 -- 172.21.15.42:0/4030704930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0073b80 msgr2=0x7f1cd0073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 --2- 172.21.15.42:0/4030704930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0073b80 0x7f1cd0073fe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009900 tx=0x7f1cc402bcd0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 -- 172.21.15.42:0/4030704930 shutdown_connections 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 --2- 172.21.15.42:0/4030704930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd0109040 0x7f1cd010b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 --2- 172.21.15.42:0/4030704930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0073b80 0x7f1cd0073fe0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 --2- 172.21.15.42:0/4030704930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cd0075160 0x7f1cd00735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 -- 172.21.15.42:0/4030704930 >> 172.21.15.42:0/4030704930 conn(0x7f1cd00fc720 msgr2=0x7f1cd00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 -- 172.21.15.42:0/4030704930 shutdown_connections 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 -- 172.21.15.42:0/4030704930 wait complete. 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 Processor -- start 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 -- start start 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cd0073b80 0x7f1cd01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd0075160 0x7f1cd0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.431+0000 7f1cd7464640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0109040 0x7f1cd01a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd7464640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01a0fd0 con 0x7f1cd0109040 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd7464640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01a1140 con 0x7f1cd0073b80 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd7464640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01a1300 con 0x7f1cd0075160 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0109040 0x7f1cd01a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0109040 0x7f1cd01a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52140/0 (socket says 172.21.15.42:52140) 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 -- 172.21.15.42:0/3984516594 learned_addr learned my addr 172.21.15.42:0/3984516594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd49ff640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd0075160 0x7f1cd0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5200640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cd0073b80 0x7f1cd01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 -- 172.21.15.42:0/3984516594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd0075160 msgr2=0x7f1cd0199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd0075160 0x7f1cd0199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 -- 172.21.15.42:0/3984516594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cd0073b80 msgr2=0x7f1cd01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cd0073b80 0x7f1cd01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 -- 172.21.15.42:0/3984516594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4009560 con 0x7f1cd0109040 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd49ff640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd0075160 0x7f1cd0199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5200640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cd0073b80 0x7f1cd01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd5a01640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0109040 0x7f1cd01a0a90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009fd0 tx=0x7f1cc800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cce7fc640 1 -- 172.21.15.42:0/3984516594 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8011830 con 0x7f1cd0109040 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cce7fc640 1 -- 172.21.15.42:0/3984516594 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1cc8011e40 con 0x7f1cd0109040 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd01a15e0 con 0x7f1cd0109040 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cce7fc640 1 -- 172.21.15.42:0/3984516594 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800f2e0 con 0x7f1cd0109040 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.432+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd01a1b20 con 0x7f1cd0109040 2024-03-12T04:14:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.433+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd0107e00 con 0x7f1cd0109040 2024-03-12T04:14:47.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.434+0000 7f1cce7fc640 1 -- 172.21.15.42:0/3984516594 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1cc8011990 con 0x7f1cd0109040 2024-03-12T04:14:47.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.435+0000 7f1cce7fc640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1cb80a1a30 0x7f1cb80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.435+0000 7f1cd5200640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1cb80a1a30 0x7f1cb80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.435+0000 7f1cce7fc640 1 -- 172.21.15.42:0/3984516594 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1cc80cccc0 con 0x7f1cd0109040 2024-03-12T04:14:47.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.435+0000 7f1cd5200640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1cb80a1a30 0x7f1cb80a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009fa0 tx=0x7f1cc0009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:47.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.439+0000 7f1cce7fc640 1 -- 172.21.15.42:0/3984516594 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1cc8095f00 con 0x7f1cd0109040 2024-03-12T04:14:47.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.621+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cd002ce40 con 0x7f1cd0109040 2024-03-12T04:14:47.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.622+0000 7f1cce7fc640 1 -- 172.21.15.42:0/3984516594 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cc80995c0 con 0x7f1cd0109040 2024-03-12T04:14:47.625 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:14:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1cb80a1a30 msgr2=0x7f1cb80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1cb80a1a30 0x7f1cb80a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009fa0 tx=0x7f1cc0009570 comp rx=0 tx=0).stop 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0109040 msgr2=0x7f1cd01a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0109040 0x7f1cd01a0a90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009fd0 tx=0x7f1cc800b8e0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 shutdown_connections 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cd0109040 0x7f1cd01a0a90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd0075160 0x7f1cd0199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f1cb80a1a30 0x7f1cb80a3ef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 --2- 172.21.15.42:0/3984516594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cd0073b80 0x7f1cd01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 >> 172.21.15.42:0/3984516594 conn(0x7f1cd00fc720 msgr2=0x7f1cd0104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 shutdown_connections 2024-03-12T04:14:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.624+0000 7f1cd7464640 1 -- 172.21.15.42:0/3984516594 wait complete. 2024-03-12T04:14:47.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:14:47.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 -- 172.21.15.42:0/691756260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c101190 msgr2=0x7f926c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 --2- 172.21.15.42:0/691756260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c101190 0x7f926c1015b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9260009900 tx=0x7f926002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:47.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 -- 172.21.15.42:0/691756260 shutdown_connections 2024-03-12T04:14:47.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 --2- 172.21.15.42:0/691756260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926c102d40 0x7f926c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 --2- 172.21.15.42:0/691756260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c102380 0x7f926c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 --2- 172.21.15.42:0/691756260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c101190 0x7f926c1015b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 -- 172.21.15.42:0/691756260 >> 172.21.15.42:0/691756260 conn(0x7f926c0fc720 msgr2=0x7f926c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 -- 172.21.15.42:0/691756260 shutdown_connections 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.741+0000 7f9272d4f640 1 -- 172.21.15.42:0/691756260 wait complete. 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 Processor -- start 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 -- start start 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926c101190 0x7f926c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c102380 0x7f926c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c102d40 0x7f926c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c19ebd0 con 0x7f926c102380 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c19ed40 con 0x7f926c102d40 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9272d4f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c19ef00 con 0x7f926c101190 2024-03-12T04:14:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c102380 0x7f926c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c102380 0x7f926c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52168/0 (socket says 172.21.15.42:52168) 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 -- 172.21.15.42:0/2542833766 learned_addr learned my addr 172.21.15.42:0/2542833766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f92712ec640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c102d40 0x7f926c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f9270aeb640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926c101190 0x7f926c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 -- 172.21.15.42:0/2542833766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926c101190 msgr2=0x7f926c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926c101190 0x7f926c1970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 -- 172.21.15.42:0/2542833766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c102d40 msgr2=0x7f926c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c102d40 0x7f926c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 -- 172.21.15.42:0/2542833766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9260009560 con 0x7f926c102380 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.742+0000 7f926bfff640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c102380 0x7f926c197620 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f925800c920 tx=0x7f925800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.743+0000 7f9269ffb640 1 -- 172.21.15.42:0/2542833766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258007c70 con 0x7f926c102380 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.743+0000 7f9269ffb640 1 -- 172.21.15.42:0/2542833766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9258007dd0 con 0x7f926c102380 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.743+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c19f1e0 con 0x7f926c102380 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.743+0000 7f9269ffb640 1 -- 172.21.15.42:0/2542833766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92580187c0 con 0x7f926c102380 2024-03-12T04:14:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.743+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c19f740 con 0x7f926c102380 2024-03-12T04:14:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.744+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c108530 con 0x7f926c102380 2024-03-12T04:14:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.745+0000 7f9269ffb640 1 -- 172.21.15.42:0/2542833766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f925800f450 con 0x7f926c102380 2024-03-12T04:14:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.745+0000 7f9269ffb640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f925c0a1ab0 0x7f925c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.745+0000 7f9269ffb640 1 -- 172.21.15.42:0/2542833766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f92580cd4b0 con 0x7f926c102380 2024-03-12T04:14:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.745+0000 7f9270aeb640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f925c0a1ab0 0x7f925c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.746+0000 7f9270aeb640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f925c0a1ab0 0x7f925c0a3f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9260036040 tx=0x7f9260005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:47.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.750+0000 7f9269ffb640 1 -- 172.21.15.42:0/2542833766 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92580966f0 con 0x7f926c102380 2024-03-12T04:14:47.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.931+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f926c02ce40 con 0x7f926c102380 2024-03-12T04:14:47.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.932+0000 7f9269ffb640 1 -- 172.21.15.42:0/2542833766 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9258099db0 con 0x7f926c102380 2024-03-12T04:14:47.935 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:14:47.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f925c0a1ab0 msgr2=0x7f925c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f925c0a1ab0 0x7f925c0a3f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9260036040 tx=0x7f9260005fd0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c102380 msgr2=0x7f926c197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c102380 0x7f926c197620 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f925800c920 tx=0x7f925800cde0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 shutdown_connections 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f925c0a1ab0 0x7f925c0a3f70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c102d40 0x7f926c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f926c102380 0x7f926c197620 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 --2- 172.21.15.42:0/2542833766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926c101190 0x7f926c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 >> 172.21.15.42:0/2542833766 conn(0x7f926c0fc720 msgr2=0x7f926c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 shutdown_connections 2024-03-12T04:14:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:47.934+0000 7f9272d4f640 1 -- 172.21.15.42:0/2542833766 wait complete. 2024-03-12T04:14:47.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:47 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/4171810808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:47.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:47 smithi119 ceph-mon[36119]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:47.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:47 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1451814515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:47.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:47 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1904958729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:47.946 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:47 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/4171810808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:47.946 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:47 smithi042 ceph-mon[32260]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:47.947 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:47 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1451814515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:47.947 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:47 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1904958729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:47.948 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:14:47.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:47 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/4171810808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:47.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:47 smithi136 ceph-mon[35571]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:47.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:47 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1451814515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:47.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:47 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1904958729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:48.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 -- 172.21.15.42:0/139387327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64102d40 msgr2=0x7f7b641095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/139387327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64102d40 0x7f7b641095d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009ca0 tx=0x7f7b5c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 -- 172.21.15.42:0/139387327 shutdown_connections 2024-03-12T04:14:48.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/139387327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64102d40 0x7f7b641095d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/139387327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64102380 0x7f7b64102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/139387327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b64101190 0x7f7b641015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 -- 172.21.15.42:0/139387327 >> 172.21.15.42:0/139387327 conn(0x7f7b640fc720 msgr2=0x7f7b640feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 -- 172.21.15.42:0/139387327 shutdown_connections 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.050+0000 7f7b6c40e640 1 -- 172.21.15.42:0/139387327 wait complete. 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 Processor -- start 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 -- start start 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64101190 0x7f7b64197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64102380 0x7f7b64197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b64102d40 0x7f7b6419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6419ec40 con 0x7f7b64101190 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6419edb0 con 0x7f7b64102d40 2024-03-12T04:14:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6419ef70 con 0x7f7b64102380 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64101190 0x7f7b64197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64101190 0x7f7b64197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52184/0 (socket says 172.21.15.42:52184) 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 -- 172.21.15.42:0/1454523622 learned_addr learned my addr 172.21.15.42:0/1454523622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 -- 172.21.15.42:0/1454523622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64102380 msgr2=0x7f7b64197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64102380 0x7f7b64197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 -- 172.21.15.42:0/1454523622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b64102d40 msgr2=0x7f7b6419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b64102d40 0x7f7b6419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 -- 172.21.15.42:0/1454523622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c009900 con 0x7f7b64101190 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6a1aa640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64101190 0x7f7b64197150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000b450 tx=0x7f7b6000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b5b7fe640 1 -- 172.21.15.42:0/1454523622 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011890 con 0x7f7b64101190 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b5b7fe640 1 -- 172.21.15.42:0/1454523622 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b600103e0 con 0x7f7b64101190 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6419f250 con 0x7f7b64101190 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.051+0000 7f7b5b7fe640 1 -- 172.21.15.42:0/1454523622 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011ea0 con 0x7f7b64101190 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.052+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6419f7e0 con 0x7f7b64101190 2024-03-12T04:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.053+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b641084b0 con 0x7f7b64101190 2024-03-12T04:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.053+0000 7f7b5b7fe640 1 -- 172.21.15.42:0/1454523622 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7b60010550 con 0x7f7b64101190 2024-03-12T04:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.054+0000 7f7b5b7fe640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7b500a1a30 0x7f7b500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.054+0000 7f7b699a9640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7b500a1a30 0x7f7b500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.054+0000 7f7b5b7fe640 1 -- 172.21.15.42:0/1454523622 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7b600cd290 con 0x7f7b64101190 2024-03-12T04:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.054+0000 7f7b699a9640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7b500a1a30 0x7f7b500a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b64198660 tx=0x7f7b54005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:48.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.058+0000 7f7b5b7fe640 1 -- 172.21.15.42:0/1454523622 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b60096470 con 0x7f7b64101190 2024-03-12T04:14:48.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.240+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b6410cab0 con 0x7f7b64101190 2024-03-12T04:14:48.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.240+0000 7f7b5b7fe640 1 -- 172.21.15.42:0/1454523622 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b60009280 con 0x7f7b64101190 2024-03-12T04:14:48.243 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:14:48.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7b500a1a30 msgr2=0x7f7b500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7b500a1a30 0x7f7b500a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b64198660 tx=0x7f7b54005c70 comp rx=0 tx=0).stop 2024-03-12T04:14:48.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64101190 msgr2=0x7f7b64197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64101190 0x7f7b64197150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000b450 tx=0x7f7b6000b910 comp rx=0 tx=0).stop 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 shutdown_connections 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f7b500a1a30 0x7f7b500a3ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b64102d40 0x7f7b6419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64102380 0x7f7b64197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 --2- 172.21.15.42:0/1454523622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64101190 0x7f7b64197150 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 >> 172.21.15.42:0/1454523622 conn(0x7f7b640fc720 msgr2=0x7f7b64105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 shutdown_connections 2024-03-12T04:14:48.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.243+0000 7f7b6c40e640 1 -- 172.21.15.42:0/1454523622 wait complete. 2024-03-12T04:14:48.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 -- 172.21.15.42:0/2111519174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 msgr2=0x7ffaa01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/2111519174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa01084e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009ca0 tx=0x7ffa9c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 -- 172.21.15.42:0/2111519174 shutdown_connections 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/2111519174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa01084e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/2111519174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa0101290 0x7ffaa0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/2111519174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaa01000a0 0x7ffaa01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 -- 172.21.15.42:0/2111519174 >> 172.21.15.42:0/2111519174 conn(0x7ffaa00fb610 msgr2=0x7ffaa00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 -- 172.21.15.42:0/2111519174 shutdown_connections 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 -- 172.21.15.42:0/2111519174 wait complete. 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.356+0000 7ffaa87d7640 1 Processor -- start 2024-03-12T04:14:48.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 -- start start 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa01000a0 0x7ffaa0103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaa0101290 0x7ffaa0102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa0102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0103340 con 0x7ffaa0101c50 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa01034b0 con 0x7ffaa0101290 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa01a1140 con 0x7ffaa01000a0 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa0102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa0102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52196/0 (socket says 172.21.15.42:52196) 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 -- 172.21.15.42:0/601402342 learned_addr learned my addr 172.21.15.42:0/601402342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa5d72640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaa0101290 0x7ffaa0102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6573640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa01000a0 0x7ffaa0103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 -- 172.21.15.42:0/601402342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa01000a0 msgr2=0x7ffaa0103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa01000a0 0x7ffaa0103fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 -- 172.21.15.42:0/601402342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaa0101290 msgr2=0x7ffaa0102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaa0101290 0x7ffaa0102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 -- 172.21.15.42:0/601402342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c009900 con 0x7ffaa0101c50 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6573640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa01000a0 0x7ffaa0103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa6d74640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa0102bb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c033000 tx=0x7ffa9c032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffa977fe640 1 -- 172.21.15.42:0/601402342 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c034e70 con 0x7ffaa0101c50 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffa977fe640 1 -- 172.21.15.42:0/601402342 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa9c003e20 con 0x7ffaa0101c50 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa01a1340 con 0x7ffaa0101c50 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.357+0000 7ffa977fe640 1 -- 172.21.15.42:0/601402342 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c004840 con 0x7ffaa0101c50 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.358+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa01a17f0 con 0x7ffaa0101c50 2024-03-12T04:14:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.358+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa010b960 con 0x7ffaa0101c50 2024-03-12T04:14:48.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.360+0000 7ffa977fe640 1 -- 172.21.15.42:0/601402342 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ffa9c03b040 con 0x7ffaa0101c50 2024-03-12T04:14:48.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.361+0000 7ffa977fe640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffa8c0a1a30 0x7ffa8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.361+0000 7ffaa6573640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffa8c0a1a30 0x7ffa8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.361+0000 7ffa977fe640 1 -- 172.21.15.42:0/601402342 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffa9c0eff30 con 0x7ffaa0101c50 2024-03-12T04:14:48.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.361+0000 7ffaa6573640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffa8c0a1a30 0x7ffa8c0a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffaa01010f0 tx=0x7ffa98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:48.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.364+0000 7ffa977fe640 1 -- 172.21.15.42:0/601402342 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa9c0b9170 con 0x7ffaa0101c50 2024-03-12T04:14:48.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.586+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa010baf0 con 0x7ffaa0101c50 2024-03-12T04:14:48.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.587+0000 7ffa977fe640 1 -- 172.21.15.42:0/601402342 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11388 (secure 0 0 0) 0x7ffa9c0bc830 con 0x7ffaa0101c50 2024-03-12T04:14:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:13:01.029979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:14:48.590 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-03-12T04:14:48.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.588+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffa8c0a1a30 msgr2=0x7ffa8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.588+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffa8c0a1a30 0x7ffa8c0a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffaa01010f0 tx=0x7ffa98005f90 comp rx=0 tx=0).stop 2024-03-12T04:14:48.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.588+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 msgr2=0x7ffaa0102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.588+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa0102bb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c033000 tx=0x7ffa9c032980 comp rx=0 tx=0).stop 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 shutdown_connections 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa0101c50 0x7ffaa0102bb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ffa8c0a1a30 0x7ffa8c0a3ef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaa0101290 0x7ffaa0102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 --2- 172.21.15.42:0/601402342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa01000a0 0x7ffaa0103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 >> 172.21.15.42:0/601402342 conn(0x7ffaa00fb610 msgr2=0x7ffaa00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 shutdown_connections 2024-03-12T04:14:48.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.589+0000 7ffaa87d7640 1 -- 172.21.15.42:0/601402342 wait complete. 2024-03-12T04:14:48.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:48.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 -- 172.21.15.42:0/3969183017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe508102d70 msgr2=0x7fe508109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3969183017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe508102d70 0x7fe508109600 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009ca0 tx=0x7fe4f802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 -- 172.21.15.42:0/3969183017 shutdown_connections 2024-03-12T04:14:48.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3969183017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe508102d70 0x7fe508109600 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3969183017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5081023b0 0x7fe508102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3969183017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5081011c0 0x7fe5081015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 -- 172.21.15.42:0/3969183017 >> 172.21.15.42:0/3969183017 conn(0x7fe5080fc730 msgr2=0x7fe5080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.702+0000 7fe50cef1640 1 -- 172.21.15.42:0/3969183017 shutdown_connections 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 -- 172.21.15.42:0/3969183017 wait complete. 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 Processor -- start 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 -- start start 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5081011c0 0x7fe508197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5081023b0 0x7fe508197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe508102d70 0x7fe50819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50819ec40 con 0x7fe5081023b0 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50819edb0 con 0x7fe508102d70 2024-03-12T04:14:48.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe50cef1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50819ef70 con 0x7fe5081011c0 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.703+0000 7fe505d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5081023b0 0x7fe508197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5081023b0 0x7fe508197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52210/0 (socket says 172.21.15.42:52210) 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 -- 172.21.15.42:0/3477463552 learned_addr learned my addr 172.21.15.42:0/3477463552 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe50659c640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5081011c0 0x7fe508197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe506d9d640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe508102d70 0x7fe50819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 -- 172.21.15.42:0/3477463552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5081011c0 msgr2=0x7fe508197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5081011c0 0x7fe508197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 -- 172.21.15.42:0/3477463552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe508102d70 msgr2=0x7fe50819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe508102d70 0x7fe50819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 -- 172.21.15.42:0/3477463552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f8009900 con 0x7fe5081023b0 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe506d9d640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe508102d70 0x7fe50819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe505d9b640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5081023b0 0x7fe508197690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000d770 tx=0x7fe4f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:48.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe4ff7fe640 1 -- 172.21.15.42:0/3477463552 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0009c30 con 0x7fe5081023b0 2024-03-12T04:14:48.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe4ff7fe640 1 -- 172.21.15.42:0/3477463552 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4f000de80 con 0x7fe5081023b0 2024-03-12T04:14:48.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50819f250 con 0x7fe5081023b0 2024-03-12T04:14:48.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe4ff7fe640 1 -- 172.21.15.42:0/3477463552 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0004e00 con 0x7fe5081023b0 2024-03-12T04:14:48.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.704+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50819f710 con 0x7fe5081023b0 2024-03-12T04:14:48.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.705+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe508108560 con 0x7fe5081023b0 2024-03-12T04:14:48.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.706+0000 7fe4ff7fe640 1 -- 172.21.15.42:0/3477463552 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe4f0004f60 con 0x7fe5081023b0 2024-03-12T04:14:48.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.707+0000 7fe4ff7fe640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4f40a1a30 0x7fe4f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:48.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.707+0000 7fe50659c640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4f40a1a30 0x7fe4f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:48.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.707+0000 7fe4ff7fe640 1 -- 172.21.15.42:0/3477463552 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe4f00ccec0 con 0x7fe5081023b0 2024-03-12T04:14:48.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.708+0000 7fe50659c640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4f40a1a30 0x7fe4f40a3ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe508102210 tx=0x7fe50000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:48.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.711+0000 7fe4ff7fe640 1 -- 172.21.15.42:0/3477463552 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4f00960a0 con 0x7fe5081023b0 2024-03-12T04:14:48.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.893+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe50802ce40 con 0x7fe5081023b0 2024-03-12T04:14:48.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.895+0000 7fe4ff7fe640 1 -- 172.21.15.42:0/3477463552 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7fe4f0020090 con 0x7fe5081023b0 2024-03-12T04:14:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:48.898 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4f40a1a30 msgr2=0x7fe4f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4f40a1a30 0x7fe4f40a3ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe508102210 tx=0x7fe50000b390 comp rx=0 tx=0).stop 2024-03-12T04:14:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5081023b0 msgr2=0x7fe508197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5081023b0 0x7fe508197690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000d770 tx=0x7fe4f000dc30 comp rx=0 tx=0).stop 2024-03-12T04:14:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 shutdown_connections 2024-03-12T04:14:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe4f40a1a30 0x7fe4f40a3ef0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe508102d70 0x7fe50819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5081023b0 0x7fe508197690 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 --2- 172.21.15.42:0/3477463552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5081011c0 0x7fe508197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 >> 172.21.15.42:0/3477463552 conn(0x7fe5080fc730 msgr2=0x7fe508105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 shutdown_connections 2024-03-12T04:14:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:48.897+0000 7fe50cef1640 1 -- 172.21.15.42:0/3477463552 wait complete. 2024-03-12T04:14:48.909 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:48 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3984516594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.909 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:48 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2542833766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.910 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:48 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1454523622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-12T04:12:54.116170+0000', 'modified': '2024-03-12T04:13:01.029979+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24515, 'mds_1': 24490, 'mds_2': 24485, 'mds_3': 24475, 'mds_4': 14634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.42:6839/1870620293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6838', 'nonce': 1870620293}, {'type': 'v1', 'addr': '172.21.15.42:6839', 'nonce': 1870620293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14634': {'gid': 14634, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.42:6841/799836757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6840', 'nonce': 799836757}, {'type': 'v1', 'addr': '172.21.15.42:6841', 'nonce': 799836757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/3418533459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 3418533459}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 3418533459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6833/2928023683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6832', 'nonce': 2928023683}, {'type': 'v1', 'addr': '172.21.15.119:6833', 'nonce': 2928023683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/2794337720', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 2794337720}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 2794337720}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/2718353674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 2718353674}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 2718353674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.136:6837/23397667', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 23397667}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 23397667}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.119:6837/3839383373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 3839383373}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 3839383373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.136:6839/3211189908', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 3211189908}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 3211189908}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6839/1451526204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1451526204}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1451526204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-12T04:14:48.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-12T04:14:48.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-12T04:14:48.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:48 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3984516594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:48 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2542833766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:48 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1454523622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:48 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3984516594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:48 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2542833766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:48.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:48 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1454523622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:49.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 -- 172.21.15.42:0/1530820351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 msgr2=0x7f16e41004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/1530820351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e41004b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009900 tx=0x7f16d402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 -- 172.21.15.42:0/1530820351 shutdown_connections 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/1530820351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16e4101c40 0x7f16e41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/1530820351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e4101280 0x7f16e4101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/1530820351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e41004b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 -- 172.21.15.42:0/1530820351 >> 172.21.15.42:0/1530820351 conn(0x7f16e40fb620 msgr2=0x7f16e40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 -- 172.21.15.42:0/1530820351 shutdown_connections 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 -- 172.21.15.42:0/1530820351 wait complete. 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 Processor -- start 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.015+0000 7f16e9fb5640 1 -- start start 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e9fb5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e9fb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16e4101280 0x7f16e4197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e9fb5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e4101c40 0x7f16e419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e9fb5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e419ee30 con 0x7f16e4100090 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e9fb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e419efa0 con 0x7f16e4101280 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e9fb5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e419f160 con 0x7f16e4101c40 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e4197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52228/0 (socket says 172.21.15.42:52228) 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 -- 172.21.15.42:0/2534017382 learned_addr learned my addr 172.21.15.42:0/2534017382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16e4101280 0x7f16e4197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e3fff640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e4101c40 0x7f16e419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 -- 172.21.15.42:0/2534017382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e4101c40 msgr2=0x7f16e419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e4101c40 0x7f16e419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 -- 172.21.15.42:0/2534017382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16e4101280 msgr2=0x7f16e4197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16e4101280 0x7f16e4197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 -- 172.21.15.42:0/2534017382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4009560 con 0x7f16e4100090 2024-03-12T04:14:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e3fff640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e4101c40 0x7f16e419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e37fe640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e4197340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16d402c7d0 tx=0x7f16d4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e0ff9640 1 -- 172.21.15.42:0/2534017382 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4003800 con 0x7f16e4100090 2024-03-12T04:14:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e419f3e0 con 0x7f16e4100090 2024-03-12T04:14:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.016+0000 7f16e0ff9640 1 -- 172.21.15.42:0/2534017382 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16d402de90 con 0x7f16e4100090 2024-03-12T04:14:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.017+0000 7f16e0ff9640 1 -- 172.21.15.42:0/2534017382 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d40055f0 con 0x7f16e4100090 2024-03-12T04:14:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.017+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e419f970 con 0x7f16e4100090 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.019+0000 7f16e0ff9640 1 -- 172.21.15.42:0/2534017382 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f16d4004dc0 con 0x7f16e4100090 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.019+0000 7f16ecb81740 1 Processor -- start 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.019+0000 7f16ecb81740 1 -- start start 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31e6f80 0x55a6e31e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31ecae0 0x55a6e31eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16ecb81740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e31a2cd0 con 0x55a6e31ecae0 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16ecb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e3155f20 con 0x55a6e31e6f80 2024-03-12T04:14:49.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16ecb81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e3153450 con 0x55a6e31e7970 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31ecae0 0x55a6e31eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31ecae0 0x55a6e31eef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52240/0 (socket says 172.21.15.42:52240) 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 -- 172.21.15.42:0/3065564110 learned_addr learned my addr 172.21.15.42:0/3065564110 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e0ff9640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f16cc0a1a30 0x7f16cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e37fe640 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e3fff640 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31e6f80 0x55a6e31e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e3fff640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f16cc0a1a30 0x7f16cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 -- 172.21.15.42:0/3065564110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 msgr2=0x55a6e31ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31ec470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 -- 172.21.15.42:0/3065564110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31e6f80 msgr2=0x55a6e31e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31e6f80 0x55a6e31e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 -- 172.21.15.42:0/3065564110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e3103f20 con 0x55a6e31ecae0 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e3fff640 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31e6f80 0x55a6e31e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e0ff9640 1 -- 172.21.15.42:0/2534017382 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f16d40ef760 con 0x7f16e4100090 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31ecae0 0x55a6e31eef70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009a40 tx=0x7f16d8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc84cea8743a007 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.020+0000 7f16c9ffb640 1 -- 172.21.15.42:0/3065564110 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8009510 con 0x55a6e31ecae0 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16e3fff640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f16cc0a1a30 0x7f16cc0a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55a6e31e81d0 tx=0x7f16dc008220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16c9ffb640 1 -- 172.21.15.42:0/3065564110 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16d8013470 con 0x55a6e31ecae0 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16c9ffb640 1 -- 172.21.15.42:0/3065564110 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8012630 con 0x55a6e31ecae0 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 -- 172.21.15.42:0/3065564110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31ecae0 msgr2=0x55a6e31eef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31ecae0 0x55a6e31eef70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009a40 tx=0x7f16d8009f00 comp rx=0 tx=0).stop 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 -- 172.21.15.42:0/3065564110 shutdown_connections 2024-03-12T04:14:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31ecae0 0x55a6e31eef70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 --2- 172.21.15.42:0/3065564110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31e6f80 0x55a6e31e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 -- 172.21.15.42:0/3065564110 >> 172.21.15.42:0/3065564110 conn(0x55a6e31e24f0 msgr2=0x55a6e31e28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 -- 172.21.15.42:0/3065564110 shutdown_connections 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.021+0000 7f16ecb81740 1 -- 172.21.15.42:0/3065564110 wait complete. 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 -- start start 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 Processor -- start 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e6f80 0x55a6e31f1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31f1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e314ec30 con 0x55a6e31e6f80 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e30f3d30 con 0x55a6e31ecae0 2024-03-12T04:14:49.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e2e22310 con 0x55a6e31e7970 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16ecb81740 10 client.0 _ll_register_callbacks cb 0x55a6e30fbd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e6f80 0x55a6e31f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e6f80 0x55a6e31f1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52250/0 (socket says 172.21.15.42:52250) 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 -- 172.21.15.42:0/1052325111 learned_addr learned my addr 172.21.15.42:0/1052325111 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 -- 172.21.15.42:0/1052325111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 msgr2=0x55a6e31f1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31f1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 -- 172.21.15.42:0/1052325111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 msgr2=0x55a6e31f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 -- 172.21.15.42:0/1052325111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8009670 con 0x55a6e31e6f80 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16e37fe640 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e6f80 0x55a6e31f1100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f16d40b8b00 tx=0x7f16d40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d32c480ab3c98f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.022+0000 7f16c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16c2ffd640 1 -- 172.21.15.42:0/1052325111 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d40dc2b0 con 0x55a6e31e6f80 2024-03-12T04:14:49.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16c2ffd640 1 -- 172.21.15.42:0/1052325111 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f16d403de90 con 0x55a6e31e6f80 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6e2e4e6c0 con 0x55a6e31e6f80 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16c2ffd640 1 -- 172.21.15.42:0/1052325111 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d40bc6a0 con 0x55a6e31e6f80 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 10 client.15258 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a6e314d2b0 con 0x55a6e31e6f80 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16c2ffd640 1 -- 172.21.15.42:0/1052325111 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a6e2e4e6c0 con 0x55a6e31e6f80 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16c2ffd640 1 -- 172.21.15.42:0/1052325111 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f16d403c070 con 0x55a6e31e6f80 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 10 client.15258 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 10 client.15258 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 20 client.15258 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e31fcb30 0x55a6e3201b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 4 client.15258 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6e2e9f270 con 0x55a6e31fcb30 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.023+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e31fcb30 0x55a6e3201b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e31fcb30 0x55a6e3201b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a138dba0b30960 server_cookie=6484a9b471321b07 in_seq=0 out_seq=0 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16c2ffd640 10 client.15258 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16c2ffd640 1 -- 172.21.15.42:0/1052325111 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a6e2e9f270 con 0x55a6e31fcb30 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16c2ffd640 10 client.15258 handle_command_reply: tid=0 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 client.15258 shutdown 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e6f80 msgr2=0x55a6e31f1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e6f80 0x55a6e31f1100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f16d40b8b00 tx=0x7f16d40045e0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 shutdown_connections 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e7970 0x55a6e31f1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e31fcb30 0x55a6e3201b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 --2- 172.21.15.42:0/1052325111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e6f80 0x55a6e31f1100 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16c2ffd640 0 client.15258 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.024+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 >> 172.21.15.42:0/1052325111 conn(0x55a6e31e24f0 msgr2=0x55a6e31e17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.025+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 shutdown_connections 2024-03-12T04:14:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.025+0000 7f16ecb81740 1 -- 172.21.15.42:0/1052325111 wait complete. 2024-03-12T04:14:49.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.025+0000 7f16ecb81740 20 client.15258 trim_cache size 0 max 16384 2024-03-12T04:14:49.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.040+0000 7f16ecb81740 1 Processor -- start 2024-03-12T04:14:49.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.040+0000 7f16ecb81740 1 -- start start 2024-03-12T04:14:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.040+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 0x55a6e31f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16ecb81740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e30d4490 con 0x55a6e31e7970 2024-03-12T04:14:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16ecb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e31ff6d0 con 0x55a6e31ecae0 2024-03-12T04:14:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16ecb81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e2e4e6c0 con 0x55a6e31e6f80 2024-03-12T04:14:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52262/0 (socket says 172.21.15.42:52262) 2024-03-12T04:14:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 -- 172.21.15.42:0/2088652474 learned_addr learned my addr 172.21.15.42:0/2088652474 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 -- 172.21.15.42:0/2088652474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 msgr2=0x55a6e31f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 0x55a6e31f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 -- 172.21.15.42:0/2088652474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 msgr2=0x55a6e31f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e3fff640 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 -- 172.21.15.42:0/2088652474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e30e9a20 con 0x55a6e31e7970 2024-03-12T04:14:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e3fff640 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f8640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009fd0 tx=0x7f16d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c35577893647d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16c97fa640 1 -- 172.21.15.42:0/2088652474 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8021040 con 0x55a6e31e7970 2024-03-12T04:14:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16c97fa640 1 -- 172.21.15.42:0/2088652474 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16d8023040 con 0x55a6e31e7970 2024-03-12T04:14:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.041+0000 7f16c97fa640 1 -- 172.21.15.42:0/2088652474 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d801c450 con 0x55a6e31e7970 2024-03-12T04:14:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 -- 172.21.15.42:0/2088652474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 msgr2=0x55a6e31f8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f8640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009fd0 tx=0x7f16d8008040 comp rx=0 tx=0).stop 2024-03-12T04:14:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 -- 172.21.15.42:0/2088652474 shutdown_connections 2024-03-12T04:14:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f8640 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 --2- 172.21.15.42:0/2088652474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 0x55a6e31f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 -- 172.21.15.42:0/2088652474 >> 172.21.15.42:0/2088652474 conn(0x55a6e31e24f0 msgr2=0x55a6e31e82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 -- 172.21.15.42:0/2088652474 shutdown_connections 2024-03-12T04:14:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 -- 172.21.15.42:0/2088652474 wait complete. 2024-03-12T04:14:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 -- start start 2024-03-12T04:14:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 Processor -- start 2024-03-12T04:14:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 0x55a6e31f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.042+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e31f5910 con 0x55a6e31e7970 2024-03-12T04:14:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e31f5a80 con 0x55a6e31ecae0 2024-03-12T04:14:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e3253830 con 0x55a6e31e6f80 2024-03-12T04:14:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 10 client.0 _ll_register_callbacks cb 0x7f16d8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52272/0 (socket says 172.21.15.42:52272) 2024-03-12T04:14:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 -- 172.21.15.42:0/829234363 learned_addr learned my addr 172.21.15.42:0/829234363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e3fff640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e37fe640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 0x55a6e31f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 -- 172.21.15.42:0/829234363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 msgr2=0x55a6e31f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 0x55a6e31f82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 -- 172.21.15.42:0/829234363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 msgr2=0x55a6e31f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 -- 172.21.15.42:0/829234363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6e3103f20 con 0x55a6e31e7970 2024-03-12T04:14:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e3fff640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16e2ffd640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f4e90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f16d8008010 tx=0x7f16d800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d580930a82831ec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16c2ffd640 1 -- 172.21.15.42:0/829234363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8021040 con 0x55a6e31e7970 2024-03-12T04:14:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16c2ffd640 1 -- 172.21.15.42:0/829234363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f16d80135e0 con 0x55a6e31e7970 2024-03-12T04:14:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6e31fda60 con 0x55a6e31e7970 2024-03-12T04:14:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16c2ffd640 1 -- 172.21.15.42:0/829234363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d8013d90 con 0x55a6e31e7970 2024-03-12T04:14:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 10 client.15264 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.043+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a6e310f780 con 0x55a6e31e7970 2024-03-12T04:14:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16c2ffd640 1 -- 172.21.15.42:0/829234363 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a6e31fda60 con 0x55a6e31e7970 2024-03-12T04:14:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16c2ffd640 1 -- 172.21.15.42:0/829234363 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f16d8011070 con 0x55a6e31e7970 2024-03-12T04:14:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 10 client.15264 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 10 client.15264 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-12T04:14:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 20 client.15264 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e3258670 0x55a6e3254b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 4 client.15264 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f16d8013470 con 0x55a6e3258670 2024-03-12T04:14:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16e37fe640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e3258670 0x55a6e3254b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16c2ffd640 10 client.15264 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16e37fe640 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e3258670 0x55a6e3254b10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4ab28792f335aab server_cookie=143f8ccf5f6d38e7 in_seq=0 out_seq=0 2024-03-12T04:14:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16c2ffd640 1 -- 172.21.15.42:0/829234363 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f16d8013470 con 0x55a6e3258670 2024-03-12T04:14:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16c2ffd640 10 client.15264 handle_command_reply: tid=0 2024-03-12T04:14:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 1 client.15264 shutdown 2024-03-12T04:14:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.044+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 msgr2=0x55a6e31f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f4e90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f16d8008010 tx=0x7f16d800ef90 comp rx=0 tx=0).stop 2024-03-12T04:14:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 shutdown_connections 2024-03-12T04:14:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e31ecae0 0x55a6e31f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55a6e3258670 0x55a6e3254b10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6e31e7970 0x55a6e31f4e90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 --2- 172.21.15.42:0/829234363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6e31e6f80 0x55a6e31f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16c2ffd640 0 client.15264 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:14:49.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 >> 172.21.15.42:0/829234363 conn(0x55a6e31e24f0 msgr2=0x55a6e31ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 shutdown_connections 2024-03-12T04:14:49.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 1 -- 172.21.15.42:0/829234363 wait complete. 2024-03-12T04:14:49.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16ecb81740 20 client.15264 trim_cache size 0 max 16384 2024-03-12T04:14:49.063 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 4, 2024-03-12T04:14:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 14634, 2024-03-12T04:14:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 109.020831515, 2024-03-12T04:14:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 123.104382726 2024-03-12T04:14:49.065 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f16cc0a1a30 msgr2=0x7f16cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f16cc0a1a30 0x7f16cc0a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55a6e31e81d0 tx=0x7f16dc008220 comp rx=0 tx=0).stop 2024-03-12T04:14:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 msgr2=0x7f16e4197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e4197340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16d402c7d0 tx=0x7f16d4004360 comp rx=0 tx=0).stop 2024-03-12T04:14:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 shutdown_connections 2024-03-12T04:14:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e4101c40 0x7f16e419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f16cc0a1a30 0x7f16cc0a3ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16e4101280 0x7f16e4197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 --2- 172.21.15.42:0/2534017382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16e4100090 0x7f16e4197340 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 >> 172.21.15.42:0/2534017382 conn(0x7f16e40fb620 msgr2=0x7f16e4104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 shutdown_connections 2024-03-12T04:14:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.045+0000 7f16e9fb5640 1 -- 172.21.15.42:0/2534017382 wait complete. 2024-03-12T04:14:49.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 -- 172.21.15.42:0/134766983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e24102df0 msgr2=0x7f6e24109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/134766983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e24102df0 0x7f6e24109680 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009900 tx=0x7f6e1402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 -- 172.21.15.42:0/134766983 shutdown_connections 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/134766983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e24102df0 0x7f6e24109680 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/134766983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e24102430 0x7f6e241028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/134766983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e240ff460 0x7f6e240ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 -- 172.21.15.42:0/134766983 >> 172.21.15.42:0/134766983 conn(0x7f6e240fad70 msgr2=0x7f6e240fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 -- 172.21.15.42:0/134766983 shutdown_connections 2024-03-12T04:14:49.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 -- 172.21.15.42:0/134766983 wait complete. 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 Processor -- start 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 -- start start 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e240ff460 0x7f6e24101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e24102430 0x7f6e241003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.172+0000 7f6e2ad77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e24102df0 0x7f6e24100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e2ad77640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24101080 con 0x7f6e240ff460 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e2ad77640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e241011f0 con 0x7f6e24102430 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e2ad77640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e241a11b0 con 0x7f6e24102df0 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e240ff460 0x7f6e24101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e240ff460 0x7f6e24101da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52282/0 (socket says 172.21.15.42:52282) 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 -- 172.21.15.42:0/3984360084 learned_addr learned my addr 172.21.15.42:0/3984360084 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 -- 172.21.15.42:0/3984360084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e24102df0 msgr2=0x7f6e24100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e23fff640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e24102430 0x7f6e241003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e24102df0 0x7f6e24100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 -- 172.21.15.42:0/3984360084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e24102430 msgr2=0x7f6e241003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e24102430 0x7f6e241003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 -- 172.21.15.42:0/3984360084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14009560 con 0x7f6e240ff460 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e28b13640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e240ff460 0x7f6e24101da0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00b9f0 tx=0x7f6e1c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e21ffb640 1 -- 172.21.15.42:0/3984360084 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00cbd0 con 0x7f6e240ff460 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e241a1460 con 0x7f6e240ff460 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e21ffb640 1 -- 172.21.15.42:0/3984360084 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e1c00cd30 con 0x7f6e240ff460 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e21ffb640 1 -- 172.21.15.42:0/3984360084 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c012690 con 0x7f6e240ff460 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.173+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e241a19f0 con 0x7f6e240ff460 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.175+0000 7f6e21ffb640 1 -- 172.21.15.42:0/3984360084 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6e1c014440 con 0x7f6e240ff460 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 Processor -- start 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e21ffb640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6e0c0a1a30 0x7f6e0c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 -- start start 2024-03-12T04:14:49.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e29314640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6e0c0a1a30 0x7f6e0c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c27470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e21ffb640 1 -- 172.21.15.42:0/3984360084 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6e1c0ccd80 con 0x7f6e240ff460 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c27ae0 0x55bb04c29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04bddcd0 con 0x55bb04c27ae0 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04b90f20 con 0x55bb04c21f80 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e2da21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04b8e450 con 0x55bb04c22970 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e23fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c27470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e23fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c27470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37966/0 (socket says 172.21.15.42:37966) 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e23fff640 1 -- 172.21.15.42:0/1351476536 learned_addr learned my addr 172.21.15.42:0/1351476536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e29314640 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c27ae0 0x55bb04c29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e28b13640 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.176+0000 7f6e29314640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6e0c0a1a30 0x7f6e0c0a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e14036040 tx=0x7f6e1402cb30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e23fff640 1 -- 172.21.15.42:0/1351476536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 msgr2=0x55bb04c223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e23fff640 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c223a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e23fff640 1 -- 172.21.15.42:0/1351476536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c27ae0 msgr2=0x55bb04c29f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e23fff640 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c27ae0 0x55bb04c29f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e23fff640 1 -- 172.21.15.42:0/1351476536 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb04b3ef20 con 0x55bb04c22970 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e29314640 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c27ae0 0x55bb04c29f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e23fff640 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c27470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ba70 tx=0x7f6e1800bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d0299f477bc268 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e0affd640 1 -- 172.21.15.42:0/1351476536 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18017420 con 0x55bb04c22970 2024-03-12T04:14:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e0affd640 1 -- 172.21.15.42:0/1351476536 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e1800b520 con 0x55bb04c22970 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.177+0000 7f6e0affd640 1 -- 172.21.15.42:0/1351476536 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1800f9b0 con 0x55bb04c22970 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 -- 172.21.15.42:0/1351476536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 msgr2=0x55bb04c27470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c27470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ba70 tx=0x7f6e1800bf30 comp rx=0 tx=0).stop 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 -- 172.21.15.42:0/1351476536 shutdown_connections 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c27ae0 0x55bb04c29f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c27470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 --2- 172.21.15.42:0/1351476536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 -- 172.21.15.42:0/1351476536 >> 172.21.15.42:0/1351476536 conn(0x55bb04c1d4f0 msgr2=0x55bb04c1d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 -- 172.21.15.42:0/1351476536 shutdown_connections 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 -- 172.21.15.42:0/1351476536 wait complete. 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 -- start start 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.178+0000 7f6e2da21740 1 Processor -- start 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c2c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c33660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04b89c30 con 0x55bb04c21f80 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04b2ed30 con 0x55bb04c27ae0 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0485d310 con 0x55bb04c22970 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 10 client.0 _ll_register_callbacks cb 0x55bb04b36d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52310/0 (socket says 172.21.15.42:52310) 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 -- 172.21.15.42:0/239764150 learned_addr learned my addr 172.21.15.42:0/239764150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e23fff640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e29314640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c33660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 -- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 msgr2=0x55bb04c2c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c2c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 -- 172.21.15.42:0/239764150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 msgr2=0x55bb04c33660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c33660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 -- 172.21.15.42:0/239764150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1800b6a0 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e23fff640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c2c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e28b13640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c099cb0 tx=0x7f6e1c09a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ace401d8a6e60a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e03fff640 1 -- 172.21.15.42:0/239764150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c09de70 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e03fff640 1 -- 172.21.15.42:0/239764150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6e1c0a24c0 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.179+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb048da270 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e03fff640 1 -- 172.21.15.42:0/239764150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1c0b81b0 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bb04b882b0 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e03fff640 1 -- 172.21.15.42:0/239764150 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e1c07c020 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e03fff640 1 -- 172.21.15.42:0/239764150 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e1c095c00 con 0x55bb04c21f80 2024-03-12T04:14:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 10 client.15276 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 20 client.15276 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c37d90 0x55bb04c3ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 4 client.15276 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb048c6610 con 0x55bb04c37d90 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.180+0000 7f6e23fff640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c37d90 0x55bb04c3ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.181+0000 7f6e03fff640 10 client.15276 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.181+0000 7f6e23fff640 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c37d90 0x55bb04c3ce10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6a454a7a2d59b93 server_cookie=8d26038fc0d1d24e in_seq=0 out_seq=0 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.181+0000 7f6e03fff640 1 -- 172.21.15.42:0/239764150 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb048c6610 con 0x55bb04c37d90 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.181+0000 7f6e03fff640 10 client.15276 handle_command_reply: tid=0 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.181+0000 7f6e2da21740 1 client.15276 shutdown 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 msgr2=0x55bb04c2c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c099cb0 tx=0x7f6e1c09a420 comp rx=0 tx=0).stop 2024-03-12T04:14:49.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 shutdown_connections 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c37d90 0x55bb04c3ce10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c33660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c2c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e03fff640 0 client.15276 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 --2- 172.21.15.42:0/239764150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c0e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 >> 172.21.15.42:0/239764150 conn(0x55bb04c1d4f0 msgr2=0x55bb04c1c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 shutdown_connections 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 1 -- 172.21.15.42:0/239764150 wait complete. 2024-03-12T04:14:49.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.182+0000 7f6e2da21740 20 client.15276 trim_cache size 0 max 16384 2024-03-12T04:14:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.197+0000 7f6e2da21740 1 Processor -- start 2024-03-12T04:14:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.197+0000 7f6e2da21740 1 -- start start 2024-03-12T04:14:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c1d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e2da21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04b0f490 con 0x55bb04c21f80 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e2da21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04c2b930 con 0x55bb04c27ae0 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e2da21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04c3a970 con 0x55bb04c22970 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52312/0 (socket says 172.21.15.42:52312) 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 -- 172.21.15.42:0/2696198061 learned_addr learned my addr 172.21.15.42:0/2696198061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e29314640 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c1d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 -- 172.21.15.42:0/2696198061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 msgr2=0x55bb04c335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 -- 172.21.15.42:0/2696198061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 msgr2=0x55bb04c1d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c1d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 -- 172.21.15.42:0/2696198061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb04b24a20 con 0x55bb04c21f80 2024-03-12T04:14:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e29314640 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c1d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e28b13640 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c099ee0 tx=0x7f6e1c098230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37535507a29a26b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e0a7fc640 1 -- 172.21.15.42:0/2696198061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0a24c0 con 0x55bb04c21f80 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e0a7fc640 1 -- 172.21.15.42:0/2696198061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e1c07c030 con 0x55bb04c21f80 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.198+0000 7f6e0a7fc640 1 -- 172.21.15.42:0/2696198061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0a2030 con 0x55bb04c21f80 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 -- 172.21.15.42:0/2696198061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 msgr2=0x55bb04c2c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c099ee0 tx=0x7f6e1c098230 comp rx=0 tx=0).stop 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 -- 172.21.15.42:0/2696198061 shutdown_connections 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c27ae0 0x55bb04c1d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c22970 0x55bb04c335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 --2- 172.21.15.42:0/2696198061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c21f80 0x55bb04c2c580 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 -- 172.21.15.42:0/2696198061 >> 172.21.15.42:0/2696198061 conn(0x55bb04c1d4f0 msgr2=0x55bb04c18720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 -- 172.21.15.42:0/2696198061 shutdown_connections 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 -- 172.21.15.42:0/2696198061 wait complete. 2024-03-12T04:14:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 -- start start 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.199+0000 7f6e2da21740 1 Processor -- start 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c30340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c22970 0x55bb04c30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c27ae0 0x55bb04c8ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04c30dc0 con 0x55bb04c22970 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04c30f00 con 0x55bb04c21f80 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb04c8f010 con 0x55bb04c27ae0 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 10 client.0 _ll_register_callbacks cb 0x7f6e1800b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c22970 0x55bb04c30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c22970 0x55bb04c30880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52320/0 (socket says 172.21.15.42:52320) 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 -- 172.21.15.42:0/10854115 learned_addr learned my addr 172.21.15.42:0/10854115 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e28b13640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c30340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e29314640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c27ae0 0x55bb04c8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 -- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c27ae0 msgr2=0x55bb04c8ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c27ae0 0x55bb04c8ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 -- 172.21.15.42:0/10854115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 msgr2=0x55bb04c30340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c30340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 -- 172.21.15.42:0/10854115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb04b882b0 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e28b13640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c30340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e23fff640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c22970 0x55bb04c30880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800dd50 tx=0x7f6e1800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4739b4851e3c2bf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e03fff640 1 -- 172.21.15.42:0/10854115 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18028d00 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e03fff640 1 -- 172.21.15.42:0/10854115 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6e18013070 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb04c399c0 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e03fff640 1 -- 172.21.15.42:0/10854115 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e18028e60 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 10 client.15282 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.200+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bb04b4a780 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e03fff640 1 -- 172.21.15.42:0/10854115 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bb04c399c0 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e03fff640 1 -- 172.21.15.42:0/10854115 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e1801f040 con 0x55bb04c22970 2024-03-12T04:14:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e2da21740 10 client.15282 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e2da21740 10 client.15282 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e2da21740 20 client.15282 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e2da21740 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c94250 0x55bb04c906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e2da21740 4 client.15282 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bb048896c0 con 0x55bb04c94250 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e28b13640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c94250 0x55bb04c906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e03fff640 10 client.15282 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.201+0000 7f6e28b13640 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c94250 0x55bb04c906d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85a62dd02b7d02c2 server_cookie=2031ea74d968077f in_seq=0 out_seq=0 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e03fff640 1 -- 172.21.15.42:0/10854115 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55bb048896c0 con 0x55bb04c94250 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e03fff640 10 client.15282 handle_command_reply: tid=0 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 client.15282 shutdown 2024-03-12T04:14:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c22970 msgr2=0x55bb04c30880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c22970 0x55bb04c30880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800dd50 tx=0x7f6e1800de80 comp rx=0 tx=0).stop 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 shutdown_connections 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55bb04c94250 0x55bb04c906d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb04c27ae0 0x55bb04c8ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb04c22970 0x55bb04c30880 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e03fff640 0 client.15282 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 --2- 172.21.15.42:0/10854115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb04c21f80 0x55bb04c30340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 >> 172.21.15.42:0/10854115 conn(0x55bb04c1d4f0 msgr2=0x55bb04c1d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 shutdown_connections 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 1 -- 172.21.15.42:0/10854115 wait complete. 2024-03-12T04:14:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2da21740 20 client.15282 trim_cache size 0 max 16384 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 3, 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24475, 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 110.15334276999999, 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 134.657131857 2024-03-12T04:14:49.212 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6e0c0a1a30 msgr2=0x7f6e0c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6e0c0a1a30 0x7f6e0c0a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e14036040 tx=0x7f6e1402cb30 comp rx=0 tx=0).stop 2024-03-12T04:14:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e240ff460 msgr2=0x7f6e24101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e240ff460 0x7f6e24101da0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00b9f0 tx=0x7f6e1c00beb0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 shutdown_connections 2024-03-12T04:14:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e24102df0 0x7f6e24100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f6e0c0a1a30 0x7f6e0c0a3ef0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e24102430 0x7f6e241003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.202+0000 7f6e2ad77640 1 --2- 172.21.15.42:0/3984360084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e240ff460 0x7f6e24101da0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.203+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 >> 172.21.15.42:0/3984360084 conn(0x7f6e240fad70 msgr2=0x7f6e241057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.203+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 shutdown_connections 2024-03-12T04:14:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.203+0000 7f6e2ad77640 1 -- 172.21.15.42:0/3984360084 wait complete. 2024-03-12T04:14:49.216 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-12T04:14:49.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 -- 172.21.15.42:0/2193329216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 msgr2=0x7f2d70106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 --2- 172.21.15.42:0/2193329216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70106940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d6402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 -- 172.21.15.42:0/2193329216 shutdown_connections 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 --2- 172.21.15.42:0/2193329216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d70106e80 0x7f2d701092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 --2- 172.21.15.42:0/2193329216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70106940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 --2- 172.21.15.42:0/2193329216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d70101c20 0x7f2d70104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 -- 172.21.15.42:0/2193329216 >> 172.21.15.42:0/2193329216 conn(0x7f2d700fb630 msgr2=0x7f2d700fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 -- 172.21.15.42:0/2193329216 shutdown_connections 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 -- 172.21.15.42:0/2193329216 wait complete. 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 Processor -- start 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.317+0000 7f2d7658c640 1 -- start start 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d7658c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d70101c20 0x7f2d701972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d7658c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d7658c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d70106e80 0x7f2d7019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d7658c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7019ede0 con 0x7f2d70104550 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d7658c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7019ef50 con 0x7f2d70101c20 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d7658c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7019f110 con 0x7f2d70106e80 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52332/0 (socket says 172.21.15.42:52332) 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3487086275 learned_addr learned my addr 172.21.15.42:0/3487086275 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d70101c20 0x7f2d701972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3487086275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d70106e80 msgr2=0x7f2d7019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d70106e80 0x7f2d7019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3487086275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d70101c20 msgr2=0x7f2d701972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d70101c20 0x7f2d701972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3487086275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009560 con 0x7f2d70104550 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70197830 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2d6402c7d0 tx=0x7f2d64004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6d7fa640 1 -- 172.21.15.42:0/3487086275 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64003800 con 0x7f2d70104550 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6d7fa640 1 -- 172.21.15.42:0/3487086275 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d6402de90 con 0x7f2d70104550 2024-03-12T04:14:49.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d6d7fa640 1 -- 172.21.15.42:0/3487086275 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640055f0 con 0x7f2d70104550 2024-03-12T04:14:49.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.318+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7019f390 con 0x7f2d70104550 2024-03-12T04:14:49.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.319+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7019f920 con 0x7f2d70104550 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.320+0000 7f2d6d7fa640 1 -- 172.21.15.42:0/3487086275 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2d64004dc0 con 0x7f2d70104550 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 Processor -- start 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 -- start start 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c5970 0x5570939ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557093980cd0 con 0x5570939caae0 2024-03-12T04:14:49.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557093933f20 con 0x5570939c4f80 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d79163740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557093931450 con 0x5570939c5970 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d6d7fa640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2d580a1a30 0x7f2d580a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d74b29640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2d580a1a30 0x7f2d580a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52334/0 (socket says 172.21.15.42:52334) 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.321+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/2373913607 learned_addr learned my addr 172.21.15.42:0/2373913607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d74b29640 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6d7fa640 1 -- 172.21.15.42:0/3487086275 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2d640ef860 con 0x7f2d70104550 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c5970 0x5570939ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/2373913607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c5970 msgr2=0x5570939ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c5970 0x5570939ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/2373913607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 msgr2=0x5570939c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/2373913607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570938e1f20 con 0x5570939caae0 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d74b29640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2d580a1a30 0x7f2d580a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d680138e0 tx=0x7f2d68001f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939ccf70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d640bc730 tx=0x7f2d6408faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aa065252646b48 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d567fc640 1 -- 172.21.15.42:0/2373913607 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640bcea0 con 0x5570939caae0 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d567fc640 1 -- 172.21.15.42:0/2373913607 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d64087f90 con 0x5570939caae0 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.322+0000 7f2d567fc640 1 -- 172.21.15.42:0/2373913607 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640c4f40 con 0x5570939caae0 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 -- 172.21.15.42:0/2373913607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 msgr2=0x5570939ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939ccf70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d640bc730 tx=0x7f2d6408faa0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 -- 172.21.15.42:0/2373913607 shutdown_connections 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939ccf70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c5970 0x5570939ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 --2- 172.21.15.42:0/2373913607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 -- 172.21.15.42:0/2373913607 >> 172.21.15.42:0/2373913607 conn(0x5570939c04f0 msgr2=0x5570939c08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 -- 172.21.15.42:0/2373913607 shutdown_connections 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 -- 172.21.15.42:0/2373913607 wait complete. 2024-03-12T04:14:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 -- start start 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.323+0000 7f2d79163740 1 Processor -- start 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c5970 0x5570939cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709392cc30 con 0x5570939caae0 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570938d1d30 con 0x5570939c5970 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557093600310 con 0x5570939c4f80 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 10 client.0 _ll_register_callbacks cb 0x5570938d9d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939d6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52342/0 (socket says 172.21.15.42:52342) 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 -- 172.21.15.42:0/2892180626 learned_addr learned my addr 172.21.15.42:0/2892180626 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c5970 0x5570939cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 -- 172.21.15.42:0/2892180626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 msgr2=0x5570939cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 -- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c5970 msgr2=0x5570939cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c5970 0x5570939cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 -- 172.21.15.42:0/2892180626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7019f920 con 0x5570939caae0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d74b29640 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939d6380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d68015aa0 tx=0x7f2d680013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baddbff77d5b466 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/2892180626 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680079e0 con 0x5570939caae0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/2892180626 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2d68001960 con 0x5570939caae0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55709362c6c0 con 0x5570939caae0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/2892180626 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d68001c20 con 0x5570939caae0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.324+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55709392b2b0 con 0x5570939caae0 2024-03-12T04:14:49.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/2892180626 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55709362c6c0 con 0x5570939caae0 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/2892180626 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d680183f0 con 0x5570939caae0 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d79163740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d79163740 10 client.15294 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d79163740 20 client.15294 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d79163740 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5570939db810 0x5570939daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d79163740 4 client.15294 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55709367d270 con 0x5570939db810 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.325+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5570939db810 0x5570939daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d4f7fe640 10 client.15294 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5570939db810 0x5570939daaa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f0e0f556be9f331 server_cookie=d1bf1e296db2212 in_seq=0 out_seq=0 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/2892180626 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55709367d270 con 0x5570939db810 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d4f7fe640 10 client.15294 handle_command_reply: tid=0 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 client.15294 shutdown 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 msgr2=0x5570939d6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939d6380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d68015aa0 tx=0x7f2d680013c0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 shutdown_connections 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5570939db810 0x5570939daaa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939caae0 0x5570939d6380 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c5970 0x5570939cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d4f7fe640 0 client.15294 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 --2- 172.21.15.42:0/2892180626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.326+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 >> 172.21.15.42:0/2892180626 conn(0x5570939c04f0 msgr2=0x5570939bf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.327+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 shutdown_connections 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.327+0000 7f2d79163740 1 -- 172.21.15.42:0/2892180626 wait complete. 2024-03-12T04:14:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.327+0000 7f2d79163740 20 client.15294 trim_cache size 0 max 16384 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 Processor -- start 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 -- start start 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939caae0 0x5570939cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570938b2490 con 0x5570939c5970 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570939d7000 con 0x5570939caae0 2024-03-12T04:14:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d79163740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570939c0f40 con 0x5570939c4f80 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52344/0 (socket says 172.21.15.42:52344) 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.342+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3195771196 learned_addr learned my addr 172.21.15.42:0/3195771196 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3195771196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 msgr2=0x5570939cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3195771196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939caae0 msgr2=0x5570939cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939caae0 0x5570939cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/3195771196 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570938c7a20 con 0x5570939c5970 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d62a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d640bc4a0 tx=0x7f2d640efaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340e670f6ae771ad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d55ffb640 1 -- 172.21.15.42:0/3195771196 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640bcea0 con 0x5570939c5970 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d55ffb640 1 -- 172.21.15.42:0/3195771196 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d640bc7a0 con 0x5570939c5970 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.343+0000 7f2d55ffb640 1 -- 172.21.15.42:0/3195771196 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640b8e10 con 0x5570939c5970 2024-03-12T04:14:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- 172.21.15.42:0/3195771196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 msgr2=0x5570939d62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d62a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d640bc4a0 tx=0x7f2d640efaa0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- 172.21.15.42:0/3195771196 shutdown_connections 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939caae0 0x5570939cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d62a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 --2- 172.21.15.42:0/3195771196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939c4f80 0x5570939cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- 172.21.15.42:0/3195771196 >> 172.21.15.42:0/3195771196 conn(0x5570939c04f0 msgr2=0x5570939c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- 172.21.15.42:0/3195771196 shutdown_connections 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- 172.21.15.42:0/3195771196 wait complete. 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- start start 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 Processor -- start 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939d6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939caae0 0x5570939d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557093a31410 con 0x5570939c5970 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557093a31550 con 0x5570939c4f80 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557093a31690 con 0x5570939caae0 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d79163740 10 client.0 _ll_register_callbacks cb 0x7f2d68002120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.344+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52346/0 (socket says 172.21.15.42:52346) 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/4135944902 learned_addr learned my addr 172.21.15.42:0/4135944902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d74b29640 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939caae0 0x5570939d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939d6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/4135944902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939caae0 msgr2=0x5570939d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939caae0 0x5570939d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 msgr2=0x5570939d6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939d6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 -- 172.21.15.42:0/4135944902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570938e1f20 con 0x5570939c5970 2024-03-12T04:14:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d6f7fe640 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d2dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2d640c4a10 tx=0x7f2d640efb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480329f06f61817c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/4135944902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640bcea0 con 0x5570939c5970 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/4135944902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2d6403de90 con 0x5570939c5970 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570939dc1e0 con 0x5570939c5970 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/4135944902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d640c4cc0 con 0x5570939c5970 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d79163740 10 client.15300 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5570938ed780 con 0x5570939c5970 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.345+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/4135944902 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5570939dc1e0 con 0x5570939c5970 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/4135944902 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d6403c070 con 0x5570939c5970 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d79163740 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d79163740 10 client.15300 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d79163740 20 client.15300 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d79163740 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x557093a364b0 0x557093a329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d79163740 4 client.15300 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f2d64087f90 con 0x557093a364b0 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x557093a364b0 0x557093a329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d4f7fe640 10 client.15300 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d6ffff640 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x557093a364b0 0x557093a329b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9be118593e8941cf server_cookie=766152798557c54b in_seq=0 out_seq=0 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d4f7fe640 1 -- 172.21.15.42:0/4135944902 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f2d64087f90 con 0x557093a364b0 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d4f7fe640 10 client.15300 handle_command_reply: tid=0 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.346+0000 7f2d79163740 1 client.15300 shutdown 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 msgr2=0x5570939d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d2dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2d640c4a10 tx=0x7f2d640efb10 comp rx=0 tx=0).stop 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 shutdown_connections 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x557093a364b0 0x557093a329b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570939caae0 0x5570939d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570939c5970 0x5570939d2dd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d4f7fe640 0 client.15300 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 --2- 172.21.15.42:0/4135944902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5570939c4f80 0x5570939d6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 >> 172.21.15.42:0/4135944902 conn(0x5570939c04f0 msgr2=0x5570939c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 shutdown_connections 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 1 -- 172.21.15.42:0/4135944902 wait complete. 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d79163740 20 client.15300 trim_cache size 0 max 16384 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-03-12T04:14:49.356 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24485, 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 111.308984505, 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 131.89932750200001 2024-03-12T04:14:49.357 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2d580a1a30 msgr2=0x7f2d580a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2d580a1a30 0x7f2d580a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d680138e0 tx=0x7f2d68001f10 comp rx=0 tx=0).stop 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 msgr2=0x7f2d70197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70197830 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2d6402c7d0 tx=0x7f2d64004360 comp rx=0 tx=0).stop 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 shutdown_connections 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d70106e80 0x7f2d7019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104550 0x7f2d70197830 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2d580a1a30 0x7f2d580a3ef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 --2- 172.21.15.42:0/3487086275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d70101c20 0x7f2d701972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 >> 172.21.15.42:0/3487086275 conn(0x7f2d700fb630 msgr2=0x7f2d70102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 shutdown_connections 2024-03-12T04:14:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.347+0000 7f2d7658c640 1 -- 172.21.15.42:0/3487086275 wait complete. 2024-03-12T04:14:49.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-12T04:14:49.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 -- 172.21.15.42:0/1338278948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42881000a0 msgr2=0x7f42881004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 --2- 172.21.15.42:0/1338278948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42881000a0 0x7f42881004c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4278009900 tx=0x7f427802bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 -- 172.21.15.42:0/1338278948 shutdown_connections 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 --2- 172.21.15.42:0/1338278948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4288101c50 0x7f42881084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 --2- 172.21.15.42:0/1338278948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4288101290 0x7f4288101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 --2- 172.21.15.42:0/1338278948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42881000a0 0x7f42881004c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 -- 172.21.15.42:0/1338278948 >> 172.21.15.42:0/1338278948 conn(0x7f42880fb630 msgr2=0x7f42880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 -- 172.21.15.42:0/1338278948 shutdown_connections 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.461+0000 7f428dc40640 1 -- 172.21.15.42:0/1338278948 wait complete. 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 Processor -- start 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 -- start start 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42881000a0 0x7f428806a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4288101290 0x7f428819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4288101c50 0x7f428819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428819ed00 con 0x7f4288101c50 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428819ee70 con 0x7f4288101290 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f428dc40640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428819f030 con 0x7f42881000a0 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f4287fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4288101c50 0x7f428819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f4287fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4288101c50 0x7f428819e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52354/0 (socket says 172.21.15.42:52354) 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f4287fff640 1 -- 172.21.15.42:0/2387777645 learned_addr learned my addr 172.21.15.42:0/2387777645 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f4286ffd640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4288101290 0x7f428819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.462+0000 7f42877fe640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42881000a0 0x7f428806a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4287fff640 1 -- 172.21.15.42:0/2387777645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42881000a0 msgr2=0x7f428806a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4287fff640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42881000a0 0x7f428806a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4287fff640 1 -- 172.21.15.42:0/2387777645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4288101290 msgr2=0x7f428819bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4287fff640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4288101290 0x7f428819bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4287fff640 1 -- 172.21.15.42:0/2387777645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278009560 con 0x7f4288101c50 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4287fff640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4288101c50 0x7f428819e520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f428000cad0 tx=0x7f428000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4284ff9640 1 -- 172.21.15.42:0/2387777645 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000ae20 con 0x7f4288101c50 2024-03-12T04:14:49.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4284ff9640 1 -- 172.21.15.42:0/2387777645 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4280005020 con 0x7f4288101c50 2024-03-12T04:14:49.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428819f2b0 con 0x7f4288101c50 2024-03-12T04:14:49.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f4284ff9640 1 -- 172.21.15.42:0/2387777645 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000f690 con 0x7f4288101c50 2024-03-12T04:14:49.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.463+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428819f770 con 0x7f4288101c50 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4284ff9640 1 -- 172.21.15.42:0/2387777645 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4280005190 con 0x7f4288101c50 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 Processor -- start 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 -- start start 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 0x55bbab2413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab246470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab248f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab1fccd0 con 0x55bbab241970 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab1aff20 con 0x55bbab240f80 2024-03-12T04:14:49.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4290815740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab1ad450 con 0x55bbab246ae0 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab246470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab246470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52370/0 (socket says 172.21.15.42:52370) 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 -- 172.21.15.42:0/3601567758 learned_addr learned my addr 172.21.15.42:0/3601567758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4284ff9640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f42700a1a30 0x7f42700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4286ffd640 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 0x55bbab2413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4286ffd640 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab248f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4287fff640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f42700a1a30 0x7f42700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 -- 172.21.15.42:0/3601567758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 msgr2=0x55bbab248f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab248f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 -- 172.21.15.42:0/3601567758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 msgr2=0x55bbab2413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 0x55bbab2413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f4284ff9640 1 -- 172.21.15.42:0/2387777645 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f42800cd450 con 0x7f4288101c50 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.466+0000 7f42877fe640 1 -- 172.21.15.42:0/3601567758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbab15df20 con 0x55bbab241970 2024-03-12T04:14:49.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f42877fe640 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab246470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f42881010f0 tx=0x7f427802c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfc678c53f541b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f4287fff640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f42700a1a30 0x7f42700a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4280096850 tx=0x7f4280092040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f426dffb640 1 -- 172.21.15.42:0/3601567758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427802cbe0 con 0x55bbab241970 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f426dffb640 1 -- 172.21.15.42:0/3601567758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f427802de90 con 0x55bbab241970 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f426dffb640 1 -- 172.21.15.42:0/3601567758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780039b0 con 0x55bbab241970 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f4290815740 1 -- 172.21.15.42:0/3601567758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 msgr2=0x55bbab246470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f4290815740 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab246470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f42881010f0 tx=0x7f427802c8f0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f4290815740 1 -- 172.21.15.42:0/3601567758 shutdown_connections 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f4290815740 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab248f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f4290815740 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab246470 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.467+0000 7f4290815740 1 --2- 172.21.15.42:0/3601567758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 0x55bbab2413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 -- 172.21.15.42:0/3601567758 >> 172.21.15.42:0/3601567758 conn(0x55bbab23c4f0 msgr2=0x55bbab23c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 -- 172.21.15.42:0/3601567758 shutdown_connections 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 -- 172.21.15.42:0/3601567758 wait complete. 2024-03-12T04:14:49.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 -- start start 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 Processor -- start 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab252600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab1a8c30 con 0x55bbab240f80 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab14dd30 con 0x55bbab241970 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaae7c310 con 0x55bbab246ae0 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4290815740 10 client.0 _ll_register_callbacks cb 0x55bbab155d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52386/0 (socket says 172.21.15.42:52386) 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.468+0000 7f4286ffd640 1 -- 172.21.15.42:0/4127378555 learned_addr learned my addr 172.21.15.42:0/4127378555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f42877fe640 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4286ffd640 1 -- 172.21.15.42:0/4127378555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 msgr2=0x55bbab252600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4286ffd640 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab252600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4286ffd640 1 -- 172.21.15.42:0/4127378555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 msgr2=0x55bbab24b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4286ffd640 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4286ffd640 1 -- 172.21.15.42:0/4127378555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4288108de0 con 0x55bbab240f80 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f42877fe640 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4286ffd640 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24b080 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f427c00bc20 tx=0x7f427c012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42328e8901b92ac server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4266ffd640 1 -- 172.21.15.42:0/4127378555 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c00fba0 con 0x55bbab240f80 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4266ffd640 1 -- 172.21.15.42:0/4127378555 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f427c00fd00 con 0x55bbab240f80 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbaaea86c0 con 0x55bbab240f80 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4266ffd640 1 -- 172.21.15.42:0/4127378555 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f427c00fec0 con 0x55bbab240f80 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bbab1a72b0 con 0x55bbab240f80 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4266ffd640 1 -- 172.21.15.42:0/4127378555 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f427c02d080 con 0x55bbab240f80 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4266ffd640 1 -- 172.21.15.42:0/4127378555 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f427c027030 con 0x55bbab240f80 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 10 client.15312 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 20 client.15312 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab257ae0 0x55bbab256d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 4 client.15312 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.469+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbaaef9270 con 0x55bbab257ae0 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.470+0000 7f42877fe640 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab257ae0 0x55bbab256d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.470+0000 7f42877fe640 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab257ae0 0x55bbab256d20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fd88aec43b0f93f server_cookie=89e1655e320c15a in_seq=0 out_seq=0 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.470+0000 7f4266ffd640 10 client.15312 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4266ffd640 1 -- 172.21.15.42:0/4127378555 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bbaaef9270 con 0x55bbab257ae0 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4266ffd640 10 client.15312 handle_command_reply: tid=0 2024-03-12T04:14:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 client.15312 shutdown 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 msgr2=0x55bbab24b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24b080 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f427c00bc20 tx=0x7f427c012ea0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 shutdown_connections 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab257ae0 0x55bbab256d20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab252600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4266ffd640 0 client.15312 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 --2- 172.21.15.42:0/4127378555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24b080 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 >> 172.21.15.42:0/4127378555 conn(0x55bbab23c4f0 msgr2=0x55bbab23b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 shutdown_connections 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 1 -- 172.21.15.42:0/4127378555 wait complete. 2024-03-12T04:14:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.471+0000 7f4290815740 20 client.15312 trim_cache size 0 max 16384 2024-03-12T04:14:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 Processor -- start 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 -- start start 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 0x55bbab23c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab252590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab242e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab12e490 con 0x55bbab241970 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaaef92a0 con 0x55bbab240f80 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f4290815740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab161f40 con 0x55bbab246ae0 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab252590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab252590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52400/0 (socket says 172.21.15.42:52400) 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 -- 172.21.15.42:0/3971179128 learned_addr learned my addr 172.21.15.42:0/3971179128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 -- 172.21.15.42:0/3971179128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 msgr2=0x55bbab242e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 --2- 172.21.15.42:0/3971179128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab242e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 -- 172.21.15.42:0/3971179128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 msgr2=0x55bbab23c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 --2- 172.21.15.42:0/3971179128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 0x55bbab23c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.487+0000 7f42877fe640 1 -- 172.21.15.42:0/3971179128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbab143a20 con 0x55bbab241970 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f42877fe640 1 --2- 172.21.15.42:0/3971179128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab252590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f427802c170 tx=0x7f4278040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491447c50097df81 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f426d7fa640 1 -- 172.21.15.42:0/3971179128 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427802cbe0 con 0x55bbab241970 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f426d7fa640 1 -- 172.21.15.42:0/3971179128 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f42780044f0 con 0x55bbab241970 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f426d7fa640 1 -- 172.21.15.42:0/3971179128 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427804f5d0 con 0x55bbab241970 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 -- 172.21.15.42:0/3971179128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 msgr2=0x55bbab252590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 --2- 172.21.15.42:0/3971179128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab252590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f427802c170 tx=0x7f4278040040 comp rx=0 tx=0).stop 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 -- 172.21.15.42:0/3971179128 shutdown_connections 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 --2- 172.21.15.42:0/3971179128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab242e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 --2- 172.21.15.42:0/3971179128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab241970 0x55bbab252590 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 --2- 172.21.15.42:0/3971179128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab240f80 0x55bbab23c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 -- 172.21.15.42:0/3971179128 >> 172.21.15.42:0/3971179128 conn(0x55bbab23c4f0 msgr2=0x55bbab2377e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 -- 172.21.15.42:0/3971179128 shutdown_connections 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.488+0000 7f4290815740 1 -- 172.21.15.42:0/3971179128 wait complete. 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 -- start start 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 Processor -- start 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab2ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab252650 con 0x55bbab240f80 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab2527c0 con 0x55bbab241970 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab252980 con 0x55bbab246ae0 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4290815740 10 client.0 _ll_register_callbacks cb 0x7f4278009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52402/0 (socket says 172.21.15.42:52402) 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 -- 172.21.15.42:0/418700378 learned_addr learned my addr 172.21.15.42:0/418700378 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f42877fe640 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 -- 172.21.15.42:0/418700378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 msgr2=0x55bbab2ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab2ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 -- 172.21.15.42:0/418700378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 msgr2=0x55bbab24f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.489+0000 7f4286ffd640 1 -- 172.21.15.42:0/418700378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbab15df20 con 0x55bbab240f80 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4286ffd640 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24f0f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f427c00b6b0 tx=0x7f427c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74db43e1fb48cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4266ffd640 1 -- 172.21.15.42:0/418700378 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c012630 con 0x55bbab240f80 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbab2577f0 con 0x55bbab240f80 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4266ffd640 1 -- 172.21.15.42:0/418700378 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f427c029ca0 con 0x55bbab240f80 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4266ffd640 1 -- 172.21.15.42:0/418700378 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f427c029400 con 0x55bbab240f80 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 10 client.15318 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bbab169780 con 0x55bbab240f80 2024-03-12T04:14:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4266ffd640 1 -- 172.21.15.42:0/418700378 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bbab2577f0 con 0x55bbab240f80 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4266ffd640 1 -- 172.21.15.42:0/418700378 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f427c027040 con 0x55bbab240f80 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 10 client.15318 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 10 client.15318 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 20 client.15318 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab2b2de0 0x55bbab2af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 4 client.15318 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.490+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f427802de90 con 0x55bbab2b2de0 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f42877fe640 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab2b2de0 0x55bbab2af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4266ffd640 10 client.15318 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f42877fe640 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab2b2de0 0x55bbab2af200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8e39f2cec42d4d9 server_cookie=335b7bb928c22674 in_seq=0 out_seq=0 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4266ffd640 1 -- 172.21.15.42:0/418700378 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f427802de90 con 0x55bbab2b2de0 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4266ffd640 10 client.15318 handle_command_reply: tid=0 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 client.15318 shutdown 2024-03-12T04:14:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 msgr2=0x55bbab24f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24f0f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f427c00b6b0 tx=0x7f427c00bf30 comp rx=0 tx=0).stop 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 shutdown_connections 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55bbab2b2de0 0x55bbab2af200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab246ae0 0x55bbab2ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4266ffd640 0 client.15318 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbab241970 0x55bbab24f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 --2- 172.21.15.42:0/418700378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbab240f80 0x55bbab24f0f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 >> 172.21.15.42:0/418700378 conn(0x55bbab23c4f0 msgr2=0x55bbab23c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 shutdown_connections 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.491+0000 7f4290815740 1 -- 172.21.15.42:0/418700378 wait complete. 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f4290815740 20 client.15318 trim_cache size 0 max 16384 2024-03-12T04:14:49.500 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24490, 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 112.460586312, 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 129.22890939199999 2024-03-12T04:14:49.501 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f42700a1a30 msgr2=0x7f42700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f42700a1a30 0x7f42700a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4280096850 tx=0x7f4280092040 comp rx=0 tx=0).stop 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4288101c50 msgr2=0x7f428819e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4288101c50 0x7f428819e520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f428000cad0 tx=0x7f428000cf90 comp rx=0 tx=0).stop 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 shutdown_connections 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4288101c50 0x7f428819e520 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f42700a1a30 0x7f42700a3ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4288101290 0x7f428819bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 --2- 172.21.15.42:0/2387777645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42881000a0 0x7f428806a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 >> 172.21.15.42:0/2387777645 conn(0x7f42880fb630 msgr2=0x7f4288104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 shutdown_connections 2024-03-12T04:14:49.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.492+0000 7f428dc40640 1 -- 172.21.15.42:0/2387777645 wait complete. 2024-03-12T04:14:49.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 -- 172.21.15.42:0/2992691090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102d50 msgr2=0x7f712c1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 --2- 172.21.15.42:0/2992691090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102d50 0x7f712c1095e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7124009ca0 tx=0x7f712402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 -- 172.21.15.42:0/2992691090 shutdown_connections 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 --2- 172.21.15.42:0/2992691090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102d50 0x7f712c1095e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 --2- 172.21.15.42:0/2992691090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712c102390 0x7f712c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 --2- 172.21.15.42:0/2992691090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f712c1011a0 0x7f712c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 -- 172.21.15.42:0/2992691090 >> 172.21.15.42:0/2992691090 conn(0x7f712c0fc710 msgr2=0x7f712c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 -- 172.21.15.42:0/2992691090 shutdown_connections 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.607+0000 7f71334c9640 1 -- 172.21.15.42:0/2992691090 wait complete. 2024-03-12T04:14:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 Processor -- start 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 -- start start 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712c1011a0 0x7f712c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102390 0x7f712c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f712c102d50 0x7f712c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19ec30 con 0x7f712c102390 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19eda0 con 0x7f712c1011a0 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71334c9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19ef60 con 0x7f712c102d50 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102390 0x7f712c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102390 0x7f712c197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52422/0 (socket says 172.21.15.42:52422) 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 -- 172.21.15.42:0/2826613616 learned_addr learned my addr 172.21.15.42:0/2826613616 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7131265640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712c1011a0 0x7f712c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7131a66640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f712c102d50 0x7f712c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 -- 172.21.15.42:0/2826613616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f712c102d50 msgr2=0x7f712c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f712c102d50 0x7f712c19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 -- 172.21.15.42:0/2826613616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712c1011a0 msgr2=0x7f712c197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712c1011a0 0x7f712c197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 -- 172.21.15.42:0/2826613616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124009900 con 0x7f712c102390 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f7130a64640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102390 0x7f712c197680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e9f0 tx=0x7f711c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71227fc640 1 -- 172.21.15.42:0/2826613616 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c009700 con 0x7f712c102390 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.608+0000 7f71227fc640 1 -- 172.21.15.42:0/2826613616 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f711c00cd80 con 0x7f712c102390 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.609+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c19f240 con 0x7f712c102390 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.609+0000 7f71227fc640 1 -- 172.21.15.42:0/2826613616 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c018840 con 0x7f712c102390 2024-03-12T04:14:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.609+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c19f7a0 con 0x7f712c102390 2024-03-12T04:14:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.610+0000 7f71227fc640 1 -- 172.21.15.42:0/2826613616 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f711c0189a0 con 0x7f712c102390 2024-03-12T04:14:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 Processor -- start 2024-03-12T04:14:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 -- start start 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f71227fc640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f71180a1ab0 0x7f71180a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c246b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2470470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 0x5651c2472f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2426cd0 con 0x5651c246af80 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c23d9f20 con 0x5651c2470ae0 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7136179740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c23d7450 con 0x5651c246b970 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7131a66640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f71180a1ab0 0x7f71180a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7131265640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c246b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7131265640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c246b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52428/0 (socket says 172.21.15.42:52428) 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7131265640 1 -- 172.21.15.42:0/1061861344 learned_addr learned my addr 172.21.15.42:0/1061861344 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f7131a66640 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 0x5651c2472f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.611+0000 7f71227fc640 1 -- 172.21.15.42:0/2826613616 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f711c014070 con 0x7f712c102390 2024-03-12T04:14:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7130a64640 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2470470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7131265640 1 -- 172.21.15.42:0/1061861344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 msgr2=0x5651c2470470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7131265640 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2470470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7131265640 1 -- 172.21.15.42:0/1061861344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 msgr2=0x5651c2472f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7131265640 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 0x5651c2472f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7131265640 1 -- 172.21.15.42:0/1061861344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651c2387f20 con 0x5651c246af80 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7131a66640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f71180a1ab0 0x7f71180a3f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7124006010 tx=0x7f7124032340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f7131265640 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c246b3a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f712800bc80 tx=0x7f712800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a393adffdac535c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f71177fe640 1 -- 172.21.15.42:0/1061861344 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712801a460 con 0x5651c246af80 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f71177fe640 1 -- 172.21.15.42:0/1061861344 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f712801aa70 con 0x5651c246af80 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.612+0000 7f71177fe640 1 -- 172.21.15.42:0/1061861344 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128011970 con 0x5651c246af80 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 -- 172.21.15.42:0/1061861344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 msgr2=0x5651c246b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c246b3a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f712800bc80 tx=0x7f712800f730 comp rx=0 tx=0).stop 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 -- 172.21.15.42:0/1061861344 shutdown_connections 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 0x5651c2472f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2470470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 --2- 172.21.15.42:0/1061861344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c246b3a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 -- 172.21.15.42:0/1061861344 >> 172.21.15.42:0/1061861344 conn(0x5651c24664f0 msgr2=0x5651c24668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 -- 172.21.15.42:0/1061861344 shutdown_connections 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 -- 172.21.15.42:0/1061861344 wait complete. 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 -- start start 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.613+0000 7f7136179740 1 Processor -- start 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2475170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 0x5651c247c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c23d2c30 con 0x5651c246b970 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2377d30 con 0x5651c2470ae0 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c20a6310 con 0x5651c246af80 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 10 client.0 _ll_register_callbacks cb 0x5651c237fd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24756b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52444/0 (socket says 172.21.15.42:52444) 2024-03-12T04:14:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 -- 172.21.15.42:0/3352328641 learned_addr learned my addr 172.21.15.42:0/3352328641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7131265640 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2475170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 -- 172.21.15.42:0/3352328641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 msgr2=0x5651c2475170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2475170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 -- 172.21.15.42:0/3352328641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 msgr2=0x5651c247c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 0x5651c247c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 -- 172.21.15.42:0/3352328641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712800b8e0 con 0x5651c246b970 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7130a64640 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24756b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f711c09a6e0 tx=0x7f711c09af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6676e3b8f4e72fd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7114ff9640 1 -- 172.21.15.42:0/3352328641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c010b50 con 0x5651c246b970 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7114ff9640 1 -- 172.21.15.42:0/3352328641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f711c0ba180 con 0x5651c246b970 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651c20d26c0 con 0x5651c246b970 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7114ff9640 1 -- 172.21.15.42:0/3352328641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f711c0ba340 con 0x5651c246b970 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 10 client.15330 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.614+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5651c23d12b0 con 0x5651c246b970 2024-03-12T04:14:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7114ff9640 1 -- 172.21.15.42:0/3352328641 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f711c0bf020 con 0x5651c246b970 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7114ff9640 1 -- 172.21.15.42:0/3352328641 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f711c0964a0 con 0x5651c246b970 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7136179740 10 client.15330 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7136179740 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7136179740 20 client.15330 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7136179740 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2481b60 0x5651c2480dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7136179740 4 client.15330 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651c2123270 con 0x5651c2481b60 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.615+0000 7f7131265640 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2481b60 0x5651c2480dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.616+0000 7f7131265640 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2481b60 0x5651c2480dc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d097b7c1ef5a31 server_cookie=f587e4518321820c in_seq=0 out_seq=0 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.616+0000 7f7114ff9640 10 client.15330 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7114ff9640 1 -- 172.21.15.42:0/3352328641 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651c2123270 con 0x5651c2481b60 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7114ff9640 10 client.15330 handle_command_reply: tid=0 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 client.15330 shutdown 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 msgr2=0x5651c24756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24756b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f711c09a6e0 tx=0x7f711c09af90 comp rx=0 tx=0).stop 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 shutdown_connections 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2481b60 0x5651c2480dc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c2470ae0 0x5651c247c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24756b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7114ff9640 0 client.15330 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 --2- 172.21.15.42:0/3352328641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2475170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 >> 172.21.15.42:0/3352328641 conn(0x5651c24664f0 msgr2=0x5651c24657d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 shutdown_connections 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 1 -- 172.21.15.42:0/3352328641 wait complete. 2024-03-12T04:14:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.617+0000 7f7136179740 20 client.15330 trim_cache size 0 max 16384 2024-03-12T04:14:49.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.632+0000 7f7136179740 1 Processor -- start 2024-03-12T04:14:49.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7136179740 1 -- start start 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7136179740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 0x5651c2480dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7136179740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c247c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7136179740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2475170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7136179740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2358490 con 0x5651c246af80 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7136179740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2483b40 con 0x5651c24664f0 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7136179740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2466fe0 con 0x5651c246b970 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c247c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c247c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52458/0 (socket says 172.21.15.42:52458) 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 -- 172.21.15.42:0/3381823602 learned_addr learned my addr 172.21.15.42:0/3381823602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7131a66640 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2475170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 -- 172.21.15.42:0/3381823602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 msgr2=0x5651c2475170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2475170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 -- 172.21.15.42:0/3381823602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 msgr2=0x5651c2480dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 0x5651c2480dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 -- 172.21.15.42:0/3381823602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651c236da20 con 0x5651c246af80 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7131a66640 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2475170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7130a64640 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c247c6b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f711c097120 tx=0x7f711c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a413fc9fab09bf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7116ffd640 1 -- 172.21.15.42:0/3381823602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c098a50 con 0x5651c246af80 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7116ffd640 1 -- 172.21.15.42:0/3381823602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5651c20d26c0 con 0x5651c246af80 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.633+0000 7f7116ffd640 1 -- 172.21.15.42:0/3381823602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0ba180 con 0x5651c246af80 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 -- 172.21.15.42:0/3381823602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 msgr2=0x5651c247c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c247c6b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f711c097120 tx=0x7f711c000bc0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 -- 172.21.15.42:0/3381823602 shutdown_connections 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246b970 0x5651c2475170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246af80 0x5651c247c6b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 --2- 172.21.15.42:0/3381823602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 0x5651c2480dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 -- 172.21.15.42:0/3381823602 >> 172.21.15.42:0/3381823602 conn(0x5651c246c750 msgr2=0x5651c24668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 -- 172.21.15.42:0/3381823602 shutdown_connections 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 -- 172.21.15.42:0/3381823602 wait complete. 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 -- start start 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.634+0000 7f7136179740 1 Processor -- start 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 0x5651c2479520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2479a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2479ff0 con 0x5651c246b970 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c247a160 con 0x5651c24664f0 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c24d7cf0 con 0x5651c246af80 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 10 client.0 _ll_register_callbacks cb 0x7f712800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24d7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52466/0 (socket says 172.21.15.42:52466) 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 -- 172.21.15.42:0/277372958 learned_addr learned my addr 172.21.15.42:0/277372958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131265640 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 0x5651c2479520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7130a64640 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2479a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 -- 172.21.15.42:0/277372958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 msgr2=0x5651c2479a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2479a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 -- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 msgr2=0x5651c2479520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 0x5651c2479520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 -- 172.21.15.42:0/277372958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651c23d12b0 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7131a66640 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24d7760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7124009ca0 tx=0x7f71240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bef0840c4564fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7114ff9640 1 -- 172.21.15.42:0/277372958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124003b70 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7114ff9640 1 -- 172.21.15.42:0/277372958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7124003cd0 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651c2481870 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7114ff9640 1 -- 172.21.15.42:0/277372958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7124003e90 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.635+0000 7f7136179740 10 client.15336 fetch_fsmap learned FSMap version 26 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5651c2393780 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7114ff9640 1 -- 172.21.15.42:0/277372958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f712405d020 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7114ff9640 1 -- 172.21.15.42:0/277372958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f712404b030 con 0x5651c246b970 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7136179740 10 client.15336 fetch_fsmap finished waiting for FSMap version 26 2024-03-12T04:14:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7136179740 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7136179740 20 client.15336 populate_metadata read hostname 'smithi042' 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7136179740 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2470ae0 0x5651c24d9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7136179740 4 client.15336 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5651c2482b90 con 0x5651c2470ae0 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7131265640 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2470ae0 0x5651c24d9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7114ff9640 10 client.15336 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.636+0000 7f7131265640 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2470ae0 0x5651c24d9510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d77ea4aa18fe3a0 server_cookie=815240d4f8f5cb1e in_seq=0 out_seq=0 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7114ff9640 1 -- 172.21.15.42:0/277372958 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5651c2482b90 con 0x5651c2470ae0 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7114ff9640 10 client.15336 handle_command_reply: tid=0 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 client.15336 shutdown 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 msgr2=0x5651c24d7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24d7760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7124009ca0 tx=0x7f71240095c0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 shutdown_connections 2024-03-12T04:14:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5651c2470ae0 0x5651c24d9510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c246b970 0x5651c24d7760 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c246af80 0x5651c2479a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7114ff9640 0 client.15336 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 --2- 172.21.15.42:0/277372958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651c24664f0 0x5651c2479520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 >> 172.21.15.42:0/277372958 conn(0x5651c246c750 msgr2=0x5651c2475170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 shutdown_connections 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 1 -- 172.21.15.42:0/277372958 wait complete. 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.637+0000 7f7136179740 20 client.15336 trim_cache size 0 max 16384 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7cca013a-e025-11ee-95c8-87774f69a715", 2024-03-12T04:14:49.646 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24515, 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 114.87651358399999, 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 120.682899605 2024-03-12T04:14:49.647 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f71180a1ab0 msgr2=0x7f71180a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f71180a1ab0 0x7f71180a3f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7124006010 tx=0x7f7124032340 comp rx=0 tx=0).stop 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102390 msgr2=0x7f712c197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102390 0x7f712c197680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e9f0 tx=0x7f711c00eeb0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 shutdown_connections 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f712c102d50 0x7f712c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c102390 0x7f712c197680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f71180a1ab0 0x7f71180a3f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 --2- 172.21.15.42:0/2826613616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712c1011a0 0x7f712c197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 >> 172.21.15.42:0/2826613616 conn(0x7f712c0fc710 msgr2=0x7f712c105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 shutdown_connections 2024-03-12T04:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:49.638+0000 7f71334c9640 1 -- 172.21.15.42:0/2826613616 wait complete. 2024-03-12T04:14:49.651 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-12T04:14:49.679 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-12T04:14:49.679 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-12T04:14:49.679 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-12T04:14:49.679 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-12T04:14:49.679 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:49.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:49.680 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-03-12T04:14:49.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:49.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-03-12T04:14:49.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:49 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/601402342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:49.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:49 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3477463552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:49.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:49 smithi042 ceph-mon[32260]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-12T04:14:49.762 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-03-12T04:14:49.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:14:49.764 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-12T04:14:49.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph fs ls 2024-03-12T04:14:49.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:49 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/601402342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:49.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:49 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3477463552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:49.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:49 smithi119 ceph-mon[36119]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-12T04:14:49.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:49 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/601402342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:14:49.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:49 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3477463552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:49.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:49 smithi136 ceph-mon[35571]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-12T04:14:50.047 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 -- 172.21.15.42:0/1638238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0101560 msgr2=0x7fe2b010d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/1638238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0101560 0x7fe2b010d9a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009990 tx=0x7fe2a0009ca0 comp rx=0 tx=0).stop 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 -- 172.21.15.42:0/1638238292 shutdown_connections 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/1638238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0101560 0x7fe2b010d9a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/1638238292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b0106bd0 0x7fe2b0106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/1638238292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b0100ba0 0x7fe2b0101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 -- 172.21.15.42:0/1638238292 >> 172.21.15.42:0/1638238292 conn(0x7fe2b0076bc0 msgr2=0x7fe2b0076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 -- 172.21.15.42:0/1638238292 shutdown_connections 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.959+0000 7fe2b7b36700 1 -- 172.21.15.42:0/1638238292 wait complete. 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 Processor -- start 2024-03-12T04:14:51.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 -- start start 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b0100ba0 0x7fe2b0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b0101560 0x7fe2b0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0106bd0 0x7fe2b019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b019cea0 con 0x7fe2b0106bd0 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b019d010 con 0x7fe2b0100ba0 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b7b36700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b019d1d0 con 0x7fe2b0101560 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b60d3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0106bd0 0x7fe2b019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b60d3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0106bd0 0x7fe2b019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49898/0 (socket says 172.21.15.42:49898) 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b60d3700 1 -- 172.21.15.42:0/2035149394 learned_addr learned my addr 172.21.15.42:0/2035149394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b50d1700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b0101560 0x7fe2b0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.960+0000 7fe2b58d2700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b0100ba0 0x7fe2b0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b60d3700 1 -- 172.21.15.42:0/2035149394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b0101560 msgr2=0x7fe2b0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b60d3700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b0101560 0x7fe2b0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b60d3700 1 -- 172.21.15.42:0/2035149394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b0100ba0 msgr2=0x7fe2b0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b60d3700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b0100ba0 0x7fe2b0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b60d3700 1 -- 172.21.15.42:0/2035149394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a00095f0 con 0x7fe2b0106bd0 2024-03-12T04:14:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b60d3700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0106bd0 0x7fe2b019c890 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005910 tx=0x7fe2a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2aeffd700 1 -- 172.21.15.42:0/2035149394 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a001d070 con 0x7fe2b0106bd0 2024-03-12T04:14:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b019d450 con 0x7fe2b0106bd0 2024-03-12T04:14:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2aeffd700 1 -- 172.21.15.42:0/2035149394 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2a000fb20 con 0x7fe2b0106bd0 2024-03-12T04:14:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2aeffd700 1 -- 172.21.15.42:0/2035149394 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0022b10 con 0x7fe2b0106bd0 2024-03-12T04:14:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.961+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b01a4d10 con 0x7fe2b0106bd0 2024-03-12T04:14:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.962+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b01021d0 con 0x7fe2b0106bd0 2024-03-12T04:14:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.963+0000 7fe2aeffd700 1 -- 172.21.15.42:0/2035149394 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe2a000fc90 con 0x7fe2b0106bd0 2024-03-12T04:14:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.964+0000 7fe2aeffd700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2980a1b40 0x7fe2980a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.964+0000 7fe2b58d2700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2980a1b40 0x7fe2980a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.964+0000 7fe2aeffd700 1 -- 172.21.15.42:0/2035149394 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe2a00cef00 con 0x7fe2b0106bd0 2024-03-12T04:14:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.964+0000 7fe2b58d2700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2980a1b40 0x7fe2980a4000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e270 tx=0x7fe2a8009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:51.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:51.967+0000 7fe2aeffd700 1 -- 172.21.15.42:0/2035149394 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2a0098170 con 0x7fe2b0106bd0 2024-03-12T04:14:52.185 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:51 smithi042 ceph-mon[32260]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:52.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.219+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2b0110a60 con 0x7fe2b0106bd0 2024-03-12T04:14:52.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.220+0000 7fe2aeffd700 1 -- 172.21.15.42:0/2035149394 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2a009b830 con 0x7fe2b0106bd0 2024-03-12T04:14:52.222 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-12T04:14:52.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2980a1b40 msgr2=0x7fe2980a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:52.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2980a1b40 0x7fe2980a4000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e270 tx=0x7fe2a8009040 comp rx=0 tx=0).stop 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0106bd0 msgr2=0x7fe2b019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0106bd0 0x7fe2b019c890 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005910 tx=0x7fe2a0004730 comp rx=0 tx=0).stop 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 shutdown_connections 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe2980a1b40 0x7fe2980a4000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b0106bd0 0x7fe2b019c890 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b0100ba0 0x7fe2b0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 --2- 172.21.15.42:0/2035149394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b0101560 0x7fe2b0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 >> 172.21.15.42:0/2035149394 conn(0x7fe2b0076bc0 msgr2=0x7fe2b00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 shutdown_connections 2024-03-12T04:14:52.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:52.222+0000 7fe2b7b36700 1 -- 172.21.15.42:0/2035149394 wait complete. 2024-03-12T04:14:52.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:51 smithi119 ceph-mon[36119]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:52.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:51 smithi136 ceph-mon[35571]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T04:14:52.892 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-12T04:14:52.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:52.893 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-03-12T04:14:52.908 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41965sec preferred_lft 41965sec 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-03-12T04:14:52.909 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-03-12T04:14:52.910 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-12T04:14:52.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:52.910 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-03-12T04:14:52.910 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-03-12T04:14:52.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-03-12T04:14:52.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-03-12T04:14:52.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-03-12T04:14:52.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-12T04:14:52.910 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-03-12T04:14:53.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.075 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T04:14:53.101 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-03-12T04:14:53.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.103 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-03-12T04:14:53.172 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-03-12T04:14:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T04:14:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-12T04:14:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-12T04:14:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T04:14:53.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.173 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-03-12T04:14:53.173 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-03-12T04:14:53.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-12T04:14:53.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-12T04:14:53.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-12T04:14:53.174 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-03-12T04:14:53.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:52 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2035149394' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-12T04:14:53.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.414 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-03-12T04:14:53.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:52 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2035149394' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-12T04:14:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:52 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2035149394' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-12T04:14:53.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.469 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-03-12T04:14:53.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.523 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-03-12T04:14:53.523 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-03-12T04:14:53.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:14:53.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-12T04:14:53.523 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-03-12T04:14:53.605 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-12T04:14:53.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-12T04:14:53.606 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-03-12T04:14:53.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:53.747 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-03-12T04:14:53.747 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-03-12T04:14:53.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-03-12T04:14:53.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-03-12T04:14:53.747 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-03-12T04:14:53.790 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-12T04:14:53.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T04:14:53.790 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:53.844 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-12T04:14:53.845 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T04:14:53.845 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:53.899 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-03-12T04:14:53.966 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-03-12T04:14:54.022 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-03-12T04:14:54.022 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-03-12T04:14:54.022 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-03-12T04:14:54.022 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-03-12T04:14:54.022 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-03-12T04:14:54.022 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-03-12T04:14:54.023 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/7a7b6618f707fda70007416825d3afd6dd89e16bd9cd4df0c9ef57898378eb35/merged 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3f50e22543facfefb6b374844324601da5909153d9e0a2f421ba2b05a359c6c5/merged 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/d9bb4bf25c6836f532201c4d00400a3f7c3d7c260928f5d2c283656902b791e9/merged 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/cef5d7aea60bc7010169bedd82aa3793aac04a095c8b2cd101b752ff48ce0552/merged 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4f5db9aa364fee3a2cd9308a305c86e324d8b1a7fa849e4a5c5cfba9ec2b766a/merged 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ede19f5b36a049244a2483d18fd0f409f5d496109f48b9d75052d6a7f4e21c57/merged 2024-03-12T04:14:54.024 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/fa67d47d8bcd6440e521bc3c58d42c8f4ec37221fe9a63a60e02a4fdade523c3/merged 2024-03-12T04:14:54.025 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/9aa6785dc874cd7038109ee3b7c406f57ed632f61df396684ba898e5b0efca8f/merged 2024-03-12T04:14:54.025 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3bcbd2975c62824aca712b30a555ea1496ecda68c3dabc6c6380a4c3856e5210/merged 2024-03-12T04:14:54.025 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0f9546f982ff1b6d6f12726c237ce555c686f3fdd8ece912859819672613209b/merged 2024-03-12T04:14:54.025 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-03-12T04:14:54.025 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:14:54.025 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:14:54.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:54.025 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-03-12T04:14:54.076 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-12T04:14:54.076 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/42c61c3a-49f9-4165-a046-fc4ded319250 --client_fs=cephfs) 2024-03-12T04:14:54.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-03-12T04:14:54.146 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-03-12T04:14:54.201 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-03-12T04:14:54.201 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-03-12T04:14:54.201 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-03-12T04:14:54.201 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-03-12T04:14:54.201 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-03-12T04:14:54.201 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-03-12T04:14:54.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/7a7b6618f707fda70007416825d3afd6dd89e16bd9cd4df0c9ef57898378eb35/merged 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3f50e22543facfefb6b374844324601da5909153d9e0a2f421ba2b05a359c6c5/merged 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/d9bb4bf25c6836f532201c4d00400a3f7c3d7c260928f5d2c283656902b791e9/merged 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/cef5d7aea60bc7010169bedd82aa3793aac04a095c8b2cd101b752ff48ce0552/merged 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4f5db9aa364fee3a2cd9308a305c86e324d8b1a7fa849e4a5c5cfba9ec2b766a/merged 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ede19f5b36a049244a2483d18fd0f409f5d496109f48b9d75052d6a7f4e21c57/merged 2024-03-12T04:14:54.203 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/fa67d47d8bcd6440e521bc3c58d42c8f4ec37221fe9a63a60e02a4fdade523c3/merged 2024-03-12T04:14:54.204 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/9aa6785dc874cd7038109ee3b7c406f57ed632f61df396684ba898e5b0efca8f/merged 2024-03-12T04:14:54.204 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3bcbd2975c62824aca712b30a555ea1496ecda68c3dabc6c6380a4c3856e5210/merged 2024-03-12T04:14:54.204 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0f9546f982ff1b6d6f12726c237ce555c686f3fdd8ece912859819672613209b/merged 2024-03-12T04:14:54.204 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-03-12T04:14:54.204 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:14:54.204 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:14:54.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:54.204 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-03-12T04:14:54.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-03-12T04:14:54.213+0000 7fe4cb80b480 -1 init, newargv = 0x55ec9dd6f030 newargc=15 2024-03-12T04:14:54.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[81627]: starting ceph client 2024-03-12T04:14:54.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[81627]: starting fuse 2024-03-12T04:14:54.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:53 smithi042 ceph-mon[32260]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:54.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:53 smithi119 ceph-mon[36119]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:54.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:53 smithi136 ceph-mon[35571]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-12T04:14:55.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-03-12T04:14:55.366 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-03-12T04:14:55.424 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/7a7b6618f707fda70007416825d3afd6dd89e16bd9cd4df0c9ef57898378eb35/merged 2024-03-12T04:14:55.425 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3f50e22543facfefb6b374844324601da5909153d9e0a2f421ba2b05a359c6c5/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/d9bb4bf25c6836f532201c4d00400a3f7c3d7c260928f5d2c283656902b791e9/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/cef5d7aea60bc7010169bedd82aa3793aac04a095c8b2cd101b752ff48ce0552/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4f5db9aa364fee3a2cd9308a305c86e324d8b1a7fa849e4a5c5cfba9ec2b766a/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ede19f5b36a049244a2483d18fd0f409f5d496109f48b9d75052d6a7f4e21c57/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/fa67d47d8bcd6440e521bc3c58d42c8f4ec37221fe9a63a60e02a4fdade523c3/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/9aa6785dc874cd7038109ee3b7c406f57ed632f61df396684ba898e5b0efca8f/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3bcbd2975c62824aca712b30a555ea1496ecda68c3dabc6c6380a4c3856e5210/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0f9546f982ff1b6d6f12726c237ce555c686f3fdd8ece912859819672613209b/merged 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:14:55.426 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:55.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:55.427 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-03-12T04:14:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:119 2024-03-12T04:14:55.481 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-03-12T04:14:55.481 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-03-12T04:14:55.482 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-03-12T04:14:55.608 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.81627.asok 2024-03-12T04:14:55.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-12T04:14:55.620 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81627.asok 2024-03-12T04:14:55.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:55.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81627.asok status 2024-03-12T04:14:55.758 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "94a539707733325322964c6a187c6a1b41be1daf", 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "81627", 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/volumes/qa/sv_0/42c61c3a-49f9-4165-a046-fc4ded319250" 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 15348, 2024-03-12T04:14:55.759 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 15348 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1447642547 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1447642547 2024-03-12T04:14:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.15348 192.168.0.1:0/1447642547", 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/1447642547", 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 5, 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 26, 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 89, 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-03-12T04:14:55.761 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:14:55.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:55.767 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-03-12T04:14:55.767 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:55.781 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-03-12T04:14:55.782 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:55.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:14:55.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-12T04:14:55.851 INFO:teuthology.run_tasks:Running task exec... 2024-03-12T04:14:55.861 INFO:teuthology.task.exec:Executing custom commands... 2024-03-12T04:14:55.861 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T04:14:55.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-12T04:14:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/70475877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0106e80 msgr2=0x7fa4a01092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/70475877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0106e80 0x7fa4a01092d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa498009ca0 tx=0x7fa49802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/70475877 shutdown_connections 2024-03-12T04:14:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/70475877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0106e80 0x7fa4a01092d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/70475877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a0104550 0x7fa4a0106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/70475877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4a0101c20 0x7fa4a0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/70475877 >> 172.21.15.42:0/70475877 conn(0x7fa4a00fb610 msgr2=0x7fa4a00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/70475877 shutdown_connections 2024-03-12T04:14:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.993+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/70475877 wait complete. 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 Processor -- start 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 -- start start 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a0101c20 0x7fa4a01970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0104550 0x7fa4a01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4a0106e80 0x7fa4a019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a019eb90 con 0x7fa4a0104550 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a019ed00 con 0x7fa4a0101c20 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a6f7a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a019eec0 con 0x7fa4a0106e80 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa49ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0104550 0x7fa4a01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa49ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0104550 0x7fa4a01975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49916/0 (socket says 172.21.15.42:49916) 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa49ffff640 1 -- 172.21.15.42:0/2130211897 learned_addr learned my addr 172.21.15.42:0/2130211897 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.994+0000 7fa4a4d16640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a0101c20 0x7fa4a01970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa4a5517640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4a0106e80 0x7fa4a019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49ffff640 1 -- 172.21.15.42:0/2130211897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4a0106e80 msgr2=0x7fa4a019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49ffff640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4a0106e80 0x7fa4a019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49ffff640 1 -- 172.21.15.42:0/2130211897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a0101c20 msgr2=0x7fa4a01970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49ffff640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a0101c20 0x7fa4a01970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49ffff640 1 -- 172.21.15.42:0/2130211897 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa498009900 con 0x7fa4a0104550 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49ffff640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0104550 0x7fa4a01975e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005fd0 tx=0x7fa48c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49dffb640 1 -- 172.21.15.42:0/2130211897 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c009c30 con 0x7fa4a0104550 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49dffb640 1 -- 172.21.15.42:0/2130211897 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa48c0044d0 con 0x7fa4a0104550 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a019f1a0 con 0x7fa4a0104550 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa49dffb640 1 -- 172.21.15.42:0/2130211897 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c003e50 con 0x7fa4a0104550 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.995+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a019f700 con 0x7fa4a0104550 2024-03-12T04:14:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.996+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a00fd100 con 0x7fa4a0104550 2024-03-12T04:14:56.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.997+0000 7fa49dffb640 1 -- 172.21.15.42:0/2130211897 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa48c004000 con 0x7fa4a0104550 2024-03-12T04:14:56.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.998+0000 7fa49dffb640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa4900a1b50 0x7fa4900a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:56.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.998+0000 7fa4a4d16640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa4900a1b50 0x7fa4900a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:56.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.998+0000 7fa49dffb640 1 -- 172.21.15.42:0/2130211897 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa48c0ccfb0 con 0x7fa4a0104550 2024-03-12T04:14:56.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:55.998+0000 7fa4a4d16640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa4900a1b50 0x7fa4900a4010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa494009bd0 tx=0x7fa494005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:56.002+0000 7fa49dffb640 1 -- 172.21.15.42:0/2130211897 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa48c096190 con 0x7fa4a0104550 2024-03-12T04:14:56.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:56.224+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fa4a010ca60 con 0x7fa4a0104550 2024-03-12T04:14:56.259 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:55 smithi136 ceph-mon[35571]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-12T04:14:56.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:55 smithi042 ceph-mon[32260]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-12T04:14:56.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:55 smithi119 ceph-mon[36119]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-12T04:14:57.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:56.997+0000 7fa49dffb640 1 -- 172.21.15.42:0/2130211897 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7fa48c099850 con 0x7fa4a0104550 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa4900a1b50 msgr2=0x7fa4900a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa4900a1b50 0x7fa4900a4010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa494009bd0 tx=0x7fa494005f90 comp rx=0 tx=0).stop 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0104550 msgr2=0x7fa4a01975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0104550 0x7fa4a01975e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005fd0 tx=0x7fa48c00dde0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 shutdown_connections 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4a0106e80 0x7fa4a019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a0104550 0x7fa4a01975e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa4900a1b50 0x7fa4900a4010 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 --2- 172.21.15.42:0/2130211897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a0101c20 0x7fa4a01970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.000+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 >> 172.21.15.42:0/2130211897 conn(0x7fa4a00fb610 msgr2=0x7fa4a0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:57.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.001+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 shutdown_connections 2024-03-12T04:14:57.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.001+0000 7fa4a6f7a640 1 -- 172.21.15.42:0/2130211897 wait complete. 2024-03-12T04:14:57.016 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-12T04:14:57.027 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-12T04:14:57.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.144+0000 7fa3c178a640 1 -- 172.21.15.42:0/3859824580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc100a40 msgr2=0x7fa3bc100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.144+0000 7fa3c178a640 1 --2- 172.21.15.42:0/3859824580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc100a40 0x7fa3bc100ec0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009900 tx=0x7fa3a402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 -- 172.21.15.42:0/3859824580 shutdown_connections 2024-03-12T04:14:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 --2- 172.21.15.42:0/3859824580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3bc101400 0x7fa3bc107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 --2- 172.21.15.42:0/3859824580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc100a40 0x7fa3bc100ec0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 --2- 172.21.15.42:0/3859824580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0ff850 0x7fa3bc0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 -- 172.21.15.42:0/3859824580 >> 172.21.15.42:0/3859824580 conn(0x7fa3bc0fadc0 msgr2=0x7fa3bc0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 -- 172.21.15.42:0/3859824580 shutdown_connections 2024-03-12T04:14:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 -- 172.21.15.42:0/3859824580 wait complete. 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 Processor -- start 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.145+0000 7fa3c178a640 1 -- start start 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3bc0ff850 0x7fa3bc196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc100a40 0x7fa3bc1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc101400 0x7fa3bc19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc19db70 con 0x7fa3bc101400 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc19dce0 con 0x7fa3bc100a40 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc19dea0 con 0x7fa3bc0ff850 2024-03-12T04:14:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc101400 0x7fa3bc19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc101400 0x7fa3bc19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49930/0 (socket says 172.21.15.42:49930) 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 -- 172.21.15.42:0/210392312 learned_addr learned my addr 172.21.15.42:0/210392312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3baffd640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3bc0ff850 0x7fa3bc196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3ba7fc640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc100a40 0x7fa3bc1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 -- 172.21.15.42:0/210392312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3bc0ff850 msgr2=0x7fa3bc196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3bc0ff850 0x7fa3bc196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 -- 172.21.15.42:0/210392312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc100a40 msgr2=0x7fa3bc1965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc100a40 0x7fa3bc1965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 -- 172.21.15.42:0/210392312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4009560 con 0x7fa3bc101400 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3bb7fe640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc101400 0x7fa3bc19d630 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eaa0 tx=0x7fa3ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3b3fff640 1 -- 172.21.15.42:0/210392312 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00cdb0 con 0x7fa3bc101400 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc19e180 con 0x7fa3bc101400 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3b3fff640 1 -- 172.21.15.42:0/210392312 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa3ac005020 con 0x7fa3bc101400 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3b3fff640 1 -- 172.21.15.42:0/210392312 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0116d0 con 0x7fa3bc101400 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.146+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc19e690 con 0x7fa3bc101400 2024-03-12T04:14:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.147+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc106bf0 con 0x7fa3bc101400 2024-03-12T04:14:57.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.148+0000 7fa3b3fff640 1 -- 172.21.15.42:0/210392312 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa3ac005190 con 0x7fa3bc101400 2024-03-12T04:14:57.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.149+0000 7fa3b3fff640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa3a80a1a30 0x7fa3a80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.149+0000 7fa3b3fff640 1 -- 172.21.15.42:0/210392312 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa3ac0cd610 con 0x7fa3bc101400 2024-03-12T04:14:57.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.149+0000 7fa3baffd640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa3a80a1a30 0x7fa3a80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.149+0000 7fa3baffd640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa3a80a1a30 0x7fa3a80a3ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4005fd0 tx=0x7fa3b4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:57.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.153+0000 7fa3b3fff640 1 -- 172.21.15.42:0/210392312 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3ac096850 con 0x7fa3bc101400 2024-03-12T04:14:57.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:56 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2130211897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-12T04:14:57.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.335+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3bc02ce40 con 0x7fa3bc101400 2024-03-12T04:14:57.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.337+0000 7fa3b3fff640 1 -- 172.21.15.42:0/210392312 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7fa3ac099f10 con 0x7fa3bc101400 2024-03-12T04:14:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:57.340 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa3a80a1a30 msgr2=0x7fa3a80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa3a80a1a30 0x7fa3a80a3ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4005fd0 tx=0x7fa3b4005850 comp rx=0 tx=0).stop 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc101400 msgr2=0x7fa3bc19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc101400 0x7fa3bc19d630 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eaa0 tx=0x7fa3ac00ef60 comp rx=0 tx=0).stop 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 shutdown_connections 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3bc101400 0x7fa3bc19d630 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa3a80a1a30 0x7fa3a80a3ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc100a40 0x7fa3bc1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 --2- 172.21.15.42:0/210392312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3bc0ff850 0x7fa3bc196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 >> 172.21.15.42:0/210392312 conn(0x7fa3bc0fadc0 msgr2=0x7fa3bc103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 shutdown_connections 2024-03-12T04:14:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.339+0000 7fa3c178a640 1 -- 172.21.15.42:0/210392312 wait complete. 2024-03-12T04:14:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:56 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2130211897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-12T04:14:57.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:14:57.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:14:57.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:56 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2130211897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-12T04:14:57.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.456+0000 7fcaee223640 1 -- 172.21.15.42:0/3866597358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8100080 msgr2=0x7fcae81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.456+0000 7fcaee223640 1 --2- 172.21.15.42:0/3866597358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8100080 0x7fcae81004a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009900 tx=0x7fcad802bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:57.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- 172.21.15.42:0/3866597358 shutdown_connections 2024-03-12T04:14:57.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 --2- 172.21.15.42:0/3866597358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae8101c30 0x7fcae81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 --2- 172.21.15.42:0/3866597358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae8101270 0x7fcae81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 --2- 172.21.15.42:0/3866597358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8100080 0x7fcae81004a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- 172.21.15.42:0/3866597358 >> 172.21.15.42:0/3866597358 conn(0x7fcae80fb610 msgr2=0x7fcae80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:57.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- 172.21.15.42:0/3866597358 shutdown_connections 2024-03-12T04:14:57.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- 172.21.15.42:0/3866597358 wait complete. 2024-03-12T04:14:57.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 Processor -- start 2024-03-12T04:14:57.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- start start 2024-03-12T04:14:57.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae8100080 0x7fcae81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae8101270 0x7fcae8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8101c30 0x7fcae819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819ecb0 con 0x7fcae8101c30 2024-03-12T04:14:57.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819ee20 con 0x7fcae8101270 2024-03-12T04:14:57.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.457+0000 7fcaee223640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819efe0 con 0x7fcae8100080 2024-03-12T04:14:57.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8101c30 0x7fcae819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8101c30 0x7fcae819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49940/0 (socket says 172.21.15.42:49940) 2024-03-12T04:14:57.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 -- 172.21.15.42:0/907935409 learned_addr learned my addr 172.21.15.42:0/907935409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae6ffd640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae8101270 0x7fcae8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae77fe640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae8100080 0x7fcae81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 -- 172.21.15.42:0/907935409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae8100080 msgr2=0x7fcae81971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae8100080 0x7fcae81971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 -- 172.21.15.42:0/907935409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae8101270 msgr2=0x7fcae8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae8101270 0x7fcae8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 -- 172.21.15.42:0/907935409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad8009560 con 0x7fcae8101c30 2024-03-12T04:14:57.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae6ffd640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae8101270 0x7fcae8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae7fff640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8101c30 0x7fcae819e770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcae000eaa0 tx=0x7fcae000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae4ff9640 1 -- 172.21.15.42:0/907935409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000cdb0 con 0x7fcae8101c30 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae4ff9640 1 -- 172.21.15.42:0/907935409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcae0005020 con 0x7fcae8101c30 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae819f2c0 con 0x7fcae8101c30 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcae4ff9640 1 -- 172.21.15.42:0/907935409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae00116b0 con 0x7fcae8101c30 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.458+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae819f780 con 0x7fcae8101c30 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.459+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae8107510 con 0x7fcae8101c30 2024-03-12T04:14:57.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.460+0000 7fcae4ff9640 1 -- 172.21.15.42:0/907935409 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fcae0005190 con 0x7fcae8101c30 2024-03-12T04:14:57.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.461+0000 7fcae4ff9640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcad00a1a30 0x7fcad00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.461+0000 7fcae77fe640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcad00a1a30 0x7fcad00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.461+0000 7fcae4ff9640 1 -- 172.21.15.42:0/907935409 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcae00cd710 con 0x7fcae8101c30 2024-03-12T04:14:57.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.462+0000 7fcae77fe640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcad00a1a30 0x7fcad00a3ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcad8036040 tx=0x7fcad8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:57.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.466+0000 7fcae4ff9640 1 -- 172.21.15.42:0/907935409 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcae00968f0 con 0x7fcae8101c30 2024-03-12T04:14:57.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.647+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcae802ce40 con 0x7fcae8101c30 2024-03-12T04:14:57.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.648+0000 7fcae4ff9640 1 -- 172.21.15.42:0/907935409 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcae0099fb0 con 0x7fcae8101c30 2024-03-12T04:14:57.650 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:14:57.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcad00a1a30 msgr2=0x7fcad00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcad00a1a30 0x7fcad00a3ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcad8036040 tx=0x7fcad8005fd0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8101c30 msgr2=0x7fcae819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8101c30 0x7fcae819e770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcae000eaa0 tx=0x7fcae000ef60 comp rx=0 tx=0).stop 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 shutdown_connections 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae8101c30 0x7fcae819e770 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fcad00a1a30 0x7fcad00a3ef0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae8101270 0x7fcae8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 --2- 172.21.15.42:0/907935409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae8100080 0x7fcae81971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 >> 172.21.15.42:0/907935409 conn(0x7fcae80fb610 msgr2=0x7fcae81046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 shutdown_connections 2024-03-12T04:14:57.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.650+0000 7fcaee223640 1 -- 172.21.15.42:0/907935409 wait complete. 2024-03-12T04:14:57.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:14:57.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 -- 172.21.15.42:0/2185084606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102380 msgr2=0x7f49e0102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2185084606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102380 0x7f49e0102800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009900 tx=0x7f49d002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:57.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 -- 172.21.15.42:0/2185084606 shutdown_connections 2024-03-12T04:14:57.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2185084606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49e0102d40 0x7f49e01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2185084606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102380 0x7f49e0102800 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2185084606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e0101190 0x7f49e01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 -- 172.21.15.42:0/2185084606 >> 172.21.15.42:0/2185084606 conn(0x7f49e00fc720 msgr2=0x7f49e00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 -- 172.21.15.42:0/2185084606 shutdown_connections 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.766+0000 7f49e84c0640 1 -- 172.21.15.42:0/2185084606 wait complete. 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 Processor -- start 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 -- start start 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49e0101190 0x7f49e01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e0102380 0x7f49e0197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102d40 0x7f49e019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e019ecb0 con 0x7f49e0102d40 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e019ee20 con 0x7f49e0101190 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e84c0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e019efe0 con 0x7f49e0102380 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102d40 0x7f49e019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102d40 0x7f49e019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49960/0 (socket says 172.21.15.42:49960) 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 -- 172.21.15.42:0/2274120072 learned_addr learned my addr 172.21.15.42:0/2274120072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e5a5b640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e0102380 0x7f49e0197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e625c640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49e0101190 0x7f49e01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 -- 172.21.15.42:0/2274120072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e0102380 msgr2=0x7f49e0197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e0102380 0x7f49e0197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 -- 172.21.15.42:0/2274120072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49e0101190 msgr2=0x7f49e01971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49e0101190 0x7f49e01971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 -- 172.21.15.42:0/2274120072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009560 con 0x7f49e0102d40 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e625c640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49e0101190 0x7f49e01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.767+0000 7f49e6a5d640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102d40 0x7f49e019e770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49d800b970 tx=0x7f49d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.768+0000 7f49d77fe640 1 -- 172.21.15.42:0/2274120072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800cb40 con 0x7f49e0102d40 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.768+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e019f2c0 con 0x7f49e0102d40 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.768+0000 7f49d77fe640 1 -- 172.21.15.42:0/2274120072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f49d800cca0 con 0x7f49e0102d40 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.768+0000 7f49d77fe640 1 -- 172.21.15.42:0/2274120072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80076e0 con 0x7f49e0102d40 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.768+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e019f800 con 0x7f49e0102d40 2024-03-12T04:14:57.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.769+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0108570 con 0x7f49e0102d40 2024-03-12T04:14:57.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.770+0000 7f49d77fe640 1 -- 172.21.15.42:0/2274120072 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f49d8014440 con 0x7f49e0102d40 2024-03-12T04:14:57.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.770+0000 7f49d77fe640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f49cc0a1a30 0x7f49cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:57.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.771+0000 7f49e625c640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f49cc0a1a30 0x7f49cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:57.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.771+0000 7f49d77fe640 1 -- 172.21.15.42:0/2274120072 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f49d80ccd40 con 0x7f49e0102d40 2024-03-12T04:14:57.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.771+0000 7f49e625c640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f49cc0a1a30 0x7f49cc0a3ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f49e01021e0 tx=0x7f49dc005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:57.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.775+0000 7f49d77fe640 1 -- 172.21.15.42:0/2274120072 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49d8095f80 con 0x7f49e0102d40 2024-03-12T04:14:57.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.957+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49e002ce40 con 0x7f49e0102d40 2024-03-12T04:14:57.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.958+0000 7f49d77fe640 1 -- 172.21.15.42:0/2274120072 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49d8019020 con 0x7f49e0102d40 2024-03-12T04:14:57.961 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f49cc0a1a30 msgr2=0x7f49cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f49cc0a1a30 0x7f49cc0a3ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f49e01021e0 tx=0x7f49dc005e70 comp rx=0 tx=0).stop 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102d40 msgr2=0x7f49e019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102d40 0x7f49e019e770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49d800b970 tx=0x7f49d800be30 comp rx=0 tx=0).stop 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 shutdown_connections 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49e0102d40 0x7f49e019e770 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e0102380 0x7f49e0197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f49cc0a1a30 0x7f49cc0a3ef0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 --2- 172.21.15.42:0/2274120072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49e0101190 0x7f49e01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 >> 172.21.15.42:0/2274120072 conn(0x7f49e00fc720 msgr2=0x7f49e0105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 shutdown_connections 2024-03-12T04:14:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:57.960+0000 7f49e84c0640 1 -- 172.21.15.42:0/2274120072 wait complete. 2024-03-12T04:14:57.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 -- 172.21.15.42:0/2376479725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50106e80 msgr2=0x7f0b501092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 --2- 172.21.15.42:0/2376479725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50106e80 0x7f0b501092d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009ca0 tx=0x7f0b4802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 -- 172.21.15.42:0/2376479725 shutdown_connections 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 --2- 172.21.15.42:0/2376479725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50106e80 0x7f0b501092d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 --2- 172.21.15.42:0/2376479725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b50104550 0x7f0b50106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 --2- 172.21.15.42:0/2376479725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b50101c20 0x7f0b50104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 -- 172.21.15.42:0/2376479725 >> 172.21.15.42:0/2376479725 conn(0x7f0b500fb630 msgr2=0x7f0b500fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 -- 172.21.15.42:0/2376479725 shutdown_connections 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 -- 172.21.15.42:0/2376479725 wait complete. 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 Processor -- start 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.076+0000 7f0b569f4640 1 -- start start 2024-03-12T04:14:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b569f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50101c20 0x7f0b50197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b569f4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b50104550 0x7f0b50197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b569f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b50106e80 0x7f0b5019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b569f4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019ebf0 con 0x7f0b50101c20 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b569f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019ed60 con 0x7f0b50106e80 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b569f4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019ef20 con 0x7f0b50104550 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50101c20 0x7f0b50197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50101c20 0x7f0b50197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49974/0 (socket says 172.21.15.42:49974) 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 -- 172.21.15.42:0/68231986 learned_addr learned my addr 172.21.15.42:0/68231986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4f7fe640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b50104550 0x7f0b50197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b54f91640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b50106e80 0x7f0b5019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 -- 172.21.15.42:0/68231986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b50104550 msgr2=0x7f0b50197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b50104550 0x7f0b50197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 -- 172.21.15.42:0/68231986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b50106e80 msgr2=0x7f0b5019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b50106e80 0x7f0b5019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b4ffff640 1 -- 172.21.15.42:0/68231986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48009900 con 0x7f0b50101c20 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.077+0000 7f0b54f91640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b50106e80 0x7f0b5019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.078+0000 7f0b4ffff640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50101c20 0x7f0b50197100 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000d950 tx=0x7f0b4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.078+0000 7f0b4d7fa640 1 -- 172.21.15.42:0/68231986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40009c30 con 0x7f0b50101c20 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.078+0000 7f0b4d7fa640 1 -- 172.21.15.42:0/68231986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b40009d90 con 0x7f0b50101c20 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.078+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5019f200 con 0x7f0b50101c20 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.078+0000 7f0b4d7fa640 1 -- 172.21.15.42:0/68231986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000f6b0 con 0x7f0b50101c20 2024-03-12T04:14:58.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.078+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5019f710 con 0x7f0b50101c20 2024-03-12T04:14:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.079+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b500fd120 con 0x7f0b50101c20 2024-03-12T04:14:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.080+0000 7f0b4d7fa640 1 -- 172.21.15.42:0/68231986 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0b40010aa0 con 0x7f0b50101c20 2024-03-12T04:14:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.080+0000 7f0b4d7fa640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b3c0a1a30 0x7f0b3c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.080+0000 7f0b4d7fa640 1 -- 172.21.15.42:0/68231986 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0b400cd0f0 con 0x7f0b50101c20 2024-03-12T04:14:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.080+0000 7f0b4f7fe640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b3c0a1a30 0x7f0b3c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.081+0000 7f0b4f7fe640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b3c0a1a30 0x7f0b3c0a3ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b50198610 tx=0x7f0b3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:58.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.085+0000 7f0b4d7fa640 1 -- 172.21.15.42:0/68231986 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b40096330 con 0x7f0b50101c20 2024-03-12T04:14:58.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.268+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b5002ce40 con 0x7f0b50101c20 2024-03-12T04:14:58.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.268+0000 7f0b4d7fa640 1 -- 172.21.15.42:0/68231986 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b400999f0 con 0x7f0b50101c20 2024-03-12T04:14:58.271 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b3c0a1a30 msgr2=0x7f0b3c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b3c0a1a30 0x7f0b3c0a3ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b50198610 tx=0x7f0b3800b390 comp rx=0 tx=0).stop 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50101c20 msgr2=0x7f0b50197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50101c20 0x7f0b50197100 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000d950 tx=0x7f0b4000de10 comp rx=0 tx=0).stop 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 shutdown_connections 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f0b3c0a1a30 0x7f0b3c0a3ef0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b50106e80 0x7f0b5019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b50104550 0x7f0b50197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 --2- 172.21.15.42:0/68231986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50101c20 0x7f0b50197100 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 >> 172.21.15.42:0/68231986 conn(0x7f0b500fb630 msgr2=0x7f0b50102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:58.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 shutdown_connections 2024-03-12T04:14:58.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.270+0000 7f0b569f4640 1 -- 172.21.15.42:0/68231986 wait complete. 2024-03-12T04:14:58.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:58 smithi042 ceph-mon[32260]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-12T04:14:58.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:58 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2130211897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-12T04:14:58.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:58 smithi042 ceph-mon[32260]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-12T04:14:58.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:58 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/210392312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:58.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:58 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/907935409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:58.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:58 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2274120072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:58.284 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:14:58.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.384+0000 7f9872e0b640 1 -- 172.21.15.42:0/3234924693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c106e80 msgr2=0x7f986c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.384+0000 7f9872e0b640 1 --2- 172.21.15.42:0/3234924693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c106e80 0x7f986c1092d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9854009900 tx=0x7f985402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:58.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- 172.21.15.42:0/3234924693 shutdown_connections 2024-03-12T04:14:58.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 --2- 172.21.15.42:0/3234924693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c106e80 0x7f986c1092d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 --2- 172.21.15.42:0/3234924693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986c104550 0x7f986c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 --2- 172.21.15.42:0/3234924693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c101c20 0x7f986c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- 172.21.15.42:0/3234924693 >> 172.21.15.42:0/3234924693 conn(0x7f986c0fb610 msgr2=0x7f986c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- 172.21.15.42:0/3234924693 shutdown_connections 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- 172.21.15.42:0/3234924693 wait complete. 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 Processor -- start 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- start start 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c101c20 0x7f986c101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c104550 0x7f986c0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986c106e80 0x7f986c100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c100870 con 0x7f986c101c20 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c1009e0 con 0x7f986c104550 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9872e0b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c1a0f80 con 0x7f986c106e80 2024-03-12T04:14:58.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9870ba7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c101c20 0x7f986c101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9870ba7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c101c20 0x7f986c101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49998/0 (socket says 172.21.15.42:49998) 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.385+0000 7f9870ba7640 1 -- 172.21.15.42:0/1854717383 learned_addr learned my addr 172.21.15.42:0/1854717383 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f98713a8640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986c106e80 0x7f986c100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f986bfff640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c104550 0x7f986c0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9870ba7640 1 -- 172.21.15.42:0/1854717383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986c106e80 msgr2=0x7f986c100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9870ba7640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986c106e80 0x7f986c100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9870ba7640 1 -- 172.21.15.42:0/1854717383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c104550 msgr2=0x7f986c0ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9870ba7640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c104550 0x7f986c0ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9870ba7640 1 -- 172.21.15.42:0/1854717383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854009560 con 0x7f986c101c20 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f98713a8640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986c106e80 0x7f986c100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9870ba7640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c101c20 0x7f986c101590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f986400ea20 tx=0x7f986400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9869ffb640 1 -- 172.21.15.42:0/1854717383 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864009700 con 0x7f986c101c20 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9869ffb640 1 -- 172.21.15.42:0/1854717383 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f986400cd80 con 0x7f986c101c20 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c1a1230 con 0x7f986c101c20 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9869ffb640 1 -- 172.21.15.42:0/1854717383 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864018710 con 0x7f986c101c20 2024-03-12T04:14:58.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.386+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c1a1790 con 0x7f986c101c20 2024-03-12T04:14:58.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.387+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c0fd100 con 0x7f986c101c20 2024-03-12T04:14:58.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.388+0000 7f9869ffb640 1 -- 172.21.15.42:0/1854717383 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9864010430 con 0x7f986c101c20 2024-03-12T04:14:58.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.389+0000 7f9869ffb640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f98580a1ab0 0x7f98580a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.389+0000 7f9869ffb640 1 -- 172.21.15.42:0/1854717383 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9864014070 con 0x7f986c101c20 2024-03-12T04:14:58.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.389+0000 7f986bfff640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f98580a1ab0 0x7f98580a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.389+0000 7f986bfff640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f98580a1ab0 0x7f98580a3f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f986c101300 tx=0x7f985c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:58.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.392+0000 7f9869ffb640 1 -- 172.21.15.42:0/1854717383 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9864096ed0 con 0x7f986c101c20 2024-03-12T04:14:58.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:58 smithi119 ceph-mon[36119]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-12T04:14:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:58 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2130211897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-12T04:14:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:58 smithi119 ceph-mon[36119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-12T04:14:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:58 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/210392312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:58 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/907935409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:58.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:58 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2274120072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:58 smithi136 ceph-mon[35571]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-12T04:14:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:58 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2130211897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-12T04:14:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:58 smithi136 ceph-mon[35571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-12T04:14:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:58 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/210392312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:58 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/907935409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:58.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:58 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2274120072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:14:58.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.577+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f986c02ce40 con 0x7f986c101c20 2024-03-12T04:14:58.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.578+0000 7f9869ffb640 1 -- 172.21.15.42:0/1854717383 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f9864026090 con 0x7f986c101c20 2024-03-12T04:14:58.581 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:58.581 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f98580a1ab0 msgr2=0x7f98580a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f98580a1ab0 0x7f98580a3f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f986c101300 tx=0x7f985c009500 comp rx=0 tx=0).stop 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c101c20 msgr2=0x7f986c101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c101c20 0x7f986c101590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f986400ea20 tx=0x7f986400eee0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 shutdown_connections 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986c106e80 0x7f986c100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f98580a1ab0 0x7f98580a3f70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c104550 0x7f986c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 --2- 172.21.15.42:0/1854717383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f986c101c20 0x7f986c101590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 >> 172.21.15.42:0/1854717383 conn(0x7f986c0fb610 msgr2=0x7f986c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:58.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 shutdown_connections 2024-03-12T04:14:58.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.580+0000 7f9872e0b640 1 -- 172.21.15.42:0/1854717383 wait complete. 2024-03-12T04:14:58.594 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:14:58.595 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 -- 172.21.15.42:0/108770261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c106d40 msgr2=0x7f2f2c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 --2- 172.21.15.42:0/108770261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c106d40 0x7f2f2c109190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009ca0 tx=0x7f2f2402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 -- 172.21.15.42:0/108770261 shutdown_connections 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 --2- 172.21.15.42:0/108770261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c106d40 0x7f2f2c109190 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 --2- 172.21.15.42:0/108770261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f2c104410 0x7f2f2c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 --2- 172.21.15.42:0/108770261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f2c101ae0 0x7f2f2c103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 -- 172.21.15.42:0/108770261 >> 172.21.15.42:0/108770261 conn(0x7f2f2c0fb4f0 msgr2=0x7f2f2c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 -- 172.21.15.42:0/108770261 shutdown_connections 2024-03-12T04:14:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.698+0000 7f2f3159e640 1 -- 172.21.15.42:0/108770261 wait complete. 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 Processor -- start 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 -- start start 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c101ae0 0x7f2f2c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f2c104410 0x7f2f2c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f2c106d40 0x7f2f2c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c19eaa0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c19ec10 con 0x7f2f2c106d40 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f3159e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c19edd0 con 0x7f2f2c104410 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f2bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c101ae0 0x7f2f2c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f2bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c101ae0 0x7f2f2c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50020/0 (socket says 172.21.15.42:50020) 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f2bfff640 1 -- 172.21.15.42:0/3333087113 learned_addr learned my addr 172.21.15.42:0/3333087113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f30d9d640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f2c106d40 0x7f2f2c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.699+0000 7f2f2b7fe640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f2c104410 0x7f2f2c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f2bfff640 1 -- 172.21.15.42:0/3333087113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f2c104410 msgr2=0x7f2f2c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f2bfff640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f2c104410 0x7f2f2c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f2bfff640 1 -- 172.21.15.42:0/3333087113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f2c106d40 msgr2=0x7f2f2c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f2bfff640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f2c106d40 0x7f2f2c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f2bfff640 1 -- 172.21.15.42:0/3333087113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24009900 con 0x7f2f2c101ae0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f2b7fe640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f2c104410 0x7f2f2c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f2bfff640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c101ae0 0x7f2f2c196fb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000d950 tx=0x7f2f2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f297fa640 1 -- 172.21.15.42:0/3333087113 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20009c30 con 0x7f2f2c101ae0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f297fa640 1 -- 172.21.15.42:0/3333087113 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f20009d90 con 0x7f2f2c101ae0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c19f0b0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f297fa640 1 -- 172.21.15.42:0/3333087113 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000f6b0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.700+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c19f5c0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.701+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c0fcfe0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.702+0000 7f2f297fa640 1 -- 172.21.15.42:0/3333087113 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2f20010aa0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.702+0000 7f2f297fa640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2f140a1a30 0x7f2f140a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:58.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.703+0000 7f2f2b7fe640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2f140a1a30 0x7f2f140a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:58.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.703+0000 7f2f297fa640 1 -- 172.21.15.42:0/3333087113 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2f200cd0f0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.703+0000 7f2f2b7fe640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2f140a1a30 0x7f2f140a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0058c0 tx=0x7f2f1c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:58.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.707+0000 7f2f297fa640 1 -- 172.21.15.42:0/3333087113 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f20096330 con 0x7f2f2c101ae0 2024-03-12T04:14:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.892+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f2c0fca50 con 0x7f2f2c101ae0 2024-03-12T04:14:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.893+0000 7f2f297fa640 1 -- 172.21.15.42:0/3333087113 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f200999f0 con 0x7f2f2c101ae0 2024-03-12T04:14:58.895 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:14:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2f140a1a30 msgr2=0x7f2f140a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2f140a1a30 0x7f2f140a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0058c0 tx=0x7f2f1c005850 comp rx=0 tx=0).stop 2024-03-12T04:14:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c101ae0 msgr2=0x7f2f2c196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c101ae0 0x7f2f2c196fb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000d950 tx=0x7f2f2000de10 comp rx=0 tx=0).stop 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 shutdown_connections 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f2f140a1a30 0x7f2f140a3ef0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f2c106d40 0x7f2f2c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f2c104410 0x7f2f2c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 --2- 172.21.15.42:0/3333087113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f2c101ae0 0x7f2f2c196fb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 >> 172.21.15.42:0/3333087113 conn(0x7f2f2c0fb4f0 msgr2=0x7f2f2c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 shutdown_connections 2024-03-12T04:14:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:58.895+0000 7f2f3159e640 1 -- 172.21.15.42:0/3333087113 wait complete. 2024-03-12T04:14:58.908 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:14:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.012+0000 7ff819676640 1 -- 172.21.15.42:0/3585169001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814106e70 msgr2=0x7ff8141092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.012+0000 7ff819676640 1 --2- 172.21.15.42:0/3585169001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814106e70 0x7ff8141092c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff804009ca0 tx=0x7ff80402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- 172.21.15.42:0/3585169001 shutdown_connections 2024-03-12T04:14:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 --2- 172.21.15.42:0/3585169001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814106e70 0x7ff8141092c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 --2- 172.21.15.42:0/3585169001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff814104540 0x7ff814106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 --2- 172.21.15.42:0/3585169001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff814101c10 0x7ff814104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- 172.21.15.42:0/3585169001 >> 172.21.15.42:0/3585169001 conn(0x7ff8140fb620 msgr2=0x7ff8140fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- 172.21.15.42:0/3585169001 shutdown_connections 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- 172.21.15.42:0/3585169001 wait complete. 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 Processor -- start 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- start start 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814101c10 0x7ff814197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff814104540 0x7ff814197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff814106e70 0x7ff81419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419ec10 con 0x7ff814101c10 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419ed80 con 0x7ff814104540 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff819676640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419ef40 con 0x7ff814106e70 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.013+0000 7ff812ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814101c10 0x7ff814197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814101c10 0x7ff814197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50036/0 (socket says 172.21.15.42:50036) 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 -- 172.21.15.42:0/1316492865 learned_addr learned my addr 172.21.15.42:0/1316492865 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff8137fe640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff814106e70 0x7ff81419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff8127fc640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff814104540 0x7ff814197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 -- 172.21.15.42:0/1316492865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff814106e70 msgr2=0x7ff81419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff814106e70 0x7ff81419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 -- 172.21.15.42:0/1316492865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff814104540 msgr2=0x7ff814197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff814104540 0x7ff814197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 -- 172.21.15.42:0/1316492865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804009900 con 0x7ff814101c10 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff812ffd640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814101c10 0x7ff814197120 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d950 tx=0x7ff80c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff80bfff640 1 -- 172.21.15.42:0/1316492865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c009a30 con 0x7ff814101c10 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff80bfff640 1 -- 172.21.15.42:0/1316492865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff80c0044d0 con 0x7ff814101c10 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81419f220 con 0x7ff814101c10 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff80bfff640 1 -- 172.21.15.42:0/1316492865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c0047a0 con 0x7ff814101c10 2024-03-12T04:14:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.014+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81419f730 con 0x7ff814101c10 2024-03-12T04:14:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.015+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8140fd110 con 0x7ff814101c10 2024-03-12T04:14:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.016+0000 7ff80bfff640 1 -- 172.21.15.42:0/1316492865 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff80c009b90 con 0x7ff814101c10 2024-03-12T04:14:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.017+0000 7ff80bfff640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff8000a1a30 0x7ff8000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.017+0000 7ff8127fc640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff8000a1a30 0x7ff8000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.017+0000 7ff80bfff640 1 -- 172.21.15.42:0/1316492865 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff80c0ccf10 con 0x7ff814101c10 2024-03-12T04:14:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.017+0000 7ff8127fc640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff8000a1a30 0x7ff8000a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff814198630 tx=0x7ff7fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.021+0000 7ff80bfff640 1 -- 172.21.15.42:0/1316492865 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff80c096150 con 0x7ff814101c10 2024-03-12T04:14:59.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.202+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff81410ca50 con 0x7ff814101c10 2024-03-12T04:14:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.202+0000 7ff80bfff640 1 -- 172.21.15.42:0/1316492865 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff80c017020 con 0x7ff814101c10 2024-03-12T04:14:59.205 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:14:59.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff8000a1a30 msgr2=0x7ff8000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff8000a1a30 0x7ff8000a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff814198630 tx=0x7ff7fc005f90 comp rx=0 tx=0).stop 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814101c10 msgr2=0x7ff814197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814101c10 0x7ff814197120 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d950 tx=0x7ff80c00de10 comp rx=0 tx=0).stop 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 shutdown_connections 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff814106e70 0x7ff81419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7ff8000a1a30 0x7ff8000a3ef0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff814104540 0x7ff814197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 --2- 172.21.15.42:0/1316492865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff814101c10 0x7ff814197120 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 >> 172.21.15.42:0/1316492865 conn(0x7ff8140fb620 msgr2=0x7ff8141024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 shutdown_connections 2024-03-12T04:14:59.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.204+0000 7ff819676640 1 -- 172.21.15.42:0/1316492865 wait complete. 2024-03-12T04:14:59.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:59 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/68231986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:59.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:59 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1854717383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:59.217 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:14:59 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3333087113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:59.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:14:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 -- 172.21.15.42:0/1735595289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420101270 msgr2=0x7f54201016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 --2- 172.21.15.42:0/1735595289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420101270 0x7f54201016f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5414009900 tx=0x7f541402bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 -- 172.21.15.42:0/1735595289 shutdown_connections 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 --2- 172.21.15.42:0/1735595289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5420101c30 0x7f54201084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 --2- 172.21.15.42:0/1735595289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420101270 0x7f54201016f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 --2- 172.21.15.42:0/1735595289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5420100080 0x7f54201004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 -- 172.21.15.42:0/1735595289 >> 172.21.15.42:0/1735595289 conn(0x7f54200fb610 msgr2=0x7f54200fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 -- 172.21.15.42:0/1735595289 shutdown_connections 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 -- 172.21.15.42:0/1735595289 wait complete. 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 Processor -- start 2024-03-12T04:14:59.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 -- start start 2024-03-12T04:14:59.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.321+0000 7f54258ff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420100080 0x7f5420068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54258ff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5420101270 0x7f542019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54258ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5420101c30 0x7f542019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54258ff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019ec00 con 0x7f5420100080 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54258ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019ed70 con 0x7f5420101c30 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54258ff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019ef30 con 0x7f5420101270 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420100080 0x7f5420068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420100080 0x7f5420068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50058/0 (socket says 172.21.15.42:50058) 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 -- 172.21.15.42:0/2171953958 learned_addr learned my addr 172.21.15.42:0/2171953958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541f7fe640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5420101c30 0x7f542019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541e7fc640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5420101270 0x7f542019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 -- 172.21.15.42:0/2171953958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5420101270 msgr2=0x7f542019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5420101270 0x7f542019be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 -- 172.21.15.42:0/2171953958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5420101c30 msgr2=0x7f542019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5420101c30 0x7f542019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 -- 172.21.15.42:0/2171953958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414009560 con 0x7f5420100080 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f541effd640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420100080 0x7f5420068e70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f541000c970 tx=0x7f541000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54248fd640 1 -- 172.21.15.42:0/2171953958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410007c10 con 0x7f5420100080 2024-03-12T04:14:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542019f210 con 0x7f5420100080 2024-03-12T04:14:59.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54248fd640 1 -- 172.21.15.42:0/2171953958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5410007d70 con 0x7f5420100080 2024-03-12T04:14:59.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54248fd640 1 -- 172.21.15.42:0/2171953958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000f7d0 con 0x7f5420100080 2024-03-12T04:14:59.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.322+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542019f720 con 0x7f5420100080 2024-03-12T04:14:59.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.323+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54201073a0 con 0x7f5420100080 2024-03-12T04:14:59.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.324+0000 7f54248fd640 1 -- 172.21.15.42:0/2171953958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5410018480 con 0x7f5420100080 2024-03-12T04:14:59.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.325+0000 7f54248fd640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f54080a1a30 0x7f54080a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.325+0000 7f541e7fc640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f54080a1a30 0x7f54080a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.325+0000 7f54248fd640 1 -- 172.21.15.42:0/2171953958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5410013070 con 0x7f5420100080 2024-03-12T04:14:59.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.325+0000 7f541e7fc640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f54080a1a30 0x7f54080a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5414036040 tx=0x7f5414005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.329+0000 7f54248fd640 1 -- 172.21.15.42:0/2171953958 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5410096a80 con 0x7f5420100080 2024-03-12T04:14:59.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:59 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/68231986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:59.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:59 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1854717383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:59.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:14:59 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3333087113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:59 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/68231986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:14:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:59 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1854717383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:14:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:14:59 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3333087113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:14:59.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.508+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f542010b940 con 0x7f5420100080 2024-03-12T04:14:59.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.509+0000 7f54248fd640 1 -- 172.21.15.42:0/2171953958 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f541009a140 con 0x7f5420100080 2024-03-12T04:14:59.511 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:14:59.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f54080a1a30 msgr2=0x7f54080a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f54080a1a30 0x7f54080a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5414036040 tx=0x7f5414005fd0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420100080 msgr2=0x7f5420068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420100080 0x7f5420068e70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f541000c970 tx=0x7f541000ce30 comp rx=0 tx=0).stop 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 shutdown_connections 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f54080a1a30 0x7f54080a3ef0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5420101c30 0x7f542019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5420101270 0x7f542019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 --2- 172.21.15.42:0/2171953958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5420100080 0x7f5420068e70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 >> 172.21.15.42:0/2171953958 conn(0x7f54200fb610 msgr2=0x7f5420104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 shutdown_connections 2024-03-12T04:14:59.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.511+0000 7f54258ff640 1 -- 172.21.15.42:0/2171953958 wait complete. 2024-03-12T04:14:59.524 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-12T04:14:59.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 -- 172.21.15.42:0/2349954404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7501000a0 msgr2=0x7fc7501004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 --2- 172.21.15.42:0/2349954404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7501000a0 0x7fc7501004c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc740009900 tx=0x7fc74002bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 -- 172.21.15.42:0/2349954404 shutdown_connections 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 --2- 172.21.15.42:0/2349954404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc750101c50 0x7fc7501084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 --2- 172.21.15.42:0/2349954404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc750101290 0x7fc750101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 --2- 172.21.15.42:0/2349954404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7501000a0 0x7fc7501004c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 -- 172.21.15.42:0/2349954404 >> 172.21.15.42:0/2349954404 conn(0x7fc7500fb610 msgr2=0x7fc7500fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 -- 172.21.15.42:0/2349954404 shutdown_connections 2024-03-12T04:14:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.624+0000 7fc754b9a640 1 -- 172.21.15.42:0/2349954404 wait complete. 2024-03-12T04:14:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 Processor -- start 2024-03-12T04:14:59.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 -- start start 2024-03-12T04:14:59.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7501000a0 0x7fc7501960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc750101290 0x7fc750196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc750101c50 0x7fc75019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75019dbc0 con 0x7fc750101290 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75019dd30 con 0x7fc750101c50 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc754b9a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75019def0 con 0x7fc7501000a0 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc747fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc750101290 0x7fc750196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc747fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc750101290 0x7fc750196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50068/0 (socket says 172.21.15.42:50068) 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc747fff640 1 -- 172.21.15.42:0/668385630 learned_addr learned my addr 172.21.15.42:0/668385630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc74e59c640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7501000a0 0x7fc7501960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.625+0000 7fc74ed9d640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc750101c50 0x7fc75019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc747fff640 1 -- 172.21.15.42:0/668385630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7501000a0 msgr2=0x7fc7501960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc747fff640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7501000a0 0x7fc7501960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc747fff640 1 -- 172.21.15.42:0/668385630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc750101c50 msgr2=0x7fc75019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc747fff640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc750101c50 0x7fc75019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc747fff640 1 -- 172.21.15.42:0/668385630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc740009560 con 0x7fc750101290 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc74ed9d640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc750101c50 0x7fc75019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc74e59c640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7501000a0 0x7fc7501960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc747fff640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc750101290 0x7fc750196610 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc73800e9f0 tx=0x7fc73800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc7477fe640 1 -- 172.21.15.42:0/668385630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738009700 con 0x7fc750101290 2024-03-12T04:14:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75019e1d0 con 0x7fc750101290 2024-03-12T04:14:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc7477fe640 1 -- 172.21.15.42:0/668385630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc73800cd80 con 0x7fc750101290 2024-03-12T04:14:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc7477fe640 1 -- 172.21.15.42:0/668385630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738018840 con 0x7fc750101290 2024-03-12T04:14:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.626+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75019e730 con 0x7fc750101290 2024-03-12T04:14:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.627+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc750107490 con 0x7fc750101290 2024-03-12T04:14:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.628+0000 7fc7477fe640 1 -- 172.21.15.42:0/668385630 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc7380189a0 con 0x7fc750101290 2024-03-12T04:14:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.629+0000 7fc7477fe640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc73c0a1a30 0x7fc73c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.629+0000 7fc74e59c640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc73c0a1a30 0x7fc73c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.629+0000 7fc7477fe640 1 -- 172.21.15.42:0/668385630 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc738014070 con 0x7fc750101290 2024-03-12T04:14:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.629+0000 7fc74e59c640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc73c0a1a30 0x7fc73c0a3ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc740006010 tx=0x7fc740005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.633+0000 7fc7477fe640 1 -- 172.21.15.42:0/668385630 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc738052d20 con 0x7fc750101290 2024-03-12T04:14:59.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.851+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc75002cda0 con 0x7fc750101290 2024-03-12T04:14:59.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.852+0000 7fc7477fe640 1 -- 172.21.15.42:0/668385630 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fc738096aa0 con 0x7fc750101290 2024-03-12T04:14:59.854 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:14:59.855 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:14:59.855 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:14:59.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc73c0a1a30 msgr2=0x7fc73c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc73c0a1a30 0x7fc73c0a3ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc740006010 tx=0x7fc740005c00 comp rx=0 tx=0).stop 2024-03-12T04:14:59.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc750101290 msgr2=0x7fc750196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc750101290 0x7fc750196610 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc73800e9f0 tx=0x7fc73800eeb0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 shutdown_connections 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc73c0a1a30 0x7fc73c0a3ef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc750101c50 0x7fc75019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc750101290 0x7fc750196610 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 --2- 172.21.15.42:0/668385630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7501000a0 0x7fc7501960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 >> 172.21.15.42:0/668385630 conn(0x7fc7500fb610 msgr2=0x7fc750104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 shutdown_connections 2024-03-12T04:14:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.854+0000 7fc754b9a640 1 -- 172.21.15.42:0/668385630 wait complete. 2024-03-12T04:14:59.867 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-12T04:14:59.867 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-12T04:14:59.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T04:14:59.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 -- 172.21.15.42:0/1224853391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe688101290 msgr2=0x7fe688101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/1224853391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe688101290 0x7fe688101710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009900 tx=0x7fe67c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:14:59.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 -- 172.21.15.42:0/1224853391 shutdown_connections 2024-03-12T04:14:59.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/1224853391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688101c50 0x7fe6881084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/1224853391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe688101290 0x7fe688101710 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/1224853391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6881000a0 0x7fe6881004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 -- 172.21.15.42:0/1224853391 >> 172.21.15.42:0/1224853391 conn(0x7fe6880fb610 msgr2=0x7fe6880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 -- 172.21.15.42:0/1224853391 shutdown_connections 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.967+0000 7fe68e8dd640 1 -- 172.21.15.42:0/1224853391 wait complete. 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 Processor -- start 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 -- start start 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6881000a0 0x7fe688197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688101290 0x7fe6881976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe688101c50 0x7fe68819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68819ec70 con 0x7fe6881000a0 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68819ede0 con 0x7fe688101290 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe68e8dd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68819efa0 con 0x7fe688101c50 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe687fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6881000a0 0x7fe688197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe687fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6881000a0 0x7fe688197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50084/0 (socket says 172.21.15.42:50084) 2024-03-12T04:14:59.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.968+0000 7fe687fff640 1 -- 172.21.15.42:0/2894348649 learned_addr learned my addr 172.21.15.42:0/2894348649 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe6877fe640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688101290 0x7fe6881976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe68ce7a640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe688101c50 0x7fe68819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe687fff640 1 -- 172.21.15.42:0/2894348649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe688101c50 msgr2=0x7fe68819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe687fff640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe688101c50 0x7fe68819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe687fff640 1 -- 172.21.15.42:0/2894348649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688101290 msgr2=0x7fe6881976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe687fff640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688101290 0x7fe6881976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe687fff640 1 -- 172.21.15.42:0/2894348649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c009560 con 0x7fe6881000a0 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe68ce7a640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe688101c50 0x7fe68819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe687fff640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6881000a0 0x7fe688197180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe67800d950 tx=0x7fe67800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe6857fa640 1 -- 172.21.15.42:0/2894348649 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678009c30 con 0x7fe6881000a0 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe6857fa640 1 -- 172.21.15.42:0/2894348649 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe678009d90 con 0x7fe6881000a0 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68819f280 con 0x7fe6881000a0 2024-03-12T04:14:59.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe6857fa640 1 -- 172.21.15.42:0/2894348649 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800f7e0 con 0x7fe6881000a0 2024-03-12T04:14:59.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.969+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68819f790 con 0x7fe6881000a0 2024-03-12T04:14:59.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.970+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6881073f0 con 0x7fe6881000a0 2024-03-12T04:14:59.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.971+0000 7fe6857fa640 1 -- 172.21.15.42:0/2894348649 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe678010460 con 0x7fe6881000a0 2024-03-12T04:14:59.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.972+0000 7fe6857fa640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe6700a1a30 0x7fe6700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:14:59.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.972+0000 7fe6857fa640 1 -- 172.21.15.42:0/2894348649 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe6780cd150 con 0x7fe6881000a0 2024-03-12T04:14:59.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.972+0000 7fe6877fe640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe6700a1a30 0x7fe6700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:14:59.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.972+0000 7fe6877fe640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe6700a1a30 0x7fe6700a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe688198690 tx=0x7fe67c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:14:59.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:14:59.976+0000 7fe6857fa640 1 -- 172.21.15.42:0/2894348649 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe678096390 con 0x7fe6881000a0 2024-03-12T04:15:00.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.154+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe68802ceb0 con 0x7fe6700a1a30 2024-03-12T04:15:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.162+0000 7fe6857fa640 1 -- 172.21.15.42:0/2894348649 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259100 (secure 0 0 0) 0x7fe68802ceb0 con 0x7fe6700a1a30 2024-03-12T04:15:00.168 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:15:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe6700a1a30 msgr2=0x7fe6700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe6700a1a30 0x7fe6700a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe688198690 tx=0x7fe67c000bc0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6881000a0 msgr2=0x7fe688197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6881000a0 0x7fe688197180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe67800d950 tx=0x7fe67800de10 comp rx=0 tx=0).stop 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 shutdown_connections 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe688101c50 0x7fe68819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fe6700a1a30 0x7fe6700a3ef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688101290 0x7fe6881976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 --2- 172.21.15.42:0/2894348649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6881000a0 0x7fe688197180 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 >> 172.21.15.42:0/2894348649 conn(0x7fe6880fb610 msgr2=0x7fe6881045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 shutdown_connections 2024-03-12T04:15:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.165+0000 7fe68e8dd640 1 -- 172.21.15.42:0/2894348649 wait complete. 2024-03-12T04:15:00.177 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:00 smithi042 ceph-mon[32260]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-12T04:15:00.177 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:00 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1316492865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:15:00.177 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:00 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2171953958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:15:00.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:00 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/668385630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:00.179 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-03-12T04:14:58.930090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1344,"num_read_kb":800,"num_write":440,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373620,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568204,"statfs":{"total":1151940427776,"available":1151557840896,"internally_reserved":0,"allocated":12275712,"data_stored":7576844,"data_compressed":114537,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":67,"num_write":11,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003272"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949806+0000","last_change":"2024-03-12T04:12:54.788665+0000","last_active":"2024-03-12T04:14:14.949806+0000","last_peered":"2024-03-12T04:14:14.949806+0000","last_clean":"2024-03-12T04:14:14.949806+0000","last_became_active":"2024-03-12T04:12:54.788334+0000","last_became_peered":"2024-03-12T04:12:54.788334+0000","last_unstale":"2024-03-12T04:14:14.949806+0000","last_undegraded":"2024-03-12T04:14:14.949806+0000","last_fullsized":"2024-03-12T04:14:14.949806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:02.442543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940407+0000","last_change":"2024-03-12T04:12:54.791942+0000","last_active":"2024-03-12T04:14:14.940407+0000","last_peered":"2024-03-12T04:14:14.940407+0000","last_clean":"2024-03-12T04:14:14.940407+0000","last_became_active":"2024-03-12T04:12:54.791579+0000","last_became_peered":"2024-03-12T04:12:54.791579+0000","last_unstale":"2024-03-12T04:14:14.940407+0000","last_undegraded":"2024-03-12T04:14:14.940407+0000","last_fullsized":"2024-03-12T04:14:14.940407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:51:08.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949288+0000","last_change":"2024-03-12T04:12:54.793238+0000","last_active":"2024-03-12T04:14:14.949288+0000","last_peered":"2024-03-12T04:14:14.949288+0000","last_clean":"2024-03-12T04:14:14.949288+0000","last_became_active":"2024-03-12T04:12:54.792877+0000","last_became_peered":"2024-03-12T04:12:54.792877+0000","last_unstale":"2024-03-12T04:14:14.949288+0000","last_undegraded":"2024-03-12T04:14:14.949288+0000","last_fullsized":"2024-03-12T04:14:14.949288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:36:46.914204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945072+0000","last_change":"2024-03-12T04:12:54.797573+0000","last_active":"2024-03-12T04:14:14.945072+0000","last_peered":"2024-03-12T04:14:14.945072+0000","last_clean":"2024-03-12T04:14:14.945072+0000","last_became_active":"2024-03-12T04:12:54.796939+0000","last_became_peered":"2024-03-12T04:12:54.796939+0000","last_unstale":"2024-03-12T04:14:14.945072+0000","last_undegraded":"2024-03-12T04:14:14.945072+0000","last_fullsized":"2024-03-12T04:14:14.945072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:34:58.047492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.912919+0000","last_change":"2024-03-12T04:12:54.780595+0000","last_active":"2024-03-12T04:14:14.912919+0000","last_peered":"2024-03-12T04:14:14.912919+0000","last_clean":"2024-03-12T04:14:14.912919+0000","last_became_active":"2024-03-12T04:12:54.780256+0000","last_became_peered":"2024-03-12T04:12:54.780256+0000","last_unstale":"2024-03-12T04:14:14.912919+0000","last_undegraded":"2024-03-12T04:14:14.912919+0000","last_fullsized":"2024-03-12T04:14:14.912919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:42.492006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935417+0000","last_change":"2024-03-12T04:13:45.934954+0000","last_active":"2024-03-12T04:14:14.935417+0000","last_peered":"2024-03-12T04:14:14.935417+0000","last_clean":"2024-03-12T04:14:14.935417+0000","last_became_active":"2024-03-12T04:13:45.934579+0000","last_became_peered":"2024-03-12T04:13:45.934579+0000","last_unstale":"2024-03-12T04:14:14.935417+0000","last_undegraded":"2024-03-12T04:14:14.935417+0000","last_fullsized":"2024-03-12T04:14:14.935417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:39:10.062304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.941999+0000","last_change":"2024-03-12T04:12:54.781510+0000","last_active":"2024-03-12T04:14:14.941999+0000","last_peered":"2024-03-12T04:14:14.941999+0000","last_clean":"2024-03-12T04:14:14.941999+0000","last_became_active":"2024-03-12T04:12:54.781141+0000","last_became_peered":"2024-03-12T04:12:54.781141+0000","last_unstale":"2024-03-12T04:14:14.941999+0000","last_undegraded":"2024-03-12T04:14:14.941999+0000","last_fullsized":"2024-03-12T04:14:14.941999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:42:05.055308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948851+0000","last_change":"2024-03-12T04:12:54.782541+0000","last_active":"2024-03-12T04:14:14.948851+0000","last_peered":"2024-03-12T04:14:14.948851+0000","last_clean":"2024-03-12T04:14:14.948851+0000","last_became_active":"2024-03-12T04:12:54.781582+0000","last_became_peered":"2024-03-12T04:12:54.781582+0000","last_unstale":"2024-03-12T04:14:14.948851+0000","last_undegraded":"2024-03-12T04:14:14.948851+0000","last_fullsized":"2024-03-12T04:14:14.948851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:15.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953459+0000","last_change":"2024-03-12T04:12:54.788201+0000","last_active":"2024-03-12T04:14:14.953459+0000","last_peered":"2024-03-12T04:14:14.953459+0000","last_clean":"2024-03-12T04:14:14.953459+0000","last_became_active":"2024-03-12T04:12:54.787875+0000","last_became_peered":"2024-03-12T04:12:54.787875+0000","last_unstale":"2024-03-12T04:14:14.953459+0000","last_undegraded":"2024-03-12T04:14:14.953459+0000","last_fullsized":"2024-03-12T04:14:14.953459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:40:39.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944262+0000","last_change":"2024-03-12T04:12:54.789748+0000","last_active":"2024-03-12T04:14:14.944262+0000","last_peered":"2024-03-12T04:14:14.944262+0000","last_clean":"2024-03-12T04:14:14.944262+0000","last_became_active":"2024-03-12T04:12:54.788838+0000","last_became_peered":"2024-03-12T04:12:54.788838+0000","last_unstale":"2024-03-12T04:14:14.944262+0000","last_undegraded":"2024-03-12T04:14:14.944262+0000","last_fullsized":"2024-03-12T04:14:14.944262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:39:51.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949656+0000","last_change":"2024-03-12T04:12:54.793101+0000","last_active":"2024-03-12T04:14:14.949656+0000","last_peered":"2024-03-12T04:14:14.949656+0000","last_clean":"2024-03-12T04:14:14.949656+0000","last_became_active":"2024-03-12T04:12:54.792488+0000","last_became_peered":"2024-03-12T04:12:54.792488+0000","last_unstale":"2024-03-12T04:14:14.949656+0000","last_undegraded":"2024-03-12T04:14:14.949656+0000","last_fullsized":"2024-03-12T04:14:14.949656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:33.862011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.942330+0000","last_change":"2024-03-12T04:12:54.797953+0000","last_active":"2024-03-12T04:14:14.942330+0000","last_peered":"2024-03-12T04:14:14.942330+0000","last_clean":"2024-03-12T04:14:14.942330+0000","last_became_active":"2024-03-12T04:12:54.797413+0000","last_became_peered":"2024-03-12T04:12:54.797413+0000","last_unstale":"2024-03-12T04:14:14.942330+0000","last_undegraded":"2024-03-12T04:14:14.942330+0000","last_fullsized":"2024-03-12T04:14:14.942330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:18:01.140872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.936082+0000","last_change":"2024-03-12T04:12:54.775570+0000","last_active":"2024-03-12T04:14:14.936082+0000","last_peered":"2024-03-12T04:14:14.936082+0000","last_clean":"2024-03-12T04:14:14.936082+0000","last_became_active":"2024-03-12T04:12:54.775186+0000","last_became_peered":"2024-03-12T04:12:54.775186+0000","last_unstale":"2024-03-12T04:14:14.936082+0000","last_undegraded":"2024-03-12T04:14:14.936082+0000","last_fullsized":"2024-03-12T04:14:14.936082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:26.004458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.934601+0000","last_change":"2024-03-12T04:12:54.780859+0000","last_active":"2024-03-12T04:14:14.934601+0000","last_peered":"2024-03-12T04:14:14.934601+0000","last_clean":"2024-03-12T04:14:14.934601+0000","last_became_active":"2024-03-12T04:12:54.780484+0000","last_became_peered":"2024-03-12T04:12:54.780484+0000","last_unstale":"2024-03-12T04:14:14.934601+0000","last_undegraded":"2024-03-12T04:14:14.934601+0000","last_fullsized":"2024-03-12T04:14:14.934601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:24:00.575324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.926041+0000","last_change":"2024-03-12T04:12:54.796087+0000","last_active":"2024-03-12T04:14:14.926041+0000","last_peered":"2024-03-12T04:14:14.926041+0000","last_clean":"2024-03-12T04:14:14.926041+0000","last_became_active":"2024-03-12T04:12:54.793702+0000","last_became_peered":"2024-03-12T04:12:54.793702+0000","last_unstale":"2024-03-12T04:14:14.926041+0000","last_undegraded":"2024-03-12T04:14:14.926041+0000","last_fullsized":"2024-03-12T04:14:14.926041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:09:27.334791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953800+0000","last_change":"2024-03-12T04:12:54.783146+0000","last_active":"2024-03-12T04:14:14.953800+0000","last_peered":"2024-03-12T04:14:14.953800+0000","last_clean":"2024-03-12T04:14:14.953800+0000","last_became_active":"2024-03-12T04:12:54.782834+0000","last_became_peered":"2024-03-12T04:12:54.782834+0000","last_unstale":"2024-03-12T04:14:14.953800+0000","last_undegraded":"2024-03-12T04:14:14.953800+0000","last_fullsized":"2024-03-12T04:14:14.953800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:35:11.126629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.950186+0000","last_change":"2024-03-12T04:12:54.794879+0000","last_active":"2024-03-12T04:14:14.950186+0000","last_peered":"2024-03-12T04:14:14.950186+0000","last_clean":"2024-03-12T04:14:14.950186+0000","last_became_active":"2024-03-12T04:12:54.794251+0000","last_became_peered":"2024-03-12T04:12:54.794251+0000","last_unstale":"2024-03-12T04:14:14.950186+0000","last_undegraded":"2024-03-12T04:14:14.950186+0000","last_fullsized":"2024-03-12T04:14:14.950186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:33.754426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946417+0000","last_change":"2024-03-12T04:12:54.791301+0000","last_active":"2024-03-12T04:14:14.946417+0000","last_peered":"2024-03-12T04:14:14.946417+0000","last_clean":"2024-03-12T04:14:14.946417+0000","last_became_active":"2024-03-12T04:12:54.790850+0000","last_became_peered":"2024-03-12T04:12:54.790850+0000","last_unstale":"2024-03-12T04:14:14.946417+0000","last_undegraded":"2024-03-12T04:14:14.946417+0000","last_fullsized":"2024-03-12T04:14:14.946417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:26:04.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949051+0000","last_change":"2024-03-12T04:12:54.793419+0000","last_active":"2024-03-12T04:14:14.949051+0000","last_peered":"2024-03-12T04:14:14.949051+0000","last_clean":"2024-03-12T04:14:14.949051+0000","last_became_active":"2024-03-12T04:12:54.793080+0000","last_became_peered":"2024-03-12T04:12:54.793080+0000","last_unstale":"2024-03-12T04:14:14.949051+0000","last_undegraded":"2024-03-12T04:14:14.949051+0000","last_fullsized":"2024-03-12T04:14:14.949051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:46:26.506899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:53.185317+0000","last_change":"2024-03-12T04:12:53.774893+0000","last_active":"2024-03-12T04:14:53.185317+0000","last_peered":"2024-03-12T04:14:53.185317+0000","last_clean":"2024-03-12T04:14:53.185317+0000","last_became_active":"2024-03-12T04:12:53.773801+0000","last_became_peered":"2024-03-12T04:12:53.773801+0000","last_unstale":"2024-03-12T04:14:53.185317+0000","last_undegraded":"2024-03-12T04:14:53.185317+0000","last_fullsized":"2024-03-12T04:14:53.185317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:30:19.995765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948615+0000","last_change":"2024-03-12T04:12:54.786795+0000","last_active":"2024-03-12T04:14:14.948615+0000","last_peered":"2024-03-12T04:14:14.948615+0000","last_clean":"2024-03-12T04:14:14.948615+0000","last_became_active":"2024-03-12T04:12:54.786481+0000","last_became_peered":"2024-03-12T04:12:54.786481+0000","last_unstale":"2024-03-12T04:14:14.948615+0000","last_undegraded":"2024-03-12T04:14:14.948615+0000","last_fullsized":"2024-03-12T04:14:14.948615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:06.849644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936225+0000","last_change":"2024-03-12T04:12:53.779582+0000","last_active":"2024-03-12T04:13:45.936225+0000","last_peered":"2024-03-12T04:13:45.936225+0000","last_clean":"2024-03-12T04:13:45.936225+0000","last_became_active":"2024-03-12T04:12:53.779062+0000","last_became_peered":"2024-03-12T04:12:53.779062+0000","last_unstale":"2024-03-12T04:13:45.936225+0000","last_undegraded":"2024-03-12T04:13:45.936225+0000","last_fullsized":"2024-03-12T04:13:45.936225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:44:01.065183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949313+0000","last_change":"2024-03-12T04:12:54.789198+0000","last_active":"2024-03-12T04:14:14.949313+0000","last_peered":"2024-03-12T04:14:14.949313+0000","last_clean":"2024-03-12T04:14:14.949313+0000","last_became_active":"2024-03-12T04:12:54.788213+0000","last_became_peered":"2024-03-12T04:12:54.788213+0000","last_unstale":"2024-03-12T04:14:14.949313+0000","last_undegraded":"2024-03-12T04:14:14.949313+0000","last_fullsized":"2024-03-12T04:14:14.949313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:24:36.561553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937326+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:45.937326+0000","last_peered":"2024-03-12T04:13:45.937326+0000","last_clean":"2024-03-12T04:13:45.937326+0000","last_became_active":"2024-03-12T04:12:53.771178+0000","last_became_peered":"2024-03-12T04:12:53.771178+0000","last_unstale":"2024-03-12T04:13:45.937326+0000","last_undegraded":"2024-03-12T04:13:45.937326+0000","last_fullsized":"2024-03-12T04:13:45.937326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:12.882128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953255+0000","last_change":"2024-03-12T04:12:54.799758+0000","last_active":"2024-03-12T04:14:14.953255+0000","last_peered":"2024-03-12T04:14:14.953255+0000","last_clean":"2024-03-12T04:14:14.953255+0000","last_became_active":"2024-03-12T04:12:54.799338+0000","last_became_peered":"2024-03-12T04:12:54.799338+0000","last_unstale":"2024-03-12T04:14:14.953255+0000","last_undegraded":"2024-03-12T04:14:14.953255+0000","last_fullsized":"2024-03-12T04:14:14.953255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:32:54.087890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937112+0000","last_change":"2024-03-12T04:12:53.774390+0000","last_active":"2024-03-12T04:13:45.937112+0000","last_peered":"2024-03-12T04:13:45.937112+0000","last_clean":"2024-03-12T04:13:45.937112+0000","last_became_active":"2024-03-12T04:12:53.774042+0000","last_became_peered":"2024-03-12T04:12:53.774042+0000","last_unstale":"2024-03-12T04:13:45.937112+0000","last_undegraded":"2024-03-12T04:13:45.937112+0000","last_fullsized":"2024-03-12T04:13:45.937112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:37.361505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953436+0000","last_change":"2024-03-12T04:13:46.104074+0000","last_active":"2024-03-12T04:14:14.953436+0000","last_peered":"2024-03-12T04:14:14.953436+0000","last_clean":"2024-03-12T04:14:14.953436+0000","last_became_active":"2024-03-12T04:13:46.103719+0000","last_became_peered":"2024-03-12T04:13:46.103719+0000","last_unstale":"2024-03-12T04:14:14.953436+0000","last_undegraded":"2024-03-12T04:14:14.953436+0000","last_fullsized":"2024-03-12T04:14:14.953436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:29.979576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.198832+0000","last_change":"2024-03-12T04:12:53.772182+0000","last_active":"2024-03-12T04:14:54.198832+0000","last_peered":"2024-03-12T04:14:54.198832+0000","last_clean":"2024-03-12T04:14:54.198832+0000","last_became_active":"2024-03-12T04:12:53.771858+0000","last_became_peered":"2024-03-12T04:12:53.771858+0000","last_unstale":"2024-03-12T04:14:54.198832+0000","last_undegraded":"2024-03-12T04:14:54.198832+0000","last_fullsized":"2024-03-12T04:14:54.198832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:42:41.466311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.922543+0000","last_change":"2024-03-12T04:12:54.779869+0000","last_active":"2024-03-12T04:14:14.922543+0000","last_peered":"2024-03-12T04:14:14.922543+0000","last_clean":"2024-03-12T04:14:14.922543+0000","last_became_active":"2024-03-12T04:12:54.779500+0000","last_became_peered":"2024-03-12T04:12:54.779500+0000","last_unstale":"2024-03-12T04:14:14.922543+0000","last_undegraded":"2024-03-12T04:14:14.922543+0000","last_fullsized":"2024-03-12T04:14:14.922543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:29.538570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932326+0000","last_change":"2024-03-12T04:12:53.774056+0000","last_active":"2024-03-12T04:13:45.932326+0000","last_peered":"2024-03-12T04:13:45.932326+0000","last_clean":"2024-03-12T04:13:45.932326+0000","last_became_active":"2024-03-12T04:12:53.773675+0000","last_became_peered":"2024-03-12T04:12:53.773675+0000","last_unstale":"2024-03-12T04:13:45.932326+0000","last_undegraded":"2024-03-12T04:13:45.932326+0000","last_fullsized":"2024-03-12T04:13:45.932326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:33.576947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945472+0000","last_change":"2024-03-12T04:12:54.790583+0000","last_active":"2024-03-12T04:14:14.945472+0000","last_peered":"2024-03-12T04:14:14.945472+0000","last_clean":"2024-03-12T04:14:14.945472+0000","last_became_active":"2024-03-12T04:12:54.788364+0000","last_became_peered":"2024-03-12T04:12:54.788364+0000","last_unstale":"2024-03-12T04:14:14.945472+0000","last_undegraded":"2024-03-12T04:14:14.945472+0000","last_fullsized":"2024-03-12T04:14:14.945472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:05:13.542279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933518+0000","last_change":"2024-03-12T04:12:53.769619+0000","last_active":"2024-03-12T04:13:45.933518+0000","last_peered":"2024-03-12T04:13:45.933518+0000","last_clean":"2024-03-12T04:13:45.933518+0000","last_became_active":"2024-03-12T04:12:53.769293+0000","last_became_peered":"2024-03-12T04:12:53.769293+0000","last_unstale":"2024-03-12T04:13:45.933518+0000","last_undegraded":"2024-03-12T04:13:45.933518+0000","last_fullsized":"2024-03-12T04:13:45.933518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:44.618499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940327+0000","last_change":"2024-03-12T04:12:54.776493+0000","last_active":"2024-03-12T04:14:14.940327+0000","last_peered":"2024-03-12T04:14:14.940327+0000","last_clean":"2024-03-12T04:14:14.940327+0000","last_became_active":"2024-03-12T04:12:54.775967+0000","last_became_peered":"2024-03-12T04:12:54.775967+0000","last_unstale":"2024-03-12T04:14:14.940327+0000","last_undegraded":"2024-03-12T04:14:14.940327+0000","last_fullsized":"2024-03-12T04:14:14.940327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:19:46.824145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934388+0000","last_change":"2024-03-12T04:12:53.773523+0000","last_active":"2024-03-12T04:13:45.934388+0000","last_peered":"2024-03-12T04:13:45.934388+0000","last_clean":"2024-03-12T04:13:45.934388+0000","last_became_active":"2024-03-12T04:12:53.773161+0000","last_became_peered":"2024-03-12T04:12:53.773161+0000","last_unstale":"2024-03-12T04:13:45.934388+0000","last_undegraded":"2024-03-12T04:13:45.934388+0000","last_fullsized":"2024-03-12T04:13:45.934388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:41:40.810994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939744+0000","last_change":"2024-03-12T04:12:54.791688+0000","last_active":"2024-03-12T04:14:14.939744+0000","last_peered":"2024-03-12T04:14:14.939744+0000","last_clean":"2024-03-12T04:14:14.939744+0000","last_became_active":"2024-03-12T04:12:54.790854+0000","last_became_peered":"2024-03-12T04:12:54.790854+0000","last_unstale":"2024-03-12T04:14:14.939744+0000","last_undegraded":"2024-03-12T04:14:14.939744+0000","last_fullsized":"2024-03-12T04:14:14.939744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:42:50.343524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":264,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:58.205524+0000","last_change":"2024-03-12T04:12:53.777534+0000","last_active":"2024-03-12T04:14:58.205524+0000","last_peered":"2024-03-12T04:14:58.205524+0000","last_clean":"2024-03-12T04:14:58.205524+0000","last_became_active":"2024-03-12T04:12:53.777201+0000","last_became_peered":"2024-03-12T04:12:53.777201+0000","last_unstale":"2024-03-12T04:14:58.205524+0000","last_undegraded":"2024-03-12T04:14:58.205524+0000","last_fullsized":"2024-03-12T04:14:58.205524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:54.708892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953198+0000","last_change":"2024-03-12T04:12:54.798962+0000","last_active":"2024-03-12T04:14:14.953198+0000","last_peered":"2024-03-12T04:14:14.953198+0000","last_clean":"2024-03-12T04:14:14.953198+0000","last_became_active":"2024-03-12T04:12:54.798650+0000","last_became_peered":"2024-03-12T04:12:54.798650+0000","last_unstale":"2024-03-12T04:14:14.953198+0000","last_undegraded":"2024-03-12T04:14:14.953198+0000","last_fullsized":"2024-03-12T04:14:14.953198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:05:29.270242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100119+0000","last_change":"2024-03-12T04:12:53.777791+0000","last_active":"2024-03-12T04:13:46.100119+0000","last_peered":"2024-03-12T04:13:46.100119+0000","last_clean":"2024-03-12T04:13:46.100119+0000","last_became_active":"2024-03-12T04:12:53.777430+0000","last_became_peered":"2024-03-12T04:12:53.777430+0000","last_unstale":"2024-03-12T04:13:46.100119+0000","last_undegraded":"2024-03-12T04:13:46.100119+0000","last_fullsized":"2024-03-12T04:13:46.100119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:38:21.858857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.921622+0000","last_change":"2024-03-12T04:12:54.784710+0000","last_active":"2024-03-12T04:14:14.921622+0000","last_peered":"2024-03-12T04:14:14.921622+0000","last_clean":"2024-03-12T04:14:14.921622+0000","last_became_active":"2024-03-12T04:12:54.784342+0000","last_became_peered":"2024-03-12T04:12:54.784342+0000","last_unstale":"2024-03-12T04:14:14.921622+0000","last_undegraded":"2024-03-12T04:14:14.921622+0000","last_fullsized":"2024-03-12T04:14:14.921622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:21.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935890+0000","last_change":"2024-03-12T04:12:53.779289+0000","last_active":"2024-03-12T04:13:45.935890+0000","last_peered":"2024-03-12T04:13:45.935890+0000","last_clean":"2024-03-12T04:13:45.935890+0000","last_became_active":"2024-03-12T04:12:53.778894+0000","last_became_peered":"2024-03-12T04:12:53.778894+0000","last_unstale":"2024-03-12T04:13:45.935890+0000","last_undegraded":"2024-03-12T04:13:45.935890+0000","last_fullsized":"2024-03-12T04:13:45.935890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:13:33.328977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.952684+0000","last_change":"2024-03-12T04:12:54.794102+0000","last_active":"2024-03-12T04:14:14.952684+0000","last_peered":"2024-03-12T04:14:14.952684+0000","last_clean":"2024-03-12T04:14:14.952684+0000","last_became_active":"2024-03-12T04:12:54.793789+0000","last_became_peered":"2024-03-12T04:12:54.793789+0000","last_unstale":"2024-03-12T04:14:14.952684+0000","last_undegraded":"2024-03-12T04:14:14.952684+0000","last_fullsized":"2024-03-12T04:14:14.952684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:55:33.632297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:58.221649+0000","last_change":"2024-03-12T04:12:53.774847+0000","last_active":"2024-03-12T04:14:58.221649+0000","last_peered":"2024-03-12T04:14:58.221649+0000","last_clean":"2024-03-12T04:14:58.221649+0000","last_became_active":"2024-03-12T04:12:53.774457+0000","last_became_peered":"2024-03-12T04:12:53.774457+0000","last_unstale":"2024-03-12T04:14:58.221649+0000","last_undegraded":"2024-03-12T04:14:58.221649+0000","last_fullsized":"2024-03-12T04:14:58.221649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:17.350636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59508,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":181,"num_write":37,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.930949+0000","last_change":"2024-03-12T04:12:54.783671+0000","last_active":"2024-03-12T04:14:14.930949+0000","last_peered":"2024-03-12T04:14:14.930949+0000","last_clean":"2024-03-12T04:14:14.930949+0000","last_became_active":"2024-03-12T04:12:54.782919+0000","last_became_peered":"2024-03-12T04:12:54.782919+0000","last_unstale":"2024-03-12T04:14:14.930949+0000","last_undegraded":"2024-03-12T04:14:14.930949+0000","last_fullsized":"2024-03-12T04:14:14.930949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:38:19.445096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.948316+0000","last_change":"2024-03-12T04:13:45.947951+0000","last_active":"2024-03-12T04:13:45.948316+0000","last_peered":"2024-03-12T04:13:45.948316+0000","last_clean":"2024-03-12T04:13:45.948316+0000","last_became_active":"2024-03-12T04:13:45.938099+0000","last_became_peered":"2024-03-12T04:13:45.938099+0000","last_unstale":"2024-03-12T04:13:45.948316+0000","last_undegraded":"2024-03-12T04:13:45.948316+0000","last_fullsized":"2024-03-12T04:13:45.948316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:03.905076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.943052+0000","last_change":"2024-03-12T04:12:54.779319+0000","last_active":"2024-03-12T04:14:14.943052+0000","last_peered":"2024-03-12T04:14:14.943052+0000","last_clean":"2024-03-12T04:14:14.943052+0000","last_became_active":"2024-03-12T04:12:54.778927+0000","last_became_peered":"2024-03-12T04:12:54.778927+0000","last_unstale":"2024-03-12T04:14:14.943052+0000","last_undegraded":"2024-03-12T04:14:14.943052+0000","last_fullsized":"2024-03-12T04:14:14.943052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:20:53.947257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934971+0000","last_change":"2024-03-12T04:12:53.774754+0000","last_active":"2024-03-12T04:13:45.934971+0000","last_peered":"2024-03-12T04:13:45.934971+0000","last_clean":"2024-03-12T04:13:45.934971+0000","last_became_active":"2024-03-12T04:12:53.774395+0000","last_became_peered":"2024-03-12T04:12:53.774395+0000","last_unstale":"2024-03-12T04:13:45.934971+0000","last_undegraded":"2024-03-12T04:13:45.934971+0000","last_fullsized":"2024-03-12T04:13:45.934971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:13.286204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946653+0000","last_change":"2024-03-12T04:12:54.786891+0000","last_active":"2024-03-12T04:14:14.946653+0000","last_peered":"2024-03-12T04:14:14.946653+0000","last_clean":"2024-03-12T04:14:14.946653+0000","last_became_active":"2024-03-12T04:12:54.786591+0000","last_became_peered":"2024-03-12T04:12:54.786591+0000","last_unstale":"2024-03-12T04:14:14.946653+0000","last_undegraded":"2024-03-12T04:14:14.946653+0000","last_fullsized":"2024-03-12T04:14:14.946653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:16:09.188557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.187545+0000","last_change":"2024-03-12T04:12:53.769839+0000","last_active":"2024-03-12T04:14:54.187545+0000","last_peered":"2024-03-12T04:14:54.187545+0000","last_clean":"2024-03-12T04:14:54.187545+0000","last_became_active":"2024-03-12T04:12:53.769486+0000","last_became_peered":"2024-03-12T04:12:53.769486+0000","last_unstale":"2024-03-12T04:14:54.187545+0000","last_undegraded":"2024-03-12T04:14:54.187545+0000","last_fullsized":"2024-03-12T04:14:54.187545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:46.050018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945596+0000","last_change":"2024-03-12T04:12:54.782447+0000","last_active":"2024-03-12T04:14:14.945596+0000","last_peered":"2024-03-12T04:14:14.945596+0000","last_clean":"2024-03-12T04:14:14.945596+0000","last_became_active":"2024-03-12T04:12:54.782091+0000","last_became_peered":"2024-03-12T04:12:54.782091+0000","last_unstale":"2024-03-12T04:14:14.945596+0000","last_undegraded":"2024-03-12T04:14:14.945596+0000","last_fullsized":"2024-03-12T04:14:14.945596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:22.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.959032+0000","last_change":"2024-03-12T04:13:45.958615+0000","last_active":"2024-03-12T04:13:45.959032+0000","last_peered":"2024-03-12T04:13:45.959032+0000","last_clean":"2024-03-12T04:13:45.959032+0000","last_became_active":"2024-03-12T04:13:45.938341+0000","last_became_peered":"2024-03-12T04:13:45.938341+0000","last_unstale":"2024-03-12T04:13:45.959032+0000","last_undegraded":"2024-03-12T04:13:45.959032+0000","last_fullsized":"2024-03-12T04:13:45.959032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:17:44.688344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.951853+0000","last_change":"2024-03-12T04:12:54.784282+0000","last_active":"2024-03-12T04:14:14.951853+0000","last_peered":"2024-03-12T04:14:14.951853+0000","last_clean":"2024-03-12T04:14:14.951853+0000","last_became_active":"2024-03-12T04:12:54.783953+0000","last_became_peered":"2024-03-12T04:12:54.783953+0000","last_unstale":"2024-03-12T04:14:14.951853+0000","last_undegraded":"2024-03-12T04:14:14.951853+0000","last_fullsized":"2024-03-12T04:14:14.951853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:14:02.182546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937316+0000","last_change":"2024-03-12T04:12:53.770230+0000","last_active":"2024-03-12T04:13:45.937316+0000","last_peered":"2024-03-12T04:13:45.937316+0000","last_clean":"2024-03-12T04:13:45.937316+0000","last_became_active":"2024-03-12T04:12:53.768817+0000","last_became_peered":"2024-03-12T04:12:53.768817+0000","last_unstale":"2024-03-12T04:13:45.937316+0000","last_undegraded":"2024-03-12T04:13:45.937316+0000","last_fullsized":"2024-03-12T04:13:45.937316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:09:12.656489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940483+0000","last_change":"2024-03-12T04:12:54.785654+0000","last_active":"2024-03-12T04:14:14.940483+0000","last_peered":"2024-03-12T04:14:14.940483+0000","last_clean":"2024-03-12T04:14:14.940483+0000","last_became_active":"2024-03-12T04:12:54.785244+0000","last_became_peered":"2024-03-12T04:12:54.785244+0000","last_unstale":"2024-03-12T04:14:14.940483+0000","last_undegraded":"2024-03-12T04:14:14.940483+0000","last_fullsized":"2024-03-12T04:14:14.940483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:00:41.961637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930880+0000","last_change":"2024-03-12T04:12:53.780962+0000","last_active":"2024-03-12T04:13:45.930880+0000","last_peered":"2024-03-12T04:13:45.930880+0000","last_clean":"2024-03-12T04:13:45.930880+0000","last_became_active":"2024-03-12T04:12:53.780623+0000","last_became_peered":"2024-03-12T04:12:53.780623+0000","last_unstale":"2024-03-12T04:13:45.930880+0000","last_undegraded":"2024-03-12T04:13:45.930880+0000","last_fullsized":"2024-03-12T04:13:45.930880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946481+0000","last_change":"2024-03-12T04:12:54.775065+0000","last_active":"2024-03-12T04:14:14.946481+0000","last_peered":"2024-03-12T04:14:14.946481+0000","last_clean":"2024-03-12T04:14:14.946481+0000","last_became_active":"2024-03-12T04:12:54.774686+0000","last_became_peered":"2024-03-12T04:12:54.774686+0000","last_unstale":"2024-03-12T04:14:14.946481+0000","last_undegraded":"2024-03-12T04:14:14.946481+0000","last_fullsized":"2024-03-12T04:14:14.946481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:56:25.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:55.189811+0000","last_change":"2024-03-12T04:12:53.777702+0000","last_active":"2024-03-12T04:14:55.189811+0000","last_peered":"2024-03-12T04:14:55.189811+0000","last_clean":"2024-03-12T04:14:55.189811+0000","last_became_active":"2024-03-12T04:12:53.777346+0000","last_became_peered":"2024-03-12T04:12:53.777346+0000","last_unstale":"2024-03-12T04:14:55.189811+0000","last_undegraded":"2024-03-12T04:14:55.189811+0000","last_fullsized":"2024-03-12T04:14:55.189811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:09.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939910+0000","last_change":"2024-03-12T04:12:54.789087+0000","last_active":"2024-03-12T04:14:14.939910+0000","last_peered":"2024-03-12T04:14:14.939910+0000","last_clean":"2024-03-12T04:14:14.939910+0000","last_became_active":"2024-03-12T04:12:54.788735+0000","last_became_peered":"2024-03-12T04:12:54.788735+0000","last_unstale":"2024-03-12T04:14:14.939910+0000","last_undegraded":"2024-03-12T04:14:14.939910+0000","last_fullsized":"2024-03-12T04:14:14.939910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:14.947137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945695+0000","last_change":"2024-03-12T04:12:54.789974+0000","last_active":"2024-03-12T04:14:14.945695+0000","last_peered":"2024-03-12T04:14:14.945695+0000","last_clean":"2024-03-12T04:14:14.945695+0000","last_became_active":"2024-03-12T04:12:54.789578+0000","last_became_peered":"2024-03-12T04:12:54.789578+0000","last_unstale":"2024-03-12T04:14:14.945695+0000","last_undegraded":"2024-03-12T04:14:14.945695+0000","last_fullsized":"2024-03-12T04:14:14.945695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:22:25.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.224216+0000","last_change":"2024-03-12T04:12:53.773601+0000","last_active":"2024-03-12T04:14:54.224216+0000","last_peered":"2024-03-12T04:14:54.224216+0000","last_clean":"2024-03-12T04:14:54.224216+0000","last_became_active":"2024-03-12T04:12:53.773264+0000","last_became_peered":"2024-03-12T04:12:53.773264+0000","last_unstale":"2024-03-12T04:14:54.224216+0000","last_undegraded":"2024-03-12T04:14:54.224216+0000","last_fullsized":"2024-03-12T04:14:54.224216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:07:03.580157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948721+0000","last_change":"2024-03-12T04:12:54.782496+0000","last_active":"2024-03-12T04:14:14.948721+0000","last_peered":"2024-03-12T04:14:14.948721+0000","last_clean":"2024-03-12T04:14:14.948721+0000","last_became_active":"2024-03-12T04:12:54.782014+0000","last_became_peered":"2024-03-12T04:12:54.782014+0000","last_unstale":"2024-03-12T04:14:14.948721+0000","last_undegraded":"2024-03-12T04:14:14.948721+0000","last_fullsized":"2024-03-12T04:14:14.948721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:08.580655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099322+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:46.099322+0000","last_peered":"2024-03-12T04:13:46.099322+0000","last_clean":"2024-03-12T04:13:46.099322+0000","last_became_active":"2024-03-12T04:12:53.770855+0000","last_became_peered":"2024-03-12T04:12:53.770855+0000","last_unstale":"2024-03-12T04:13:46.099322+0000","last_undegraded":"2024-03-12T04:13:46.099322+0000","last_fullsized":"2024-03-12T04:13:46.099322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:32:33.969240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100306+0000","last_change":"2024-03-12T04:11:37.642780+0000","last_active":"2024-03-12T04:13:46.100306+0000","last_peered":"2024-03-12T04:13:46.100306+0000","last_clean":"2024-03-12T04:13:46.100306+0000","last_became_active":"2024-03-12T04:11:37.642145+0000","last_became_peered":"2024-03-12T04:11:37.642145+0000","last_unstale":"2024-03-12T04:13:46.100306+0000","last_undegraded":"2024-03-12T04:13:46.100306+0000","last_fullsized":"2024-03-12T04:13:46.100306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_clean_scrub_stamp":"2024-03-12T04:08:09.604393+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:46:36.666483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935254+0000","last_change":"2024-03-12T04:12:54.790464+0000","last_active":"2024-03-12T04:14:14.935254+0000","last_peered":"2024-03-12T04:14:14.935254+0000","last_clean":"2024-03-12T04:14:14.935254+0000","last_became_active":"2024-03-12T04:12:54.788440+0000","last_became_peered":"2024-03-12T04:12:54.788440+0000","last_unstale":"2024-03-12T04:14:14.935254+0000","last_undegraded":"2024-03-12T04:14:14.935254+0000","last_fullsized":"2024-03-12T04:14:14.935254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:54.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100152+0000","last_change":"2024-03-12T04:12:53.773334+0000","last_active":"2024-03-12T04:13:46.100152+0000","last_peered":"2024-03-12T04:13:46.100152+0000","last_clean":"2024-03-12T04:13:46.100152+0000","last_became_active":"2024-03-12T04:12:53.772759+0000","last_became_peered":"2024-03-12T04:12:53.772759+0000","last_unstale":"2024-03-12T04:13:46.100152+0000","last_undegraded":"2024-03-12T04:13:46.100152+0000","last_fullsized":"2024-03-12T04:13:46.100152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:03:17.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937169+0000","last_change":"2024-03-12T04:12:53.777035+0000","last_active":"2024-03-12T04:13:45.937169+0000","last_peered":"2024-03-12T04:13:45.937169+0000","last_clean":"2024-03-12T04:13:45.937169+0000","last_became_active":"2024-03-12T04:12:53.776587+0000","last_became_peered":"2024-03-12T04:12:53.776587+0000","last_unstale":"2024-03-12T04:13:45.937169+0000","last_undegraded":"2024-03-12T04:13:45.937169+0000","last_fullsized":"2024-03-12T04:13:45.937169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:21.390525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.934802+0000","last_change":"2024-03-12T04:12:54.790835+0000","last_active":"2024-03-12T04:14:14.934802+0000","last_peered":"2024-03-12T04:14:14.934802+0000","last_clean":"2024-03-12T04:14:14.934802+0000","last_became_active":"2024-03-12T04:12:54.790512+0000","last_became_peered":"2024-03-12T04:12:54.790512+0000","last_unstale":"2024-03-12T04:14:14.934802+0000","last_undegraded":"2024-03-12T04:14:14.934802+0000","last_fullsized":"2024-03-12T04:14:14.934802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:37:58.029450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931069+0000","last_change":"2024-03-12T04:12:53.774604+0000","last_active":"2024-03-12T04:13:45.931069+0000","last_peered":"2024-03-12T04:13:45.931069+0000","last_clean":"2024-03-12T04:13:45.931069+0000","last_became_active":"2024-03-12T04:12:53.774270+0000","last_became_peered":"2024-03-12T04:12:53.774270+0000","last_unstale":"2024-03-12T04:13:45.931069+0000","last_undegraded":"2024-03-12T04:13:45.931069+0000","last_fullsized":"2024-03-12T04:13:45.931069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:23.635179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946179+0000","last_change":"2024-03-12T04:12:54.786259+0000","last_active":"2024-03-12T04:14:14.946179+0000","last_peered":"2024-03-12T04:14:14.946179+0000","last_clean":"2024-03-12T04:14:14.946179+0000","last_became_active":"2024-03-12T04:12:54.783114+0000","last_became_peered":"2024-03-12T04:12:54.783114+0000","last_unstale":"2024-03-12T04:14:14.946179+0000","last_undegraded":"2024-03-12T04:14:14.946179+0000","last_fullsized":"2024-03-12T04:14:14.946179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:53.036505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099781+0000","last_change":"2024-03-12T04:12:53.779618+0000","last_active":"2024-03-12T04:13:46.099781+0000","last_peered":"2024-03-12T04:13:46.099781+0000","last_clean":"2024-03-12T04:13:46.099781+0000","last_became_active":"2024-03-12T04:12:53.779264+0000","last_became_peered":"2024-03-12T04:12:53.779264+0000","last_unstale":"2024-03-12T04:13:46.099781+0000","last_undegraded":"2024-03-12T04:13:46.099781+0000","last_fullsized":"2024-03-12T04:13:46.099781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:57.633014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945120+0000","last_change":"2024-03-12T04:12:54.789012+0000","last_active":"2024-03-12T04:14:14.945120+0000","last_peered":"2024-03-12T04:14:14.945120+0000","last_clean":"2024-03-12T04:14:14.945120+0000","last_became_active":"2024-03-12T04:12:54.788441+0000","last_became_peered":"2024-03-12T04:12:54.788441+0000","last_unstale":"2024-03-12T04:14:14.945120+0000","last_undegraded":"2024-03-12T04:14:14.945120+0000","last_fullsized":"2024-03-12T04:14:14.945120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:55.858926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102816+0000","last_change":"2024-03-12T04:12:53.776621+0000","last_active":"2024-03-12T04:13:46.102816+0000","last_peered":"2024-03-12T04:13:46.102816+0000","last_clean":"2024-03-12T04:13:46.102816+0000","last_became_active":"2024-03-12T04:12:53.776313+0000","last_became_peered":"2024-03-12T04:12:53.776313+0000","last_unstale":"2024-03-12T04:13:46.102816+0000","last_undegraded":"2024-03-12T04:13:46.102816+0000","last_fullsized":"2024-03-12T04:13:46.102816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:43.536159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.900148+0000","last_change":"2024-03-12T04:12:54.786929+0000","last_active":"2024-03-12T04:14:14.900148+0000","last_peered":"2024-03-12T04:14:14.900148+0000","last_clean":"2024-03-12T04:14:14.900148+0000","last_became_active":"2024-03-12T04:12:54.786580+0000","last_became_peered":"2024-03-12T04:12:54.786580+0000","last_unstale":"2024-03-12T04:14:14.900148+0000","last_undegraded":"2024-03-12T04:14:14.900148+0000","last_fullsized":"2024-03-12T04:14:14.900148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:58.955293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935801+0000","last_change":"2024-03-12T04:12:53.778577+0000","last_active":"2024-03-12T04:13:45.935801+0000","last_peered":"2024-03-12T04:13:45.935801+0000","last_clean":"2024-03-12T04:13:45.935801+0000","last_became_active":"2024-03-12T04:12:53.778033+0000","last_became_peered":"2024-03-12T04:12:53.778033+0000","last_unstale":"2024-03-12T04:13:45.935801+0000","last_undegraded":"2024-03-12T04:13:45.935801+0000","last_fullsized":"2024-03-12T04:13:45.935801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:02.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939536+0000","last_change":"2024-03-12T04:12:54.785861+0000","last_active":"2024-03-12T04:14:14.939536+0000","last_peered":"2024-03-12T04:14:14.939536+0000","last_clean":"2024-03-12T04:14:14.939536+0000","last_became_active":"2024-03-12T04:12:54.784682+0000","last_became_peered":"2024-03-12T04:12:54.784682+0000","last_unstale":"2024-03-12T04:14:14.939536+0000","last_undegraded":"2024-03-12T04:14:14.939536+0000","last_fullsized":"2024-03-12T04:14:14.939536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:35:12.288309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931757+0000","last_change":"2024-03-12T04:12:53.772865+0000","last_active":"2024-03-12T04:13:45.931757+0000","last_peered":"2024-03-12T04:13:45.931757+0000","last_clean":"2024-03-12T04:13:45.931757+0000","last_became_active":"2024-03-12T04:12:53.772502+0000","last_became_peered":"2024-03-12T04:12:53.772502+0000","last_unstale":"2024-03-12T04:13:45.931757+0000","last_undegraded":"2024-03-12T04:13:45.931757+0000","last_fullsized":"2024-03-12T04:13:45.931757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:40:24.112197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.942329+0000","last_change":"2024-03-12T04:12:54.780722+0000","last_active":"2024-03-12T04:14:14.942329+0000","last_peered":"2024-03-12T04:14:14.942329+0000","last_clean":"2024-03-12T04:14:14.942329+0000","last_became_active":"2024-03-12T04:12:54.780404+0000","last_became_peered":"2024-03-12T04:12:54.780404+0000","last_unstale":"2024-03-12T04:14:14.942329+0000","last_undegraded":"2024-03-12T04:14:14.942329+0000","last_fullsized":"2024-03-12T04:14:14.942329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.068094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936882+0000","last_change":"2024-03-12T04:12:53.771471+0000","last_active":"2024-03-12T04:13:45.936882+0000","last_peered":"2024-03-12T04:13:45.936882+0000","last_clean":"2024-03-12T04:13:45.936882+0000","last_became_active":"2024-03-12T04:12:53.771147+0000","last_became_peered":"2024-03-12T04:12:53.771147+0000","last_unstale":"2024-03-12T04:13:45.936882+0000","last_undegraded":"2024-03-12T04:13:45.936882+0000","last_fullsized":"2024-03-12T04:13:45.936882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:46.159168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949687+0000","last_change":"2024-03-12T04:12:54.796253+0000","last_active":"2024-03-12T04:14:14.949687+0000","last_peered":"2024-03-12T04:14:14.949687+0000","last_clean":"2024-03-12T04:14:14.949687+0000","last_became_active":"2024-03-12T04:12:54.795656+0000","last_became_peered":"2024-03-12T04:12:54.795656+0000","last_unstale":"2024-03-12T04:14:14.949687+0000","last_undegraded":"2024-03-12T04:14:14.949687+0000","last_fullsized":"2024-03-12T04:14:14.949687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:39:32.788874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.198832+0000","last_change":"2024-03-12T04:12:53.780257+0000","last_active":"2024-03-12T04:14:54.198832+0000","last_peered":"2024-03-12T04:14:54.198832+0000","last_clean":"2024-03-12T04:14:54.198832+0000","last_became_active":"2024-03-12T04:12:53.779891+0000","last_became_peered":"2024-03-12T04:12:53.779891+0000","last_unstale":"2024-03-12T04:14:54.198832+0000","last_undegraded":"2024-03-12T04:14:54.198832+0000","last_fullsized":"2024-03-12T04:14:54.198832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:49:30.218372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.950676+0000","last_change":"2024-03-12T04:12:54.792760+0000","last_active":"2024-03-12T04:14:14.950676+0000","last_peered":"2024-03-12T04:14:14.950676+0000","last_clean":"2024-03-12T04:14:14.950676+0000","last_became_active":"2024-03-12T04:12:54.792363+0000","last_became_peered":"2024-03-12T04:12:54.792363+0000","last_unstale":"2024-03-12T04:14:14.950676+0000","last_undegraded":"2024-03-12T04:14:14.950676+0000","last_fullsized":"2024-03-12T04:14:14.950676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:30:15.394318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935702+0000","last_change":"2024-03-12T04:12:53.775556+0000","last_active":"2024-03-12T04:13:45.935702+0000","last_peered":"2024-03-12T04:13:45.935702+0000","last_clean":"2024-03-12T04:13:45.935702+0000","last_became_active":"2024-03-12T04:12:53.775182+0000","last_became_peered":"2024-03-12T04:12:53.775182+0000","last_unstale":"2024-03-12T04:13:45.935702+0000","last_undegraded":"2024-03-12T04:13:45.935702+0000","last_fullsized":"2024-03-12T04:13:45.935702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:13:50.076672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948544+0000","last_change":"2024-03-12T04:12:54.795876+0000","last_active":"2024-03-12T04:14:14.948544+0000","last_peered":"2024-03-12T04:14:14.948544+0000","last_clean":"2024-03-12T04:14:14.948544+0000","last_became_active":"2024-03-12T04:12:54.793874+0000","last_became_peered":"2024-03-12T04:12:54.793874+0000","last_unstale":"2024-03-12T04:14:14.948544+0000","last_undegraded":"2024-03-12T04:14:14.948544+0000","last_fullsized":"2024-03-12T04:14:14.948544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:33:31.165974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933769+0000","last_change":"2024-03-12T04:12:53.774879+0000","last_active":"2024-03-12T04:13:45.933769+0000","last_peered":"2024-03-12T04:13:45.933769+0000","last_clean":"2024-03-12T04:13:45.933769+0000","last_became_active":"2024-03-12T04:12:53.774530+0000","last_became_peered":"2024-03-12T04:12:53.774530+0000","last_unstale":"2024-03-12T04:13:45.933769+0000","last_undegraded":"2024-03-12T04:13:45.933769+0000","last_fullsized":"2024-03-12T04:13:45.933769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:25.858165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946946+0000","last_change":"2024-03-12T04:12:54.790140+0000","last_active":"2024-03-12T04:14:14.946946+0000","last_peered":"2024-03-12T04:14:14.946946+0000","last_clean":"2024-03-12T04:14:14.946946+0000","last_became_active":"2024-03-12T04:12:54.789798+0000","last_became_peered":"2024-03-12T04:12:54.789798+0000","last_unstale":"2024-03-12T04:14:14.946946+0000","last_undegraded":"2024-03-12T04:14:14.946946+0000","last_fullsized":"2024-03-12T04:14:14.946946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:12:38.722638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.936334+0000","last_change":"2024-03-12T04:12:54.785029+0000","last_active":"2024-03-12T04:14:14.936334+0000","last_peered":"2024-03-12T04:14:14.936334+0000","last_clean":"2024-03-12T04:14:14.936334+0000","last_became_active":"2024-03-12T04:12:54.784197+0000","last_became_peered":"2024-03-12T04:12:54.784197+0000","last_unstale":"2024-03-12T04:14:14.936334+0000","last_undegraded":"2024-03-12T04:14:14.936334+0000","last_fullsized":"2024-03-12T04:14:14.936334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:22:41.352093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.937779+0000","last_change":"2024-03-12T04:12:54.793132+0000","last_active":"2024-03-12T04:14:14.937779+0000","last_peered":"2024-03-12T04:14:14.937779+0000","last_clean":"2024-03-12T04:14:14.937779+0000","last_became_active":"2024-03-12T04:12:54.792838+0000","last_became_peered":"2024-03-12T04:12:54.792838+0000","last_unstale":"2024-03-12T04:14:14.937779+0000","last_undegraded":"2024-03-12T04:14:14.937779+0000","last_fullsized":"2024-03-12T04:14:14.937779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:25:22.641963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935192+0000","last_change":"2024-03-12T04:13:45.938126+0000","last_active":"2024-03-12T04:14:14.935192+0000","last_peered":"2024-03-12T04:14:14.935192+0000","last_clean":"2024-03-12T04:14:14.935192+0000","last_became_active":"2024-03-12T04:13:45.937698+0000","last_became_peered":"2024-03-12T04:13:45.937698+0000","last_unstale":"2024-03-12T04:14:14.935192+0000","last_undegraded":"2024-03-12T04:14:14.935192+0000","last_fullsized":"2024-03-12T04:14:14.935192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:10:25.378634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944252+0000","last_change":"2024-03-12T04:12:54.785830+0000","last_active":"2024-03-12T04:14:14.944252+0000","last_peered":"2024-03-12T04:14:14.944252+0000","last_clean":"2024-03-12T04:14:14.944252+0000","last_became_active":"2024-03-12T04:12:54.785087+0000","last_became_peered":"2024-03-12T04:12:54.785087+0000","last_unstale":"2024-03-12T04:14:14.944252+0000","last_undegraded":"2024-03-12T04:14:14.944252+0000","last_fullsized":"2024-03-12T04:14:14.944252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:08:09.860311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944035+0000","last_change":"2024-03-12T04:13:45.937883+0000","last_active":"2024-03-12T04:14:14.944035+0000","last_peered":"2024-03-12T04:14:14.944035+0000","last_clean":"2024-03-12T04:14:14.944035+0000","last_became_active":"2024-03-12T04:13:45.937471+0000","last_became_peered":"2024-03-12T04:13:45.937471+0000","last_unstale":"2024-03-12T04:14:14.944035+0000","last_undegraded":"2024-03-12T04:14:14.944035+0000","last_fullsized":"2024-03-12T04:14:14.944035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:28:21.997033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.932702+0000","last_change":"2024-03-12T04:13:45.938864+0000","last_active":"2024-03-12T04:14:14.932702+0000","last_peered":"2024-03-12T04:14:14.932702+0000","last_clean":"2024-03-12T04:14:14.932702+0000","last_became_active":"2024-03-12T04:13:45.938336+0000","last_became_peered":"2024-03-12T04:13:45.938336+0000","last_unstale":"2024-03-12T04:14:14.932702+0000","last_undegraded":"2024-03-12T04:14:14.932702+0000","last_fullsized":"2024-03-12T04:14:14.932702+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:44.990282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.938228+0000","last_change":"2024-03-12T04:12:54.796733+0000","last_active":"2024-03-12T04:14:14.938228+0000","last_peered":"2024-03-12T04:14:14.938228+0000","last_clean":"2024-03-12T04:14:14.938228+0000","last_became_active":"2024-03-12T04:12:54.796388+0000","last_became_peered":"2024-03-12T04:12:54.796388+0000","last_unstale":"2024-03-12T04:14:14.938228+0000","last_undegraded":"2024-03-12T04:14:14.938228+0000","last_fullsized":"2024-03-12T04:14:14.938228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:01.580286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.947274+0000","last_change":"2024-03-12T04:12:54.787200+0000","last_active":"2024-03-12T04:14:14.947274+0000","last_peered":"2024-03-12T04:14:14.947274+0000","last_clean":"2024-03-12T04:14:14.947274+0000","last_became_active":"2024-03-12T04:12:54.786657+0000","last_became_peered":"2024-03-12T04:12:54.786657+0000","last_unstale":"2024-03-12T04:14:14.947274+0000","last_undegraded":"2024-03-12T04:14:14.947274+0000","last_fullsized":"2024-03-12T04:14:14.947274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:15:35.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.943388+0000","last_change":"2024-03-12T04:12:54.785132+0000","last_active":"2024-03-12T04:14:14.943388+0000","last_peered":"2024-03-12T04:14:14.943388+0000","last_clean":"2024-03-12T04:14:14.943388+0000","last_became_active":"2024-03-12T04:12:54.782548+0000","last_became_peered":"2024-03-12T04:12:54.782548+0000","last_unstale":"2024-03-12T04:14:14.943388+0000","last_undegraded":"2024-03-12T04:14:14.943388+0000","last_fullsized":"2024-03-12T04:14:14.943388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:40:49.284778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948256+0000","last_change":"2024-03-12T04:12:54.790227+0000","last_active":"2024-03-12T04:14:14.948256+0000","last_peered":"2024-03-12T04:14:14.948256+0000","last_clean":"2024-03-12T04:14:14.948256+0000","last_became_active":"2024-03-12T04:12:54.789853+0000","last_became_peered":"2024-03-12T04:12:54.789853+0000","last_unstale":"2024-03-12T04:14:14.948256+0000","last_undegraded":"2024-03-12T04:14:14.948256+0000","last_fullsized":"2024-03-12T04:14:14.948256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:38:16.683661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953923+0000","last_change":"2024-03-12T04:12:54.796039+0000","last_active":"2024-03-12T04:14:14.953923+0000","last_peered":"2024-03-12T04:14:14.953923+0000","last_clean":"2024-03-12T04:14:14.953923+0000","last_became_active":"2024-03-12T04:12:54.795568+0000","last_became_peered":"2024-03-12T04:12:54.795568+0000","last_unstale":"2024-03-12T04:14:14.953923+0000","last_undegraded":"2024-03-12T04:14:14.953923+0000","last_fullsized":"2024-03-12T04:14:14.953923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:46.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946286+0000","last_change":"2024-03-12T04:12:54.785610+0000","last_active":"2024-03-12T04:14:14.946286+0000","last_peered":"2024-03-12T04:14:14.946286+0000","last_clean":"2024-03-12T04:14:14.946286+0000","last_became_active":"2024-03-12T04:12:54.785256+0000","last_became_peered":"2024-03-12T04:12:54.785256+0000","last_unstale":"2024-03-12T04:14:14.946286+0000","last_undegraded":"2024-03-12T04:14:14.946286+0000","last_fullsized":"2024-03-12T04:14:14.946286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:34.152361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940458+0000","last_change":"2024-03-12T04:12:54.787064+0000","last_active":"2024-03-12T04:14:14.940458+0000","last_peered":"2024-03-12T04:14:14.940458+0000","last_clean":"2024-03-12T04:14:14.940458+0000","last_became_active":"2024-03-12T04:12:54.786701+0000","last_became_peered":"2024-03-12T04:12:54.786701+0000","last_unstale":"2024-03-12T04:14:14.940458+0000","last_undegraded":"2024-03-12T04:14:14.940458+0000","last_fullsized":"2024-03-12T04:14:14.940458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:44:43.582171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250964,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1208,"num_read_kb":681,"num_write":257,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":206898,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":468541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":1119430,"data_compressed":38179,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":467798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":468064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1062846,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":468008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":528429,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":469347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":528127,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":467340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":468401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1060513,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61753,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62930,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62269,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T04:15:00.183 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T04:15:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 -- 172.21.15.42:0/2633032635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 msgr2=0x7f89881092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 --2- 172.21.15.42:0/2633032635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f89881092d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8978009ca0 tx=0x7f897802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 -- 172.21.15.42:0/2633032635 shutdown_connections 2024-03-12T04:15:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 --2- 172.21.15.42:0/2633032635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f89881092d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 --2- 172.21.15.42:0/2633032635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8988104550 0x7f8988106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 --2- 172.21.15.42:0/2633032635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8988101c20 0x7f8988104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 -- 172.21.15.42:0/2633032635 >> 172.21.15.42:0/2633032635 conn(0x7f89880fb610 msgr2=0x7f89880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 -- 172.21.15.42:0/2633032635 shutdown_connections 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.319+0000 7f898f78a640 1 -- 172.21.15.42:0/2633032635 wait complete. 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 Processor -- start 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 -- start start 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8988101c20 0x7f8988197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8988104550 0x7f8988197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f898819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898819ec20 con 0x7f8988106e80 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898819ed90 con 0x7f8988104550 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898f78a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898819ef50 con 0x7f8988101c20 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f898819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f898819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50098/0 (socket says 172.21.15.42:50098) 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 -- 172.21.15.42:0/922402573 learned_addr learned my addr 172.21.15.42:0/922402573 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898d526640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8988101c20 0x7f8988197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 -- 172.21.15.42:0/922402573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8988101c20 msgr2=0x7f8988197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8988101c20 0x7f8988197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 -- 172.21.15.42:0/922402573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8988104550 msgr2=0x7f8988197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8988104550 0x7f8988197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 -- 172.21.15.42:0/922402573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8978009900 con 0x7f8988106e80 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898d526640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8988101c20 0x7f8988197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.320+0000 7f898dd27640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f898819e6e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8978037040 tx=0x7f8978032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.321+0000 7f89867fc640 1 -- 172.21.15.42:0/922402573 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978034e70 con 0x7f8988106e80 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.321+0000 7f89867fc640 1 -- 172.21.15.42:0/922402573 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8978003e20 con 0x7f8988106e80 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.321+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898819f1d0 con 0x7f8988106e80 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.321+0000 7f89867fc640 1 -- 172.21.15.42:0/922402573 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897803d3f0 con 0x7f8988106e80 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.321+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898819f680 con 0x7f8988106e80 2024-03-12T04:15:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.322+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89880fd100 con 0x7f8988106e80 2024-03-12T04:15:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.323+0000 7f89867fc640 1 -- 172.21.15.42:0/922402573 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f897803b040 con 0x7f8988106e80 2024-03-12T04:15:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.323+0000 7f89867fc640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89700a1a30 0x7f89700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.323+0000 7f898d526640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89700a1a30 0x7f89700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.324+0000 7f89867fc640 1 -- 172.21.15.42:0/922402573 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f89780efc30 con 0x7f8988106e80 2024-03-12T04:15:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.324+0000 7f898d526640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89700a1a30 0x7f89700a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89800058c0 tx=0x7f8980005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:00.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.328+0000 7f89867fc640 1 -- 172.21.15.42:0/922402573 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89780b8e70 con 0x7f8988106e80 2024-03-12T04:15:00.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:00 smithi119 ceph-mon[36119]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-12T04:15:00.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:00 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1316492865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:15:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:00 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2171953958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:15:00.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:00 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/668385630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:00 smithi136 ceph-mon[35571]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-12T04:15:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:00 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1316492865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:15:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:00 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2171953958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:15:00.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:00 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/668385630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:00.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.505+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f898802ceb0 con 0x7f89700a1a30 2024-03-12T04:15:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.512+0000 7f89867fc640 1 -- 172.21.15.42:0/922402573 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259100 (secure 0 0 0) 0x7f898802ceb0 con 0x7f89700a1a30 2024-03-12T04:15:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:15:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89700a1a30 msgr2=0x7f89700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89700a1a30 0x7f89700a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89800058c0 tx=0x7f8980005850 comp rx=0 tx=0).stop 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 msgr2=0x7f898819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f898819e6e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8978037040 tx=0x7f8978032980 comp rx=0 tx=0).stop 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 shutdown_connections 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8988106e80 0x7f898819e6e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f89700a1a30 0x7f89700a3ef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8988104550 0x7f8988197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 --2- 172.21.15.42:0/922402573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8988101c20 0x7f8988197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 >> 172.21.15.42:0/922402573 conn(0x7f89880fb610 msgr2=0x7f8988102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 shutdown_connections 2024-03-12T04:15:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.515+0000 7f898f78a640 1 -- 172.21.15.42:0/922402573 wait complete. 2024-03-12T04:15:00.529 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-03-12T04:14:58.930090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1344,"num_read_kb":800,"num_write":440,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373620,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568204,"statfs":{"total":1151940427776,"available":1151557840896,"internally_reserved":0,"allocated":12275712,"data_stored":7576844,"data_compressed":114537,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":67,"num_write":11,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003272"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949806+0000","last_change":"2024-03-12T04:12:54.788665+0000","last_active":"2024-03-12T04:14:14.949806+0000","last_peered":"2024-03-12T04:14:14.949806+0000","last_clean":"2024-03-12T04:14:14.949806+0000","last_became_active":"2024-03-12T04:12:54.788334+0000","last_became_peered":"2024-03-12T04:12:54.788334+0000","last_unstale":"2024-03-12T04:14:14.949806+0000","last_undegraded":"2024-03-12T04:14:14.949806+0000","last_fullsized":"2024-03-12T04:14:14.949806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:02.442543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940407+0000","last_change":"2024-03-12T04:12:54.791942+0000","last_active":"2024-03-12T04:14:14.940407+0000","last_peered":"2024-03-12T04:14:14.940407+0000","last_clean":"2024-03-12T04:14:14.940407+0000","last_became_active":"2024-03-12T04:12:54.791579+0000","last_became_peered":"2024-03-12T04:12:54.791579+0000","last_unstale":"2024-03-12T04:14:14.940407+0000","last_undegraded":"2024-03-12T04:14:14.940407+0000","last_fullsized":"2024-03-12T04:14:14.940407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:51:08.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949288+0000","last_change":"2024-03-12T04:12:54.793238+0000","last_active":"2024-03-12T04:14:14.949288+0000","last_peered":"2024-03-12T04:14:14.949288+0000","last_clean":"2024-03-12T04:14:14.949288+0000","last_became_active":"2024-03-12T04:12:54.792877+0000","last_became_peered":"2024-03-12T04:12:54.792877+0000","last_unstale":"2024-03-12T04:14:14.949288+0000","last_undegraded":"2024-03-12T04:14:14.949288+0000","last_fullsized":"2024-03-12T04:14:14.949288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:36:46.914204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945072+0000","last_change":"2024-03-12T04:12:54.797573+0000","last_active":"2024-03-12T04:14:14.945072+0000","last_peered":"2024-03-12T04:14:14.945072+0000","last_clean":"2024-03-12T04:14:14.945072+0000","last_became_active":"2024-03-12T04:12:54.796939+0000","last_became_peered":"2024-03-12T04:12:54.796939+0000","last_unstale":"2024-03-12T04:14:14.945072+0000","last_undegraded":"2024-03-12T04:14:14.945072+0000","last_fullsized":"2024-03-12T04:14:14.945072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:34:58.047492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.912919+0000","last_change":"2024-03-12T04:12:54.780595+0000","last_active":"2024-03-12T04:14:14.912919+0000","last_peered":"2024-03-12T04:14:14.912919+0000","last_clean":"2024-03-12T04:14:14.912919+0000","last_became_active":"2024-03-12T04:12:54.780256+0000","last_became_peered":"2024-03-12T04:12:54.780256+0000","last_unstale":"2024-03-12T04:14:14.912919+0000","last_undegraded":"2024-03-12T04:14:14.912919+0000","last_fullsized":"2024-03-12T04:14:14.912919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:42.492006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935417+0000","last_change":"2024-03-12T04:13:45.934954+0000","last_active":"2024-03-12T04:14:14.935417+0000","last_peered":"2024-03-12T04:14:14.935417+0000","last_clean":"2024-03-12T04:14:14.935417+0000","last_became_active":"2024-03-12T04:13:45.934579+0000","last_became_peered":"2024-03-12T04:13:45.934579+0000","last_unstale":"2024-03-12T04:14:14.935417+0000","last_undegraded":"2024-03-12T04:14:14.935417+0000","last_fullsized":"2024-03-12T04:14:14.935417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:39:10.062304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.941999+0000","last_change":"2024-03-12T04:12:54.781510+0000","last_active":"2024-03-12T04:14:14.941999+0000","last_peered":"2024-03-12T04:14:14.941999+0000","last_clean":"2024-03-12T04:14:14.941999+0000","last_became_active":"2024-03-12T04:12:54.781141+0000","last_became_peered":"2024-03-12T04:12:54.781141+0000","last_unstale":"2024-03-12T04:14:14.941999+0000","last_undegraded":"2024-03-12T04:14:14.941999+0000","last_fullsized":"2024-03-12T04:14:14.941999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:42:05.055308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948851+0000","last_change":"2024-03-12T04:12:54.782541+0000","last_active":"2024-03-12T04:14:14.948851+0000","last_peered":"2024-03-12T04:14:14.948851+0000","last_clean":"2024-03-12T04:14:14.948851+0000","last_became_active":"2024-03-12T04:12:54.781582+0000","last_became_peered":"2024-03-12T04:12:54.781582+0000","last_unstale":"2024-03-12T04:14:14.948851+0000","last_undegraded":"2024-03-12T04:14:14.948851+0000","last_fullsized":"2024-03-12T04:14:14.948851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:15.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953459+0000","last_change":"2024-03-12T04:12:54.788201+0000","last_active":"2024-03-12T04:14:14.953459+0000","last_peered":"2024-03-12T04:14:14.953459+0000","last_clean":"2024-03-12T04:14:14.953459+0000","last_became_active":"2024-03-12T04:12:54.787875+0000","last_became_peered":"2024-03-12T04:12:54.787875+0000","last_unstale":"2024-03-12T04:14:14.953459+0000","last_undegraded":"2024-03-12T04:14:14.953459+0000","last_fullsized":"2024-03-12T04:14:14.953459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:40:39.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944262+0000","last_change":"2024-03-12T04:12:54.789748+0000","last_active":"2024-03-12T04:14:14.944262+0000","last_peered":"2024-03-12T04:14:14.944262+0000","last_clean":"2024-03-12T04:14:14.944262+0000","last_became_active":"2024-03-12T04:12:54.788838+0000","last_became_peered":"2024-03-12T04:12:54.788838+0000","last_unstale":"2024-03-12T04:14:14.944262+0000","last_undegraded":"2024-03-12T04:14:14.944262+0000","last_fullsized":"2024-03-12T04:14:14.944262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:39:51.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949656+0000","last_change":"2024-03-12T04:12:54.793101+0000","last_active":"2024-03-12T04:14:14.949656+0000","last_peered":"2024-03-12T04:14:14.949656+0000","last_clean":"2024-03-12T04:14:14.949656+0000","last_became_active":"2024-03-12T04:12:54.792488+0000","last_became_peered":"2024-03-12T04:12:54.792488+0000","last_unstale":"2024-03-12T04:14:14.949656+0000","last_undegraded":"2024-03-12T04:14:14.949656+0000","last_fullsized":"2024-03-12T04:14:14.949656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:33.862011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.942330+0000","last_change":"2024-03-12T04:12:54.797953+0000","last_active":"2024-03-12T04:14:14.942330+0000","last_peered":"2024-03-12T04:14:14.942330+0000","last_clean":"2024-03-12T04:14:14.942330+0000","last_became_active":"2024-03-12T04:12:54.797413+0000","last_became_peered":"2024-03-12T04:12:54.797413+0000","last_unstale":"2024-03-12T04:14:14.942330+0000","last_undegraded":"2024-03-12T04:14:14.942330+0000","last_fullsized":"2024-03-12T04:14:14.942330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:18:01.140872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.936082+0000","last_change":"2024-03-12T04:12:54.775570+0000","last_active":"2024-03-12T04:14:14.936082+0000","last_peered":"2024-03-12T04:14:14.936082+0000","last_clean":"2024-03-12T04:14:14.936082+0000","last_became_active":"2024-03-12T04:12:54.775186+0000","last_became_peered":"2024-03-12T04:12:54.775186+0000","last_unstale":"2024-03-12T04:14:14.936082+0000","last_undegraded":"2024-03-12T04:14:14.936082+0000","last_fullsized":"2024-03-12T04:14:14.936082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:26.004458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.934601+0000","last_change":"2024-03-12T04:12:54.780859+0000","last_active":"2024-03-12T04:14:14.934601+0000","last_peered":"2024-03-12T04:14:14.934601+0000","last_clean":"2024-03-12T04:14:14.934601+0000","last_became_active":"2024-03-12T04:12:54.780484+0000","last_became_peered":"2024-03-12T04:12:54.780484+0000","last_unstale":"2024-03-12T04:14:14.934601+0000","last_undegraded":"2024-03-12T04:14:14.934601+0000","last_fullsized":"2024-03-12T04:14:14.934601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:24:00.575324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.926041+0000","last_change":"2024-03-12T04:12:54.796087+0000","last_active":"2024-03-12T04:14:14.926041+0000","last_peered":"2024-03-12T04:14:14.926041+0000","last_clean":"2024-03-12T04:14:14.926041+0000","last_became_active":"2024-03-12T04:12:54.793702+0000","last_became_peered":"2024-03-12T04:12:54.793702+0000","last_unstale":"2024-03-12T04:14:14.926041+0000","last_undegraded":"2024-03-12T04:14:14.926041+0000","last_fullsized":"2024-03-12T04:14:14.926041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:09:27.334791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953800+0000","last_change":"2024-03-12T04:12:54.783146+0000","last_active":"2024-03-12T04:14:14.953800+0000","last_peered":"2024-03-12T04:14:14.953800+0000","last_clean":"2024-03-12T04:14:14.953800+0000","last_became_active":"2024-03-12T04:12:54.782834+0000","last_became_peered":"2024-03-12T04:12:54.782834+0000","last_unstale":"2024-03-12T04:14:14.953800+0000","last_undegraded":"2024-03-12T04:14:14.953800+0000","last_fullsized":"2024-03-12T04:14:14.953800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:35:11.126629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.950186+0000","last_change":"2024-03-12T04:12:54.794879+0000","last_active":"2024-03-12T04:14:14.950186+0000","last_peered":"2024-03-12T04:14:14.950186+0000","last_clean":"2024-03-12T04:14:14.950186+0000","last_became_active":"2024-03-12T04:12:54.794251+0000","last_became_peered":"2024-03-12T04:12:54.794251+0000","last_unstale":"2024-03-12T04:14:14.950186+0000","last_undegraded":"2024-03-12T04:14:14.950186+0000","last_fullsized":"2024-03-12T04:14:14.950186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:33.754426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946417+0000","last_change":"2024-03-12T04:12:54.791301+0000","last_active":"2024-03-12T04:14:14.946417+0000","last_peered":"2024-03-12T04:14:14.946417+0000","last_clean":"2024-03-12T04:14:14.946417+0000","last_became_active":"2024-03-12T04:12:54.790850+0000","last_became_peered":"2024-03-12T04:12:54.790850+0000","last_unstale":"2024-03-12T04:14:14.946417+0000","last_undegraded":"2024-03-12T04:14:14.946417+0000","last_fullsized":"2024-03-12T04:14:14.946417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:26:04.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949051+0000","last_change":"2024-03-12T04:12:54.793419+0000","last_active":"2024-03-12T04:14:14.949051+0000","last_peered":"2024-03-12T04:14:14.949051+0000","last_clean":"2024-03-12T04:14:14.949051+0000","last_became_active":"2024-03-12T04:12:54.793080+0000","last_became_peered":"2024-03-12T04:12:54.793080+0000","last_unstale":"2024-03-12T04:14:14.949051+0000","last_undegraded":"2024-03-12T04:14:14.949051+0000","last_fullsized":"2024-03-12T04:14:14.949051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:46:26.506899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:53.185317+0000","last_change":"2024-03-12T04:12:53.774893+0000","last_active":"2024-03-12T04:14:53.185317+0000","last_peered":"2024-03-12T04:14:53.185317+0000","last_clean":"2024-03-12T04:14:53.185317+0000","last_became_active":"2024-03-12T04:12:53.773801+0000","last_became_peered":"2024-03-12T04:12:53.773801+0000","last_unstale":"2024-03-12T04:14:53.185317+0000","last_undegraded":"2024-03-12T04:14:53.185317+0000","last_fullsized":"2024-03-12T04:14:53.185317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:30:19.995765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948615+0000","last_change":"2024-03-12T04:12:54.786795+0000","last_active":"2024-03-12T04:14:14.948615+0000","last_peered":"2024-03-12T04:14:14.948615+0000","last_clean":"2024-03-12T04:14:14.948615+0000","last_became_active":"2024-03-12T04:12:54.786481+0000","last_became_peered":"2024-03-12T04:12:54.786481+0000","last_unstale":"2024-03-12T04:14:14.948615+0000","last_undegraded":"2024-03-12T04:14:14.948615+0000","last_fullsized":"2024-03-12T04:14:14.948615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:06.849644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936225+0000","last_change":"2024-03-12T04:12:53.779582+0000","last_active":"2024-03-12T04:13:45.936225+0000","last_peered":"2024-03-12T04:13:45.936225+0000","last_clean":"2024-03-12T04:13:45.936225+0000","last_became_active":"2024-03-12T04:12:53.779062+0000","last_became_peered":"2024-03-12T04:12:53.779062+0000","last_unstale":"2024-03-12T04:13:45.936225+0000","last_undegraded":"2024-03-12T04:13:45.936225+0000","last_fullsized":"2024-03-12T04:13:45.936225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:44:01.065183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949313+0000","last_change":"2024-03-12T04:12:54.789198+0000","last_active":"2024-03-12T04:14:14.949313+0000","last_peered":"2024-03-12T04:14:14.949313+0000","last_clean":"2024-03-12T04:14:14.949313+0000","last_became_active":"2024-03-12T04:12:54.788213+0000","last_became_peered":"2024-03-12T04:12:54.788213+0000","last_unstale":"2024-03-12T04:14:14.949313+0000","last_undegraded":"2024-03-12T04:14:14.949313+0000","last_fullsized":"2024-03-12T04:14:14.949313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:24:36.561553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937326+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:45.937326+0000","last_peered":"2024-03-12T04:13:45.937326+0000","last_clean":"2024-03-12T04:13:45.937326+0000","last_became_active":"2024-03-12T04:12:53.771178+0000","last_became_peered":"2024-03-12T04:12:53.771178+0000","last_unstale":"2024-03-12T04:13:45.937326+0000","last_undegraded":"2024-03-12T04:13:45.937326+0000","last_fullsized":"2024-03-12T04:13:45.937326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:12.882128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953255+0000","last_change":"2024-03-12T04:12:54.799758+0000","last_active":"2024-03-12T04:14:14.953255+0000","last_peered":"2024-03-12T04:14:14.953255+0000","last_clean":"2024-03-12T04:14:14.953255+0000","last_became_active":"2024-03-12T04:12:54.799338+0000","last_became_peered":"2024-03-12T04:12:54.799338+0000","last_unstale":"2024-03-12T04:14:14.953255+0000","last_undegraded":"2024-03-12T04:14:14.953255+0000","last_fullsized":"2024-03-12T04:14:14.953255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:32:54.087890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937112+0000","last_change":"2024-03-12T04:12:53.774390+0000","last_active":"2024-03-12T04:13:45.937112+0000","last_peered":"2024-03-12T04:13:45.937112+0000","last_clean":"2024-03-12T04:13:45.937112+0000","last_became_active":"2024-03-12T04:12:53.774042+0000","last_became_peered":"2024-03-12T04:12:53.774042+0000","last_unstale":"2024-03-12T04:13:45.937112+0000","last_undegraded":"2024-03-12T04:13:45.937112+0000","last_fullsized":"2024-03-12T04:13:45.937112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:37.361505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953436+0000","last_change":"2024-03-12T04:13:46.104074+0000","last_active":"2024-03-12T04:14:14.953436+0000","last_peered":"2024-03-12T04:14:14.953436+0000","last_clean":"2024-03-12T04:14:14.953436+0000","last_became_active":"2024-03-12T04:13:46.103719+0000","last_became_peered":"2024-03-12T04:13:46.103719+0000","last_unstale":"2024-03-12T04:14:14.953436+0000","last_undegraded":"2024-03-12T04:14:14.953436+0000","last_fullsized":"2024-03-12T04:14:14.953436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:29.979576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.198832+0000","last_change":"2024-03-12T04:12:53.772182+0000","last_active":"2024-03-12T04:14:54.198832+0000","last_peered":"2024-03-12T04:14:54.198832+0000","last_clean":"2024-03-12T04:14:54.198832+0000","last_became_active":"2024-03-12T04:12:53.771858+0000","last_became_peered":"2024-03-12T04:12:53.771858+0000","last_unstale":"2024-03-12T04:14:54.198832+0000","last_undegraded":"2024-03-12T04:14:54.198832+0000","last_fullsized":"2024-03-12T04:14:54.198832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:42:41.466311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.922543+0000","last_change":"2024-03-12T04:12:54.779869+0000","last_active":"2024-03-12T04:14:14.922543+0000","last_peered":"2024-03-12T04:14:14.922543+0000","last_clean":"2024-03-12T04:14:14.922543+0000","last_became_active":"2024-03-12T04:12:54.779500+0000","last_became_peered":"2024-03-12T04:12:54.779500+0000","last_unstale":"2024-03-12T04:14:14.922543+0000","last_undegraded":"2024-03-12T04:14:14.922543+0000","last_fullsized":"2024-03-12T04:14:14.922543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:29.538570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932326+0000","last_change":"2024-03-12T04:12:53.774056+0000","last_active":"2024-03-12T04:13:45.932326+0000","last_peered":"2024-03-12T04:13:45.932326+0000","last_clean":"2024-03-12T04:13:45.932326+0000","last_became_active":"2024-03-12T04:12:53.773675+0000","last_became_peered":"2024-03-12T04:12:53.773675+0000","last_unstale":"2024-03-12T04:13:45.932326+0000","last_undegraded":"2024-03-12T04:13:45.932326+0000","last_fullsized":"2024-03-12T04:13:45.932326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:33.576947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945472+0000","last_change":"2024-03-12T04:12:54.790583+0000","last_active":"2024-03-12T04:14:14.945472+0000","last_peered":"2024-03-12T04:14:14.945472+0000","last_clean":"2024-03-12T04:14:14.945472+0000","last_became_active":"2024-03-12T04:12:54.788364+0000","last_became_peered":"2024-03-12T04:12:54.788364+0000","last_unstale":"2024-03-12T04:14:14.945472+0000","last_undegraded":"2024-03-12T04:14:14.945472+0000","last_fullsized":"2024-03-12T04:14:14.945472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:05:13.542279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933518+0000","last_change":"2024-03-12T04:12:53.769619+0000","last_active":"2024-03-12T04:13:45.933518+0000","last_peered":"2024-03-12T04:13:45.933518+0000","last_clean":"2024-03-12T04:13:45.933518+0000","last_became_active":"2024-03-12T04:12:53.769293+0000","last_became_peered":"2024-03-12T04:12:53.769293+0000","last_unstale":"2024-03-12T04:13:45.933518+0000","last_undegraded":"2024-03-12T04:13:45.933518+0000","last_fullsized":"2024-03-12T04:13:45.933518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:44.618499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940327+0000","last_change":"2024-03-12T04:12:54.776493+0000","last_active":"2024-03-12T04:14:14.940327+0000","last_peered":"2024-03-12T04:14:14.940327+0000","last_clean":"2024-03-12T04:14:14.940327+0000","last_became_active":"2024-03-12T04:12:54.775967+0000","last_became_peered":"2024-03-12T04:12:54.775967+0000","last_unstale":"2024-03-12T04:14:14.940327+0000","last_undegraded":"2024-03-12T04:14:14.940327+0000","last_fullsized":"2024-03-12T04:14:14.940327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:19:46.824145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934388+0000","last_change":"2024-03-12T04:12:53.773523+0000","last_active":"2024-03-12T04:13:45.934388+0000","last_peered":"2024-03-12T04:13:45.934388+0000","last_clean":"2024-03-12T04:13:45.934388+0000","last_became_active":"2024-03-12T04:12:53.773161+0000","last_became_peered":"2024-03-12T04:12:53.773161+0000","last_unstale":"2024-03-12T04:13:45.934388+0000","last_undegraded":"2024-03-12T04:13:45.934388+0000","last_fullsized":"2024-03-12T04:13:45.934388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:41:40.810994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939744+0000","last_change":"2024-03-12T04:12:54.791688+0000","last_active":"2024-03-12T04:14:14.939744+0000","last_peered":"2024-03-12T04:14:14.939744+0000","last_clean":"2024-03-12T04:14:14.939744+0000","last_became_active":"2024-03-12T04:12:54.790854+0000","last_became_peered":"2024-03-12T04:12:54.790854+0000","last_unstale":"2024-03-12T04:14:14.939744+0000","last_undegraded":"2024-03-12T04:14:14.939744+0000","last_fullsized":"2024-03-12T04:14:14.939744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:42:50.343524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":264,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:58.205524+0000","last_change":"2024-03-12T04:12:53.777534+0000","last_active":"2024-03-12T04:14:58.205524+0000","last_peered":"2024-03-12T04:14:58.205524+0000","last_clean":"2024-03-12T04:14:58.205524+0000","last_became_active":"2024-03-12T04:12:53.777201+0000","last_became_peered":"2024-03-12T04:12:53.777201+0000","last_unstale":"2024-03-12T04:14:58.205524+0000","last_undegraded":"2024-03-12T04:14:58.205524+0000","last_fullsized":"2024-03-12T04:14:58.205524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:54.708892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953198+0000","last_change":"2024-03-12T04:12:54.798962+0000","last_active":"2024-03-12T04:14:14.953198+0000","last_peered":"2024-03-12T04:14:14.953198+0000","last_clean":"2024-03-12T04:14:14.953198+0000","last_became_active":"2024-03-12T04:12:54.798650+0000","last_became_peered":"2024-03-12T04:12:54.798650+0000","last_unstale":"2024-03-12T04:14:14.953198+0000","last_undegraded":"2024-03-12T04:14:14.953198+0000","last_fullsized":"2024-03-12T04:14:14.953198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:05:29.270242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100119+0000","last_change":"2024-03-12T04:12:53.777791+0000","last_active":"2024-03-12T04:13:46.100119+0000","last_peered":"2024-03-12T04:13:46.100119+0000","last_clean":"2024-03-12T04:13:46.100119+0000","last_became_active":"2024-03-12T04:12:53.777430+0000","last_became_peered":"2024-03-12T04:12:53.777430+0000","last_unstale":"2024-03-12T04:13:46.100119+0000","last_undegraded":"2024-03-12T04:13:46.100119+0000","last_fullsized":"2024-03-12T04:13:46.100119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:38:21.858857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.921622+0000","last_change":"2024-03-12T04:12:54.784710+0000","last_active":"2024-03-12T04:14:14.921622+0000","last_peered":"2024-03-12T04:14:14.921622+0000","last_clean":"2024-03-12T04:14:14.921622+0000","last_became_active":"2024-03-12T04:12:54.784342+0000","last_became_peered":"2024-03-12T04:12:54.784342+0000","last_unstale":"2024-03-12T04:14:14.921622+0000","last_undegraded":"2024-03-12T04:14:14.921622+0000","last_fullsized":"2024-03-12T04:14:14.921622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:21.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935890+0000","last_change":"2024-03-12T04:12:53.779289+0000","last_active":"2024-03-12T04:13:45.935890+0000","last_peered":"2024-03-12T04:13:45.935890+0000","last_clean":"2024-03-12T04:13:45.935890+0000","last_became_active":"2024-03-12T04:12:53.778894+0000","last_became_peered":"2024-03-12T04:12:53.778894+0000","last_unstale":"2024-03-12T04:13:45.935890+0000","last_undegraded":"2024-03-12T04:13:45.935890+0000","last_fullsized":"2024-03-12T04:13:45.935890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:13:33.328977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.952684+0000","last_change":"2024-03-12T04:12:54.794102+0000","last_active":"2024-03-12T04:14:14.952684+0000","last_peered":"2024-03-12T04:14:14.952684+0000","last_clean":"2024-03-12T04:14:14.952684+0000","last_became_active":"2024-03-12T04:12:54.793789+0000","last_became_peered":"2024-03-12T04:12:54.793789+0000","last_unstale":"2024-03-12T04:14:14.952684+0000","last_undegraded":"2024-03-12T04:14:14.952684+0000","last_fullsized":"2024-03-12T04:14:14.952684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:55:33.632297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:58.221649+0000","last_change":"2024-03-12T04:12:53.774847+0000","last_active":"2024-03-12T04:14:58.221649+0000","last_peered":"2024-03-12T04:14:58.221649+0000","last_clean":"2024-03-12T04:14:58.221649+0000","last_became_active":"2024-03-12T04:12:53.774457+0000","last_became_peered":"2024-03-12T04:12:53.774457+0000","last_unstale":"2024-03-12T04:14:58.221649+0000","last_undegraded":"2024-03-12T04:14:58.221649+0000","last_fullsized":"2024-03-12T04:14:58.221649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:17.350636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59508,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":181,"num_write":37,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.930949+0000","last_change":"2024-03-12T04:12:54.783671+0000","last_active":"2024-03-12T04:14:14.930949+0000","last_peered":"2024-03-12T04:14:14.930949+0000","last_clean":"2024-03-12T04:14:14.930949+0000","last_became_active":"2024-03-12T04:12:54.782919+0000","last_became_peered":"2024-03-12T04:12:54.782919+0000","last_unstale":"2024-03-12T04:14:14.930949+0000","last_undegraded":"2024-03-12T04:14:14.930949+0000","last_fullsized":"2024-03-12T04:14:14.930949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:38:19.445096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.948316+0000","last_change":"2024-03-12T04:13:45.947951+0000","last_active":"2024-03-12T04:13:45.948316+0000","last_peered":"2024-03-12T04:13:45.948316+0000","last_clean":"2024-03-12T04:13:45.948316+0000","last_became_active":"2024-03-12T04:13:45.938099+0000","last_became_peered":"2024-03-12T04:13:45.938099+0000","last_unstale":"2024-03-12T04:13:45.948316+0000","last_undegraded":"2024-03-12T04:13:45.948316+0000","last_fullsized":"2024-03-12T04:13:45.948316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:03.905076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.943052+0000","last_change":"2024-03-12T04:12:54.779319+0000","last_active":"2024-03-12T04:14:14.943052+0000","last_peered":"2024-03-12T04:14:14.943052+0000","last_clean":"2024-03-12T04:14:14.943052+0000","last_became_active":"2024-03-12T04:12:54.778927+0000","last_became_peered":"2024-03-12T04:12:54.778927+0000","last_unstale":"2024-03-12T04:14:14.943052+0000","last_undegraded":"2024-03-12T04:14:14.943052+0000","last_fullsized":"2024-03-12T04:14:14.943052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:20:53.947257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934971+0000","last_change":"2024-03-12T04:12:53.774754+0000","last_active":"2024-03-12T04:13:45.934971+0000","last_peered":"2024-03-12T04:13:45.934971+0000","last_clean":"2024-03-12T04:13:45.934971+0000","last_became_active":"2024-03-12T04:12:53.774395+0000","last_became_peered":"2024-03-12T04:12:53.774395+0000","last_unstale":"2024-03-12T04:13:45.934971+0000","last_undegraded":"2024-03-12T04:13:45.934971+0000","last_fullsized":"2024-03-12T04:13:45.934971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:13.286204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946653+0000","last_change":"2024-03-12T04:12:54.786891+0000","last_active":"2024-03-12T04:14:14.946653+0000","last_peered":"2024-03-12T04:14:14.946653+0000","last_clean":"2024-03-12T04:14:14.946653+0000","last_became_active":"2024-03-12T04:12:54.786591+0000","last_became_peered":"2024-03-12T04:12:54.786591+0000","last_unstale":"2024-03-12T04:14:14.946653+0000","last_undegraded":"2024-03-12T04:14:14.946653+0000","last_fullsized":"2024-03-12T04:14:14.946653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:16:09.188557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.187545+0000","last_change":"2024-03-12T04:12:53.769839+0000","last_active":"2024-03-12T04:14:54.187545+0000","last_peered":"2024-03-12T04:14:54.187545+0000","last_clean":"2024-03-12T04:14:54.187545+0000","last_became_active":"2024-03-12T04:12:53.769486+0000","last_became_peered":"2024-03-12T04:12:53.769486+0000","last_unstale":"2024-03-12T04:14:54.187545+0000","last_undegraded":"2024-03-12T04:14:54.187545+0000","last_fullsized":"2024-03-12T04:14:54.187545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:46.050018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945596+0000","last_change":"2024-03-12T04:12:54.782447+0000","last_active":"2024-03-12T04:14:14.945596+0000","last_peered":"2024-03-12T04:14:14.945596+0000","last_clean":"2024-03-12T04:14:14.945596+0000","last_became_active":"2024-03-12T04:12:54.782091+0000","last_became_peered":"2024-03-12T04:12:54.782091+0000","last_unstale":"2024-03-12T04:14:14.945596+0000","last_undegraded":"2024-03-12T04:14:14.945596+0000","last_fullsized":"2024-03-12T04:14:14.945596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:22.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.959032+0000","last_change":"2024-03-12T04:13:45.958615+0000","last_active":"2024-03-12T04:13:45.959032+0000","last_peered":"2024-03-12T04:13:45.959032+0000","last_clean":"2024-03-12T04:13:45.959032+0000","last_became_active":"2024-03-12T04:13:45.938341+0000","last_became_peered":"2024-03-12T04:13:45.938341+0000","last_unstale":"2024-03-12T04:13:45.959032+0000","last_undegraded":"2024-03-12T04:13:45.959032+0000","last_fullsized":"2024-03-12T04:13:45.959032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:17:44.688344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.951853+0000","last_change":"2024-03-12T04:12:54.784282+0000","last_active":"2024-03-12T04:14:14.951853+0000","last_peered":"2024-03-12T04:14:14.951853+0000","last_clean":"2024-03-12T04:14:14.951853+0000","last_became_active":"2024-03-12T04:12:54.783953+0000","last_became_peered":"2024-03-12T04:12:54.783953+0000","last_unstale":"2024-03-12T04:14:14.951853+0000","last_undegraded":"2024-03-12T04:14:14.951853+0000","last_fullsized":"2024-03-12T04:14:14.951853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:14:02.182546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937316+0000","last_change":"2024-03-12T04:12:53.770230+0000","last_active":"2024-03-12T04:13:45.937316+0000","last_peered":"2024-03-12T04:13:45.937316+0000","last_clean":"2024-03-12T04:13:45.937316+0000","last_became_active":"2024-03-12T04:12:53.768817+0000","last_became_peered":"2024-03-12T04:12:53.768817+0000","last_unstale":"2024-03-12T04:13:45.937316+0000","last_undegraded":"2024-03-12T04:13:45.937316+0000","last_fullsized":"2024-03-12T04:13:45.937316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:09:12.656489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940483+0000","last_change":"2024-03-12T04:12:54.785654+0000","last_active":"2024-03-12T04:14:14.940483+0000","last_peered":"2024-03-12T04:14:14.940483+0000","last_clean":"2024-03-12T04:14:14.940483+0000","last_became_active":"2024-03-12T04:12:54.785244+0000","last_became_peered":"2024-03-12T04:12:54.785244+0000","last_unstale":"2024-03-12T04:14:14.940483+0000","last_undegraded":"2024-03-12T04:14:14.940483+0000","last_fullsized":"2024-03-12T04:14:14.940483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:00:41.961637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930880+0000","last_change":"2024-03-12T04:12:53.780962+0000","last_active":"2024-03-12T04:13:45.930880+0000","last_peered":"2024-03-12T04:13:45.930880+0000","last_clean":"2024-03-12T04:13:45.930880+0000","last_became_active":"2024-03-12T04:12:53.780623+0000","last_became_peered":"2024-03-12T04:12:53.780623+0000","last_unstale":"2024-03-12T04:13:45.930880+0000","last_undegraded":"2024-03-12T04:13:45.930880+0000","last_fullsized":"2024-03-12T04:13:45.930880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946481+0000","last_change":"2024-03-12T04:12:54.775065+0000","last_active":"2024-03-12T04:14:14.946481+0000","last_peered":"2024-03-12T04:14:14.946481+0000","last_clean":"2024-03-12T04:14:14.946481+0000","last_became_active":"2024-03-12T04:12:54.774686+0000","last_became_peered":"2024-03-12T04:12:54.774686+0000","last_unstale":"2024-03-12T04:14:14.946481+0000","last_undegraded":"2024-03-12T04:14:14.946481+0000","last_fullsized":"2024-03-12T04:14:14.946481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:56:25.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:55.189811+0000","last_change":"2024-03-12T04:12:53.777702+0000","last_active":"2024-03-12T04:14:55.189811+0000","last_peered":"2024-03-12T04:14:55.189811+0000","last_clean":"2024-03-12T04:14:55.189811+0000","last_became_active":"2024-03-12T04:12:53.777346+0000","last_became_peered":"2024-03-12T04:12:53.777346+0000","last_unstale":"2024-03-12T04:14:55.189811+0000","last_undegraded":"2024-03-12T04:14:55.189811+0000","last_fullsized":"2024-03-12T04:14:55.189811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:09.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939910+0000","last_change":"2024-03-12T04:12:54.789087+0000","last_active":"2024-03-12T04:14:14.939910+0000","last_peered":"2024-03-12T04:14:14.939910+0000","last_clean":"2024-03-12T04:14:14.939910+0000","last_became_active":"2024-03-12T04:12:54.788735+0000","last_became_peered":"2024-03-12T04:12:54.788735+0000","last_unstale":"2024-03-12T04:14:14.939910+0000","last_undegraded":"2024-03-12T04:14:14.939910+0000","last_fullsized":"2024-03-12T04:14:14.939910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:14.947137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945695+0000","last_change":"2024-03-12T04:12:54.789974+0000","last_active":"2024-03-12T04:14:14.945695+0000","last_peered":"2024-03-12T04:14:14.945695+0000","last_clean":"2024-03-12T04:14:14.945695+0000","last_became_active":"2024-03-12T04:12:54.789578+0000","last_became_peered":"2024-03-12T04:12:54.789578+0000","last_unstale":"2024-03-12T04:14:14.945695+0000","last_undegraded":"2024-03-12T04:14:14.945695+0000","last_fullsized":"2024-03-12T04:14:14.945695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:22:25.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.224216+0000","last_change":"2024-03-12T04:12:53.773601+0000","last_active":"2024-03-12T04:14:54.224216+0000","last_peered":"2024-03-12T04:14:54.224216+0000","last_clean":"2024-03-12T04:14:54.224216+0000","last_became_active":"2024-03-12T04:12:53.773264+0000","last_became_peered":"2024-03-12T04:12:53.773264+0000","last_unstale":"2024-03-12T04:14:54.224216+0000","last_undegraded":"2024-03-12T04:14:54.224216+0000","last_fullsized":"2024-03-12T04:14:54.224216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:07:03.580157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948721+0000","last_change":"2024-03-12T04:12:54.782496+0000","last_active":"2024-03-12T04:14:14.948721+0000","last_peered":"2024-03-12T04:14:14.948721+0000","last_clean":"2024-03-12T04:14:14.948721+0000","last_became_active":"2024-03-12T04:12:54.782014+0000","last_became_peered":"2024-03-12T04:12:54.782014+0000","last_unstale":"2024-03-12T04:14:14.948721+0000","last_undegraded":"2024-03-12T04:14:14.948721+0000","last_fullsized":"2024-03-12T04:14:14.948721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:08.580655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099322+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:46.099322+0000","last_peered":"2024-03-12T04:13:46.099322+0000","last_clean":"2024-03-12T04:13:46.099322+0000","last_became_active":"2024-03-12T04:12:53.770855+0000","last_became_peered":"2024-03-12T04:12:53.770855+0000","last_unstale":"2024-03-12T04:13:46.099322+0000","last_undegraded":"2024-03-12T04:13:46.099322+0000","last_fullsized":"2024-03-12T04:13:46.099322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:32:33.969240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100306+0000","last_change":"2024-03-12T04:11:37.642780+0000","last_active":"2024-03-12T04:13:46.100306+0000","last_peered":"2024-03-12T04:13:46.100306+0000","last_clean":"2024-03-12T04:13:46.100306+0000","last_became_active":"2024-03-12T04:11:37.642145+0000","last_became_peered":"2024-03-12T04:11:37.642145+0000","last_unstale":"2024-03-12T04:13:46.100306+0000","last_undegraded":"2024-03-12T04:13:46.100306+0000","last_fullsized":"2024-03-12T04:13:46.100306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_clean_scrub_stamp":"2024-03-12T04:08:09.604393+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:46:36.666483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935254+0000","last_change":"2024-03-12T04:12:54.790464+0000","last_active":"2024-03-12T04:14:14.935254+0000","last_peered":"2024-03-12T04:14:14.935254+0000","last_clean":"2024-03-12T04:14:14.935254+0000","last_became_active":"2024-03-12T04:12:54.788440+0000","last_became_peered":"2024-03-12T04:12:54.788440+0000","last_unstale":"2024-03-12T04:14:14.935254+0000","last_undegraded":"2024-03-12T04:14:14.935254+0000","last_fullsized":"2024-03-12T04:14:14.935254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:54.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100152+0000","last_change":"2024-03-12T04:12:53.773334+0000","last_active":"2024-03-12T04:13:46.100152+0000","last_peered":"2024-03-12T04:13:46.100152+0000","last_clean":"2024-03-12T04:13:46.100152+0000","last_became_active":"2024-03-12T04:12:53.772759+0000","last_became_peered":"2024-03-12T04:12:53.772759+0000","last_unstale":"2024-03-12T04:13:46.100152+0000","last_undegraded":"2024-03-12T04:13:46.100152+0000","last_fullsized":"2024-03-12T04:13:46.100152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:03:17.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937169+0000","last_change":"2024-03-12T04:12:53.777035+0000","last_active":"2024-03-12T04:13:45.937169+0000","last_peered":"2024-03-12T04:13:45.937169+0000","last_clean":"2024-03-12T04:13:45.937169+0000","last_became_active":"2024-03-12T04:12:53.776587+0000","last_became_peered":"2024-03-12T04:12:53.776587+0000","last_unstale":"2024-03-12T04:13:45.937169+0000","last_undegraded":"2024-03-12T04:13:45.937169+0000","last_fullsized":"2024-03-12T04:13:45.937169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:21.390525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.934802+0000","last_change":"2024-03-12T04:12:54.790835+0000","last_active":"2024-03-12T04:14:14.934802+0000","last_peered":"2024-03-12T04:14:14.934802+0000","last_clean":"2024-03-12T04:14:14.934802+0000","last_became_active":"2024-03-12T04:12:54.790512+0000","last_became_peered":"2024-03-12T04:12:54.790512+0000","last_unstale":"2024-03-12T04:14:14.934802+0000","last_undegraded":"2024-03-12T04:14:14.934802+0000","last_fullsized":"2024-03-12T04:14:14.934802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:37:58.029450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931069+0000","last_change":"2024-03-12T04:12:53.774604+0000","last_active":"2024-03-12T04:13:45.931069+0000","last_peered":"2024-03-12T04:13:45.931069+0000","last_clean":"2024-03-12T04:13:45.931069+0000","last_became_active":"2024-03-12T04:12:53.774270+0000","last_became_peered":"2024-03-12T04:12:53.774270+0000","last_unstale":"2024-03-12T04:13:45.931069+0000","last_undegraded":"2024-03-12T04:13:45.931069+0000","last_fullsized":"2024-03-12T04:13:45.931069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:23.635179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946179+0000","last_change":"2024-03-12T04:12:54.786259+0000","last_active":"2024-03-12T04:14:14.946179+0000","last_peered":"2024-03-12T04:14:14.946179+0000","last_clean":"2024-03-12T04:14:14.946179+0000","last_became_active":"2024-03-12T04:12:54.783114+0000","last_became_peered":"2024-03-12T04:12:54.783114+0000","last_unstale":"2024-03-12T04:14:14.946179+0000","last_undegraded":"2024-03-12T04:14:14.946179+0000","last_fullsized":"2024-03-12T04:14:14.946179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:53.036505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099781+0000","last_change":"2024-03-12T04:12:53.779618+0000","last_active":"2024-03-12T04:13:46.099781+0000","last_peered":"2024-03-12T04:13:46.099781+0000","last_clean":"2024-03-12T04:13:46.099781+0000","last_became_active":"2024-03-12T04:12:53.779264+0000","last_became_peered":"2024-03-12T04:12:53.779264+0000","last_unstale":"2024-03-12T04:13:46.099781+0000","last_undegraded":"2024-03-12T04:13:46.099781+0000","last_fullsized":"2024-03-12T04:13:46.099781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:57.633014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945120+0000","last_change":"2024-03-12T04:12:54.789012+0000","last_active":"2024-03-12T04:14:14.945120+0000","last_peered":"2024-03-12T04:14:14.945120+0000","last_clean":"2024-03-12T04:14:14.945120+0000","last_became_active":"2024-03-12T04:12:54.788441+0000","last_became_peered":"2024-03-12T04:12:54.788441+0000","last_unstale":"2024-03-12T04:14:14.945120+0000","last_undegraded":"2024-03-12T04:14:14.945120+0000","last_fullsized":"2024-03-12T04:14:14.945120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:55.858926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102816+0000","last_change":"2024-03-12T04:12:53.776621+0000","last_active":"2024-03-12T04:13:46.102816+0000","last_peered":"2024-03-12T04:13:46.102816+0000","last_clean":"2024-03-12T04:13:46.102816+0000","last_became_active":"2024-03-12T04:12:53.776313+0000","last_became_peered":"2024-03-12T04:12:53.776313+0000","last_unstale":"2024-03-12T04:13:46.102816+0000","last_undegraded":"2024-03-12T04:13:46.102816+0000","last_fullsized":"2024-03-12T04:13:46.102816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:43.536159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.900148+0000","last_change":"2024-03-12T04:12:54.786929+0000","last_active":"2024-03-12T04:14:14.900148+0000","last_peered":"2024-03-12T04:14:14.900148+0000","last_clean":"2024-03-12T04:14:14.900148+0000","last_became_active":"2024-03-12T04:12:54.786580+0000","last_became_peered":"2024-03-12T04:12:54.786580+0000","last_unstale":"2024-03-12T04:14:14.900148+0000","last_undegraded":"2024-03-12T04:14:14.900148+0000","last_fullsized":"2024-03-12T04:14:14.900148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:58.955293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935801+0000","last_change":"2024-03-12T04:12:53.778577+0000","last_active":"2024-03-12T04:13:45.935801+0000","last_peered":"2024-03-12T04:13:45.935801+0000","last_clean":"2024-03-12T04:13:45.935801+0000","last_became_active":"2024-03-12T04:12:53.778033+0000","last_became_peered":"2024-03-12T04:12:53.778033+0000","last_unstale":"2024-03-12T04:13:45.935801+0000","last_undegraded":"2024-03-12T04:13:45.935801+0000","last_fullsized":"2024-03-12T04:13:45.935801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:02.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939536+0000","last_change":"2024-03-12T04:12:54.785861+0000","last_active":"2024-03-12T04:14:14.939536+0000","last_peered":"2024-03-12T04:14:14.939536+0000","last_clean":"2024-03-12T04:14:14.939536+0000","last_became_active":"2024-03-12T04:12:54.784682+0000","last_became_peered":"2024-03-12T04:12:54.784682+0000","last_unstale":"2024-03-12T04:14:14.939536+0000","last_undegraded":"2024-03-12T04:14:14.939536+0000","last_fullsized":"2024-03-12T04:14:14.939536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:35:12.288309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931757+0000","last_change":"2024-03-12T04:12:53.772865+0000","last_active":"2024-03-12T04:13:45.931757+0000","last_peered":"2024-03-12T04:13:45.931757+0000","last_clean":"2024-03-12T04:13:45.931757+0000","last_became_active":"2024-03-12T04:12:53.772502+0000","last_became_peered":"2024-03-12T04:12:53.772502+0000","last_unstale":"2024-03-12T04:13:45.931757+0000","last_undegraded":"2024-03-12T04:13:45.931757+0000","last_fullsized":"2024-03-12T04:13:45.931757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:40:24.112197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.942329+0000","last_change":"2024-03-12T04:12:54.780722+0000","last_active":"2024-03-12T04:14:14.942329+0000","last_peered":"2024-03-12T04:14:14.942329+0000","last_clean":"2024-03-12T04:14:14.942329+0000","last_became_active":"2024-03-12T04:12:54.780404+0000","last_became_peered":"2024-03-12T04:12:54.780404+0000","last_unstale":"2024-03-12T04:14:14.942329+0000","last_undegraded":"2024-03-12T04:14:14.942329+0000","last_fullsized":"2024-03-12T04:14:14.942329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.068094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936882+0000","last_change":"2024-03-12T04:12:53.771471+0000","last_active":"2024-03-12T04:13:45.936882+0000","last_peered":"2024-03-12T04:13:45.936882+0000","last_clean":"2024-03-12T04:13:45.936882+0000","last_became_active":"2024-03-12T04:12:53.771147+0000","last_became_peered":"2024-03-12T04:12:53.771147+0000","last_unstale":"2024-03-12T04:13:45.936882+0000","last_undegraded":"2024-03-12T04:13:45.936882+0000","last_fullsized":"2024-03-12T04:13:45.936882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:46.159168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949687+0000","last_change":"2024-03-12T04:12:54.796253+0000","last_active":"2024-03-12T04:14:14.949687+0000","last_peered":"2024-03-12T04:14:14.949687+0000","last_clean":"2024-03-12T04:14:14.949687+0000","last_became_active":"2024-03-12T04:12:54.795656+0000","last_became_peered":"2024-03-12T04:12:54.795656+0000","last_unstale":"2024-03-12T04:14:14.949687+0000","last_undegraded":"2024-03-12T04:14:14.949687+0000","last_fullsized":"2024-03-12T04:14:14.949687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:39:32.788874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.198832+0000","last_change":"2024-03-12T04:12:53.780257+0000","last_active":"2024-03-12T04:14:54.198832+0000","last_peered":"2024-03-12T04:14:54.198832+0000","last_clean":"2024-03-12T04:14:54.198832+0000","last_became_active":"2024-03-12T04:12:53.779891+0000","last_became_peered":"2024-03-12T04:12:53.779891+0000","last_unstale":"2024-03-12T04:14:54.198832+0000","last_undegraded":"2024-03-12T04:14:54.198832+0000","last_fullsized":"2024-03-12T04:14:54.198832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:49:30.218372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.950676+0000","last_change":"2024-03-12T04:12:54.792760+0000","last_active":"2024-03-12T04:14:14.950676+0000","last_peered":"2024-03-12T04:14:14.950676+0000","last_clean":"2024-03-12T04:14:14.950676+0000","last_became_active":"2024-03-12T04:12:54.792363+0000","last_became_peered":"2024-03-12T04:12:54.792363+0000","last_unstale":"2024-03-12T04:14:14.950676+0000","last_undegraded":"2024-03-12T04:14:14.950676+0000","last_fullsized":"2024-03-12T04:14:14.950676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:30:15.394318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935702+0000","last_change":"2024-03-12T04:12:53.775556+0000","last_active":"2024-03-12T04:13:45.935702+0000","last_peered":"2024-03-12T04:13:45.935702+0000","last_clean":"2024-03-12T04:13:45.935702+0000","last_became_active":"2024-03-12T04:12:53.775182+0000","last_became_peered":"2024-03-12T04:12:53.775182+0000","last_unstale":"2024-03-12T04:13:45.935702+0000","last_undegraded":"2024-03-12T04:13:45.935702+0000","last_fullsized":"2024-03-12T04:13:45.935702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:13:50.076672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948544+0000","last_change":"2024-03-12T04:12:54.795876+0000","last_active":"2024-03-12T04:14:14.948544+0000","last_peered":"2024-03-12T04:14:14.948544+0000","last_clean":"2024-03-12T04:14:14.948544+0000","last_became_active":"2024-03-12T04:12:54.793874+0000","last_became_peered":"2024-03-12T04:12:54.793874+0000","last_unstale":"2024-03-12T04:14:14.948544+0000","last_undegraded":"2024-03-12T04:14:14.948544+0000","last_fullsized":"2024-03-12T04:14:14.948544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:33:31.165974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933769+0000","last_change":"2024-03-12T04:12:53.774879+0000","last_active":"2024-03-12T04:13:45.933769+0000","last_peered":"2024-03-12T04:13:45.933769+0000","last_clean":"2024-03-12T04:13:45.933769+0000","last_became_active":"2024-03-12T04:12:53.774530+0000","last_became_peered":"2024-03-12T04:12:53.774530+0000","last_unstale":"2024-03-12T04:13:45.933769+0000","last_undegraded":"2024-03-12T04:13:45.933769+0000","last_fullsized":"2024-03-12T04:13:45.933769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:25.858165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946946+0000","last_change":"2024-03-12T04:12:54.790140+0000","last_active":"2024-03-12T04:14:14.946946+0000","last_peered":"2024-03-12T04:14:14.946946+0000","last_clean":"2024-03-12T04:14:14.946946+0000","last_became_active":"2024-03-12T04:12:54.789798+0000","last_became_peered":"2024-03-12T04:12:54.789798+0000","last_unstale":"2024-03-12T04:14:14.946946+0000","last_undegraded":"2024-03-12T04:14:14.946946+0000","last_fullsized":"2024-03-12T04:14:14.946946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:12:38.722638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.936334+0000","last_change":"2024-03-12T04:12:54.785029+0000","last_active":"2024-03-12T04:14:14.936334+0000","last_peered":"2024-03-12T04:14:14.936334+0000","last_clean":"2024-03-12T04:14:14.936334+0000","last_became_active":"2024-03-12T04:12:54.784197+0000","last_became_peered":"2024-03-12T04:12:54.784197+0000","last_unstale":"2024-03-12T04:14:14.936334+0000","last_undegraded":"2024-03-12T04:14:14.936334+0000","last_fullsized":"2024-03-12T04:14:14.936334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:22:41.352093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.937779+0000","last_change":"2024-03-12T04:12:54.793132+0000","last_active":"2024-03-12T04:14:14.937779+0000","last_peered":"2024-03-12T04:14:14.937779+0000","last_clean":"2024-03-12T04:14:14.937779+0000","last_became_active":"2024-03-12T04:12:54.792838+0000","last_became_peered":"2024-03-12T04:12:54.792838+0000","last_unstale":"2024-03-12T04:14:14.937779+0000","last_undegraded":"2024-03-12T04:14:14.937779+0000","last_fullsized":"2024-03-12T04:14:14.937779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:25:22.641963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935192+0000","last_change":"2024-03-12T04:13:45.938126+0000","last_active":"2024-03-12T04:14:14.935192+0000","last_peered":"2024-03-12T04:14:14.935192+0000","last_clean":"2024-03-12T04:14:14.935192+0000","last_became_active":"2024-03-12T04:13:45.937698+0000","last_became_peered":"2024-03-12T04:13:45.937698+0000","last_unstale":"2024-03-12T04:14:14.935192+0000","last_undegraded":"2024-03-12T04:14:14.935192+0000","last_fullsized":"2024-03-12T04:14:14.935192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:10:25.378634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944252+0000","last_change":"2024-03-12T04:12:54.785830+0000","last_active":"2024-03-12T04:14:14.944252+0000","last_peered":"2024-03-12T04:14:14.944252+0000","last_clean":"2024-03-12T04:14:14.944252+0000","last_became_active":"2024-03-12T04:12:54.785087+0000","last_became_peered":"2024-03-12T04:12:54.785087+0000","last_unstale":"2024-03-12T04:14:14.944252+0000","last_undegraded":"2024-03-12T04:14:14.944252+0000","last_fullsized":"2024-03-12T04:14:14.944252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:08:09.860311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944035+0000","last_change":"2024-03-12T04:13:45.937883+0000","last_active":"2024-03-12T04:14:14.944035+0000","last_peered":"2024-03-12T04:14:14.944035+0000","last_clean":"2024-03-12T04:14:14.944035+0000","last_became_active":"2024-03-12T04:13:45.937471+0000","last_became_peered":"2024-03-12T04:13:45.937471+0000","last_unstale":"2024-03-12T04:14:14.944035+0000","last_undegraded":"2024-03-12T04:14:14.944035+0000","last_fullsized":"2024-03-12T04:14:14.944035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:28:21.997033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.932702+0000","last_change":"2024-03-12T04:13:45.938864+0000","last_active":"2024-03-12T04:14:14.932702+0000","last_peered":"2024-03-12T04:14:14.932702+0000","last_clean":"2024-03-12T04:14:14.932702+0000","last_became_active":"2024-03-12T04:13:45.938336+0000","last_became_peered":"2024-03-12T04:13:45.938336+0000","last_unstale":"2024-03-12T04:14:14.932702+0000","last_undegraded":"2024-03-12T04:14:14.932702+0000","last_fullsized":"2024-03-12T04:14:14.932702+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:44.990282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.938228+0000","last_change":"2024-03-12T04:12:54.796733+0000","last_active":"2024-03-12T04:14:14.938228+0000","last_peered":"2024-03-12T04:14:14.938228+0000","last_clean":"2024-03-12T04:14:14.938228+0000","last_became_active":"2024-03-12T04:12:54.796388+0000","last_became_peered":"2024-03-12T04:12:54.796388+0000","last_unstale":"2024-03-12T04:14:14.938228+0000","last_undegraded":"2024-03-12T04:14:14.938228+0000","last_fullsized":"2024-03-12T04:14:14.938228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:01.580286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.947274+0000","last_change":"2024-03-12T04:12:54.787200+0000","last_active":"2024-03-12T04:14:14.947274+0000","last_peered":"2024-03-12T04:14:14.947274+0000","last_clean":"2024-03-12T04:14:14.947274+0000","last_became_active":"2024-03-12T04:12:54.786657+0000","last_became_peered":"2024-03-12T04:12:54.786657+0000","last_unstale":"2024-03-12T04:14:14.947274+0000","last_undegraded":"2024-03-12T04:14:14.947274+0000","last_fullsized":"2024-03-12T04:14:14.947274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:15:35.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.943388+0000","last_change":"2024-03-12T04:12:54.785132+0000","last_active":"2024-03-12T04:14:14.943388+0000","last_peered":"2024-03-12T04:14:14.943388+0000","last_clean":"2024-03-12T04:14:14.943388+0000","last_became_active":"2024-03-12T04:12:54.782548+0000","last_became_peered":"2024-03-12T04:12:54.782548+0000","last_unstale":"2024-03-12T04:14:14.943388+0000","last_undegraded":"2024-03-12T04:14:14.943388+0000","last_fullsized":"2024-03-12T04:14:14.943388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:40:49.284778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948256+0000","last_change":"2024-03-12T04:12:54.790227+0000","last_active":"2024-03-12T04:14:14.948256+0000","last_peered":"2024-03-12T04:14:14.948256+0000","last_clean":"2024-03-12T04:14:14.948256+0000","last_became_active":"2024-03-12T04:12:54.789853+0000","last_became_peered":"2024-03-12T04:12:54.789853+0000","last_unstale":"2024-03-12T04:14:14.948256+0000","last_undegraded":"2024-03-12T04:14:14.948256+0000","last_fullsized":"2024-03-12T04:14:14.948256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:38:16.683661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953923+0000","last_change":"2024-03-12T04:12:54.796039+0000","last_active":"2024-03-12T04:14:14.953923+0000","last_peered":"2024-03-12T04:14:14.953923+0000","last_clean":"2024-03-12T04:14:14.953923+0000","last_became_active":"2024-03-12T04:12:54.795568+0000","last_became_peered":"2024-03-12T04:12:54.795568+0000","last_unstale":"2024-03-12T04:14:14.953923+0000","last_undegraded":"2024-03-12T04:14:14.953923+0000","last_fullsized":"2024-03-12T04:14:14.953923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:46.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946286+0000","last_change":"2024-03-12T04:12:54.785610+0000","last_active":"2024-03-12T04:14:14.946286+0000","last_peered":"2024-03-12T04:14:14.946286+0000","last_clean":"2024-03-12T04:14:14.946286+0000","last_became_active":"2024-03-12T04:12:54.785256+0000","last_became_peered":"2024-03-12T04:12:54.785256+0000","last_unstale":"2024-03-12T04:14:14.946286+0000","last_undegraded":"2024-03-12T04:14:14.946286+0000","last_fullsized":"2024-03-12T04:14:14.946286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:34.152361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940458+0000","last_change":"2024-03-12T04:12:54.787064+0000","last_active":"2024-03-12T04:14:14.940458+0000","last_peered":"2024-03-12T04:14:14.940458+0000","last_clean":"2024-03-12T04:14:14.940458+0000","last_became_active":"2024-03-12T04:12:54.786701+0000","last_became_peered":"2024-03-12T04:12:54.786701+0000","last_unstale":"2024-03-12T04:14:14.940458+0000","last_undegraded":"2024-03-12T04:14:14.940458+0000","last_fullsized":"2024-03-12T04:14:14.940458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:44:43.582171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250964,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1208,"num_read_kb":681,"num_write":257,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":206898,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":468541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":1119430,"data_compressed":38179,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":467798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":468064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1062846,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":468008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":528429,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":469347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":528127,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":467340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":468401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1060513,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61753,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62930,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62269,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T04:15:00.533 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-12T04:15:00.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T04:15:00.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 -- 172.21.15.42:0/1975081642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 msgr2=0x7fc960109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 --2- 172.21.15.42:0/1975081642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc960109610 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc948009900 tx=0x7fc94802bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:00.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 -- 172.21.15.42:0/1975081642 shutdown_connections 2024-03-12T04:15:00.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 --2- 172.21.15.42:0/1975081642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc960109610 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 --2- 172.21.15.42:0/1975081642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9601023c0 0x7fc960102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 --2- 172.21.15.42:0/1975081642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9601011d0 0x7fc9601015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 -- 172.21.15.42:0/1975081642 >> 172.21.15.42:0/1975081642 conn(0x7fc9600fc720 msgr2=0x7fc9600feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 -- 172.21.15.42:0/1975081642 shutdown_connections 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.669+0000 7fc967096640 1 -- 172.21.15.42:0/1975081642 wait complete. 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 Processor -- start 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 -- start start 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9601011d0 0x7fc960197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9601023c0 0x7fc9601976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc96019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96019ec70 con 0x7fc960102d80 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96019ede0 con 0x7fc9601023c0 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc967096640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96019efa0 con 0x7fc9601011d0 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc965633640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc96019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc965633640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc96019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50116/0 (socket says 172.21.15.42:50116) 2024-03-12T04:15:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc965633640 1 -- 172.21.15.42:0/377244195 learned_addr learned my addr 172.21.15.42:0/377244195 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc964e32640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9601011d0 0x7fc960197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.670+0000 7fc95ffff640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9601023c0 0x7fc9601976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc965633640 1 -- 172.21.15.42:0/377244195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9601011d0 msgr2=0x7fc960197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc965633640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9601011d0 0x7fc960197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc965633640 1 -- 172.21.15.42:0/377244195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9601023c0 msgr2=0x7fc9601976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc965633640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9601023c0 0x7fc9601976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc965633640 1 -- 172.21.15.42:0/377244195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948009560 con 0x7fc960102d80 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc965633640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc96019e730 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc948005ab0 tx=0x7fc9480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc95dffb640 1 -- 172.21.15.42:0/377244195 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480044e0 con 0x7fc960102d80 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc95dffb640 1 -- 172.21.15.42:0/377244195 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc94802de90 con 0x7fc960102d80 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96019f220 con 0x7fc960102d80 2024-03-12T04:15:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc95dffb640 1 -- 172.21.15.42:0/377244195 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948005560 con 0x7fc960102d80 2024-03-12T04:15:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.671+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96019f800 con 0x7fc960102d80 2024-03-12T04:15:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.672+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9601084f0 con 0x7fc960102d80 2024-03-12T04:15:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.673+0000 7fc95dffb640 1 -- 172.21.15.42:0/377244195 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc948004dc0 con 0x7fc960102d80 2024-03-12T04:15:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.674+0000 7fc95dffb640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc94c0a1ab0 0x7fc94c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.674+0000 7fc95dffb640 1 -- 172.21.15.42:0/377244195 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc9480ef6b0 con 0x7fc960102d80 2024-03-12T04:15:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.674+0000 7fc964e32640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc94c0a1ab0 0x7fc94c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.674+0000 7fc964e32640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc94c0a1ab0 0x7fc94c0a3f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc960102220 tx=0x7fc95800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:00.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.678+0000 7fc95dffb640 1 -- 172.21.15.42:0/377244195 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9480b88f0 con 0x7fc960102d80 2024-03-12T04:15:00.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.857+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc96002ce10 con 0x7fc94c0a1ab0 2024-03-12T04:15:00.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.864+0000 7fc95dffb640 1 -- 172.21.15.42:0/377244195 <== mgr.14150 v2:172.21.15.42:6800/2757555113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259100 (secure 0 0 0) 0x7fc96002ce10 con 0x7fc94c0a1ab0 2024-03-12T04:15:00.870 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:15:00.871 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc94c0a1ab0 msgr2=0x7fc94c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc94c0a1ab0 0x7fc94c0a3f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc960102220 tx=0x7fc95800b390 comp rx=0 tx=0).stop 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 msgr2=0x7fc96019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc96019e730 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc948005ab0 tx=0x7fc9480042f0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 shutdown_connections 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102d80 0x7fc96019e730 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fc94c0a1ab0 0x7fc94c0a3f70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9601023c0 0x7fc9601976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 --2- 172.21.15.42:0/377244195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9601011d0 0x7fc960197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 >> 172.21.15.42:0/377244195 conn(0x7fc9600fc720 msgr2=0x7fc9601056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 shutdown_connections 2024-03-12T04:15:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:00.868+0000 7fc967096640 1 -- 172.21.15.42:0/377244195 wait complete. 2024-03-12T04:15:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-03-12T04:14:58.930090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1344,"num_read_kb":800,"num_write":440,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373620,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568204,"statfs":{"total":1151940427776,"available":1151557840896,"internally_reserved":0,"allocated":12275712,"data_stored":7576844,"data_compressed":114537,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":67,"num_write":11,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003272"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949806+0000","last_change":"2024-03-12T04:12:54.788665+0000","last_active":"2024-03-12T04:14:14.949806+0000","last_peered":"2024-03-12T04:14:14.949806+0000","last_clean":"2024-03-12T04:14:14.949806+0000","last_became_active":"2024-03-12T04:12:54.788334+0000","last_became_peered":"2024-03-12T04:12:54.788334+0000","last_unstale":"2024-03-12T04:14:14.949806+0000","last_undegraded":"2024-03-12T04:14:14.949806+0000","last_fullsized":"2024-03-12T04:14:14.949806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:02.442543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940407+0000","last_change":"2024-03-12T04:12:54.791942+0000","last_active":"2024-03-12T04:14:14.940407+0000","last_peered":"2024-03-12T04:14:14.940407+0000","last_clean":"2024-03-12T04:14:14.940407+0000","last_became_active":"2024-03-12T04:12:54.791579+0000","last_became_peered":"2024-03-12T04:12:54.791579+0000","last_unstale":"2024-03-12T04:14:14.940407+0000","last_undegraded":"2024-03-12T04:14:14.940407+0000","last_fullsized":"2024-03-12T04:14:14.940407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:51:08.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949288+0000","last_change":"2024-03-12T04:12:54.793238+0000","last_active":"2024-03-12T04:14:14.949288+0000","last_peered":"2024-03-12T04:14:14.949288+0000","last_clean":"2024-03-12T04:14:14.949288+0000","last_became_active":"2024-03-12T04:12:54.792877+0000","last_became_peered":"2024-03-12T04:12:54.792877+0000","last_unstale":"2024-03-12T04:14:14.949288+0000","last_undegraded":"2024-03-12T04:14:14.949288+0000","last_fullsized":"2024-03-12T04:14:14.949288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:36:46.914204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945072+0000","last_change":"2024-03-12T04:12:54.797573+0000","last_active":"2024-03-12T04:14:14.945072+0000","last_peered":"2024-03-12T04:14:14.945072+0000","last_clean":"2024-03-12T04:14:14.945072+0000","last_became_active":"2024-03-12T04:12:54.796939+0000","last_became_peered":"2024-03-12T04:12:54.796939+0000","last_unstale":"2024-03-12T04:14:14.945072+0000","last_undegraded":"2024-03-12T04:14:14.945072+0000","last_fullsized":"2024-03-12T04:14:14.945072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:34:58.047492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.912919+0000","last_change":"2024-03-12T04:12:54.780595+0000","last_active":"2024-03-12T04:14:14.912919+0000","last_peered":"2024-03-12T04:14:14.912919+0000","last_clean":"2024-03-12T04:14:14.912919+0000","last_became_active":"2024-03-12T04:12:54.780256+0000","last_became_peered":"2024-03-12T04:12:54.780256+0000","last_unstale":"2024-03-12T04:14:14.912919+0000","last_undegraded":"2024-03-12T04:14:14.912919+0000","last_fullsized":"2024-03-12T04:14:14.912919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:42.492006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935417+0000","last_change":"2024-03-12T04:13:45.934954+0000","last_active":"2024-03-12T04:14:14.935417+0000","last_peered":"2024-03-12T04:14:14.935417+0000","last_clean":"2024-03-12T04:14:14.935417+0000","last_became_active":"2024-03-12T04:13:45.934579+0000","last_became_peered":"2024-03-12T04:13:45.934579+0000","last_unstale":"2024-03-12T04:14:14.935417+0000","last_undegraded":"2024-03-12T04:14:14.935417+0000","last_fullsized":"2024-03-12T04:14:14.935417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:39:10.062304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.941999+0000","last_change":"2024-03-12T04:12:54.781510+0000","last_active":"2024-03-12T04:14:14.941999+0000","last_peered":"2024-03-12T04:14:14.941999+0000","last_clean":"2024-03-12T04:14:14.941999+0000","last_became_active":"2024-03-12T04:12:54.781141+0000","last_became_peered":"2024-03-12T04:12:54.781141+0000","last_unstale":"2024-03-12T04:14:14.941999+0000","last_undegraded":"2024-03-12T04:14:14.941999+0000","last_fullsized":"2024-03-12T04:14:14.941999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:42:05.055308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948851+0000","last_change":"2024-03-12T04:12:54.782541+0000","last_active":"2024-03-12T04:14:14.948851+0000","last_peered":"2024-03-12T04:14:14.948851+0000","last_clean":"2024-03-12T04:14:14.948851+0000","last_became_active":"2024-03-12T04:12:54.781582+0000","last_became_peered":"2024-03-12T04:12:54.781582+0000","last_unstale":"2024-03-12T04:14:14.948851+0000","last_undegraded":"2024-03-12T04:14:14.948851+0000","last_fullsized":"2024-03-12T04:14:14.948851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:15.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953459+0000","last_change":"2024-03-12T04:12:54.788201+0000","last_active":"2024-03-12T04:14:14.953459+0000","last_peered":"2024-03-12T04:14:14.953459+0000","last_clean":"2024-03-12T04:14:14.953459+0000","last_became_active":"2024-03-12T04:12:54.787875+0000","last_became_peered":"2024-03-12T04:12:54.787875+0000","last_unstale":"2024-03-12T04:14:14.953459+0000","last_undegraded":"2024-03-12T04:14:14.953459+0000","last_fullsized":"2024-03-12T04:14:14.953459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:40:39.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944262+0000","last_change":"2024-03-12T04:12:54.789748+0000","last_active":"2024-03-12T04:14:14.944262+0000","last_peered":"2024-03-12T04:14:14.944262+0000","last_clean":"2024-03-12T04:14:14.944262+0000","last_became_active":"2024-03-12T04:12:54.788838+0000","last_became_peered":"2024-03-12T04:12:54.788838+0000","last_unstale":"2024-03-12T04:14:14.944262+0000","last_undegraded":"2024-03-12T04:14:14.944262+0000","last_fullsized":"2024-03-12T04:14:14.944262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:39:51.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949656+0000","last_change":"2024-03-12T04:12:54.793101+0000","last_active":"2024-03-12T04:14:14.949656+0000","last_peered":"2024-03-12T04:14:14.949656+0000","last_clean":"2024-03-12T04:14:14.949656+0000","last_became_active":"2024-03-12T04:12:54.792488+0000","last_became_peered":"2024-03-12T04:12:54.792488+0000","last_unstale":"2024-03-12T04:14:14.949656+0000","last_undegraded":"2024-03-12T04:14:14.949656+0000","last_fullsized":"2024-03-12T04:14:14.949656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:33.862011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.942330+0000","last_change":"2024-03-12T04:12:54.797953+0000","last_active":"2024-03-12T04:14:14.942330+0000","last_peered":"2024-03-12T04:14:14.942330+0000","last_clean":"2024-03-12T04:14:14.942330+0000","last_became_active":"2024-03-12T04:12:54.797413+0000","last_became_peered":"2024-03-12T04:12:54.797413+0000","last_unstale":"2024-03-12T04:14:14.942330+0000","last_undegraded":"2024-03-12T04:14:14.942330+0000","last_fullsized":"2024-03-12T04:14:14.942330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:18:01.140872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.936082+0000","last_change":"2024-03-12T04:12:54.775570+0000","last_active":"2024-03-12T04:14:14.936082+0000","last_peered":"2024-03-12T04:14:14.936082+0000","last_clean":"2024-03-12T04:14:14.936082+0000","last_became_active":"2024-03-12T04:12:54.775186+0000","last_became_peered":"2024-03-12T04:12:54.775186+0000","last_unstale":"2024-03-12T04:14:14.936082+0000","last_undegraded":"2024-03-12T04:14:14.936082+0000","last_fullsized":"2024-03-12T04:14:14.936082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:26.004458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.934601+0000","last_change":"2024-03-12T04:12:54.780859+0000","last_active":"2024-03-12T04:14:14.934601+0000","last_peered":"2024-03-12T04:14:14.934601+0000","last_clean":"2024-03-12T04:14:14.934601+0000","last_became_active":"2024-03-12T04:12:54.780484+0000","last_became_peered":"2024-03-12T04:12:54.780484+0000","last_unstale":"2024-03-12T04:14:14.934601+0000","last_undegraded":"2024-03-12T04:14:14.934601+0000","last_fullsized":"2024-03-12T04:14:14.934601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:24:00.575324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.926041+0000","last_change":"2024-03-12T04:12:54.796087+0000","last_active":"2024-03-12T04:14:14.926041+0000","last_peered":"2024-03-12T04:14:14.926041+0000","last_clean":"2024-03-12T04:14:14.926041+0000","last_became_active":"2024-03-12T04:12:54.793702+0000","last_became_peered":"2024-03-12T04:12:54.793702+0000","last_unstale":"2024-03-12T04:14:14.926041+0000","last_undegraded":"2024-03-12T04:14:14.926041+0000","last_fullsized":"2024-03-12T04:14:14.926041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:09:27.334791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953800+0000","last_change":"2024-03-12T04:12:54.783146+0000","last_active":"2024-03-12T04:14:14.953800+0000","last_peered":"2024-03-12T04:14:14.953800+0000","last_clean":"2024-03-12T04:14:14.953800+0000","last_became_active":"2024-03-12T04:12:54.782834+0000","last_became_peered":"2024-03-12T04:12:54.782834+0000","last_unstale":"2024-03-12T04:14:14.953800+0000","last_undegraded":"2024-03-12T04:14:14.953800+0000","last_fullsized":"2024-03-12T04:14:14.953800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:35:11.126629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.950186+0000","last_change":"2024-03-12T04:12:54.794879+0000","last_active":"2024-03-12T04:14:14.950186+0000","last_peered":"2024-03-12T04:14:14.950186+0000","last_clean":"2024-03-12T04:14:14.950186+0000","last_became_active":"2024-03-12T04:12:54.794251+0000","last_became_peered":"2024-03-12T04:12:54.794251+0000","last_unstale":"2024-03-12T04:14:14.950186+0000","last_undegraded":"2024-03-12T04:14:14.950186+0000","last_fullsized":"2024-03-12T04:14:14.950186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:54:33.754426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946417+0000","last_change":"2024-03-12T04:12:54.791301+0000","last_active":"2024-03-12T04:14:14.946417+0000","last_peered":"2024-03-12T04:14:14.946417+0000","last_clean":"2024-03-12T04:14:14.946417+0000","last_became_active":"2024-03-12T04:12:54.790850+0000","last_became_peered":"2024-03-12T04:12:54.790850+0000","last_unstale":"2024-03-12T04:14:14.946417+0000","last_undegraded":"2024-03-12T04:14:14.946417+0000","last_fullsized":"2024-03-12T04:14:14.946417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:26:04.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949051+0000","last_change":"2024-03-12T04:12:54.793419+0000","last_active":"2024-03-12T04:14:14.949051+0000","last_peered":"2024-03-12T04:14:14.949051+0000","last_clean":"2024-03-12T04:14:14.949051+0000","last_became_active":"2024-03-12T04:12:54.793080+0000","last_became_peered":"2024-03-12T04:12:54.793080+0000","last_unstale":"2024-03-12T04:14:14.949051+0000","last_undegraded":"2024-03-12T04:14:14.949051+0000","last_fullsized":"2024-03-12T04:14:14.949051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:46:26.506899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:53.185317+0000","last_change":"2024-03-12T04:12:53.774893+0000","last_active":"2024-03-12T04:14:53.185317+0000","last_peered":"2024-03-12T04:14:53.185317+0000","last_clean":"2024-03-12T04:14:53.185317+0000","last_became_active":"2024-03-12T04:12:53.773801+0000","last_became_peered":"2024-03-12T04:12:53.773801+0000","last_unstale":"2024-03-12T04:14:53.185317+0000","last_undegraded":"2024-03-12T04:14:53.185317+0000","last_fullsized":"2024-03-12T04:14:53.185317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:30:19.995765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948615+0000","last_change":"2024-03-12T04:12:54.786795+0000","last_active":"2024-03-12T04:14:14.948615+0000","last_peered":"2024-03-12T04:14:14.948615+0000","last_clean":"2024-03-12T04:14:14.948615+0000","last_became_active":"2024-03-12T04:12:54.786481+0000","last_became_peered":"2024-03-12T04:12:54.786481+0000","last_unstale":"2024-03-12T04:14:14.948615+0000","last_undegraded":"2024-03-12T04:14:14.948615+0000","last_fullsized":"2024-03-12T04:14:14.948615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:06.849644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936225+0000","last_change":"2024-03-12T04:12:53.779582+0000","last_active":"2024-03-12T04:13:45.936225+0000","last_peered":"2024-03-12T04:13:45.936225+0000","last_clean":"2024-03-12T04:13:45.936225+0000","last_became_active":"2024-03-12T04:12:53.779062+0000","last_became_peered":"2024-03-12T04:12:53.779062+0000","last_unstale":"2024-03-12T04:13:45.936225+0000","last_undegraded":"2024-03-12T04:13:45.936225+0000","last_fullsized":"2024-03-12T04:13:45.936225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:44:01.065183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949313+0000","last_change":"2024-03-12T04:12:54.789198+0000","last_active":"2024-03-12T04:14:14.949313+0000","last_peered":"2024-03-12T04:14:14.949313+0000","last_clean":"2024-03-12T04:14:14.949313+0000","last_became_active":"2024-03-12T04:12:54.788213+0000","last_became_peered":"2024-03-12T04:12:54.788213+0000","last_unstale":"2024-03-12T04:14:14.949313+0000","last_undegraded":"2024-03-12T04:14:14.949313+0000","last_fullsized":"2024-03-12T04:14:14.949313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:24:36.561553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937326+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:45.937326+0000","last_peered":"2024-03-12T04:13:45.937326+0000","last_clean":"2024-03-12T04:13:45.937326+0000","last_became_active":"2024-03-12T04:12:53.771178+0000","last_became_peered":"2024-03-12T04:12:53.771178+0000","last_unstale":"2024-03-12T04:13:45.937326+0000","last_undegraded":"2024-03-12T04:13:45.937326+0000","last_fullsized":"2024-03-12T04:13:45.937326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:12.882128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953255+0000","last_change":"2024-03-12T04:12:54.799758+0000","last_active":"2024-03-12T04:14:14.953255+0000","last_peered":"2024-03-12T04:14:14.953255+0000","last_clean":"2024-03-12T04:14:14.953255+0000","last_became_active":"2024-03-12T04:12:54.799338+0000","last_became_peered":"2024-03-12T04:12:54.799338+0000","last_unstale":"2024-03-12T04:14:14.953255+0000","last_undegraded":"2024-03-12T04:14:14.953255+0000","last_fullsized":"2024-03-12T04:14:14.953255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:32:54.087890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937112+0000","last_change":"2024-03-12T04:12:53.774390+0000","last_active":"2024-03-12T04:13:45.937112+0000","last_peered":"2024-03-12T04:13:45.937112+0000","last_clean":"2024-03-12T04:13:45.937112+0000","last_became_active":"2024-03-12T04:12:53.774042+0000","last_became_peered":"2024-03-12T04:12:53.774042+0000","last_unstale":"2024-03-12T04:13:45.937112+0000","last_undegraded":"2024-03-12T04:13:45.937112+0000","last_fullsized":"2024-03-12T04:13:45.937112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:37.361505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953436+0000","last_change":"2024-03-12T04:13:46.104074+0000","last_active":"2024-03-12T04:14:14.953436+0000","last_peered":"2024-03-12T04:14:14.953436+0000","last_clean":"2024-03-12T04:14:14.953436+0000","last_became_active":"2024-03-12T04:13:46.103719+0000","last_became_peered":"2024-03-12T04:13:46.103719+0000","last_unstale":"2024-03-12T04:14:14.953436+0000","last_undegraded":"2024-03-12T04:14:14.953436+0000","last_fullsized":"2024-03-12T04:14:14.953436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:29.979576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.198832+0000","last_change":"2024-03-12T04:12:53.772182+0000","last_active":"2024-03-12T04:14:54.198832+0000","last_peered":"2024-03-12T04:14:54.198832+0000","last_clean":"2024-03-12T04:14:54.198832+0000","last_became_active":"2024-03-12T04:12:53.771858+0000","last_became_peered":"2024-03-12T04:12:53.771858+0000","last_unstale":"2024-03-12T04:14:54.198832+0000","last_undegraded":"2024-03-12T04:14:54.198832+0000","last_fullsized":"2024-03-12T04:14:54.198832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:42:41.466311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.922543+0000","last_change":"2024-03-12T04:12:54.779869+0000","last_active":"2024-03-12T04:14:14.922543+0000","last_peered":"2024-03-12T04:14:14.922543+0000","last_clean":"2024-03-12T04:14:14.922543+0000","last_became_active":"2024-03-12T04:12:54.779500+0000","last_became_peered":"2024-03-12T04:12:54.779500+0000","last_unstale":"2024-03-12T04:14:14.922543+0000","last_undegraded":"2024-03-12T04:14:14.922543+0000","last_fullsized":"2024-03-12T04:14:14.922543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:29.538570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.932326+0000","last_change":"2024-03-12T04:12:53.774056+0000","last_active":"2024-03-12T04:13:45.932326+0000","last_peered":"2024-03-12T04:13:45.932326+0000","last_clean":"2024-03-12T04:13:45.932326+0000","last_became_active":"2024-03-12T04:12:53.773675+0000","last_became_peered":"2024-03-12T04:12:53.773675+0000","last_unstale":"2024-03-12T04:13:45.932326+0000","last_undegraded":"2024-03-12T04:13:45.932326+0000","last_fullsized":"2024-03-12T04:13:45.932326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:21:33.576947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945472+0000","last_change":"2024-03-12T04:12:54.790583+0000","last_active":"2024-03-12T04:14:14.945472+0000","last_peered":"2024-03-12T04:14:14.945472+0000","last_clean":"2024-03-12T04:14:14.945472+0000","last_became_active":"2024-03-12T04:12:54.788364+0000","last_became_peered":"2024-03-12T04:12:54.788364+0000","last_unstale":"2024-03-12T04:14:14.945472+0000","last_undegraded":"2024-03-12T04:14:14.945472+0000","last_fullsized":"2024-03-12T04:14:14.945472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:05:13.542279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933518+0000","last_change":"2024-03-12T04:12:53.769619+0000","last_active":"2024-03-12T04:13:45.933518+0000","last_peered":"2024-03-12T04:13:45.933518+0000","last_clean":"2024-03-12T04:13:45.933518+0000","last_became_active":"2024-03-12T04:12:53.769293+0000","last_became_peered":"2024-03-12T04:12:53.769293+0000","last_unstale":"2024-03-12T04:13:45.933518+0000","last_undegraded":"2024-03-12T04:13:45.933518+0000","last_fullsized":"2024-03-12T04:13:45.933518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:44.618499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940327+0000","last_change":"2024-03-12T04:12:54.776493+0000","last_active":"2024-03-12T04:14:14.940327+0000","last_peered":"2024-03-12T04:14:14.940327+0000","last_clean":"2024-03-12T04:14:14.940327+0000","last_became_active":"2024-03-12T04:12:54.775967+0000","last_became_peered":"2024-03-12T04:12:54.775967+0000","last_unstale":"2024-03-12T04:14:14.940327+0000","last_undegraded":"2024-03-12T04:14:14.940327+0000","last_fullsized":"2024-03-12T04:14:14.940327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:19:46.824145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934388+0000","last_change":"2024-03-12T04:12:53.773523+0000","last_active":"2024-03-12T04:13:45.934388+0000","last_peered":"2024-03-12T04:13:45.934388+0000","last_clean":"2024-03-12T04:13:45.934388+0000","last_became_active":"2024-03-12T04:12:53.773161+0000","last_became_peered":"2024-03-12T04:12:53.773161+0000","last_unstale":"2024-03-12T04:13:45.934388+0000","last_undegraded":"2024-03-12T04:13:45.934388+0000","last_fullsized":"2024-03-12T04:13:45.934388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:41:40.810994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939744+0000","last_change":"2024-03-12T04:12:54.791688+0000","last_active":"2024-03-12T04:14:14.939744+0000","last_peered":"2024-03-12T04:14:14.939744+0000","last_clean":"2024-03-12T04:14:14.939744+0000","last_became_active":"2024-03-12T04:12:54.790854+0000","last_became_peered":"2024-03-12T04:12:54.790854+0000","last_unstale":"2024-03-12T04:14:14.939744+0000","last_undegraded":"2024-03-12T04:14:14.939744+0000","last_fullsized":"2024-03-12T04:14:14.939744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:42:50.343524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":264,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:58.205524+0000","last_change":"2024-03-12T04:12:53.777534+0000","last_active":"2024-03-12T04:14:58.205524+0000","last_peered":"2024-03-12T04:14:58.205524+0000","last_clean":"2024-03-12T04:14:58.205524+0000","last_became_active":"2024-03-12T04:12:53.777201+0000","last_became_peered":"2024-03-12T04:12:53.777201+0000","last_unstale":"2024-03-12T04:14:58.205524+0000","last_undegraded":"2024-03-12T04:14:58.205524+0000","last_fullsized":"2024-03-12T04:14:58.205524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:54.708892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953198+0000","last_change":"2024-03-12T04:12:54.798962+0000","last_active":"2024-03-12T04:14:14.953198+0000","last_peered":"2024-03-12T04:14:14.953198+0000","last_clean":"2024-03-12T04:14:14.953198+0000","last_became_active":"2024-03-12T04:12:54.798650+0000","last_became_peered":"2024-03-12T04:12:54.798650+0000","last_unstale":"2024-03-12T04:14:14.953198+0000","last_undegraded":"2024-03-12T04:14:14.953198+0000","last_fullsized":"2024-03-12T04:14:14.953198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:05:29.270242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100119+0000","last_change":"2024-03-12T04:12:53.777791+0000","last_active":"2024-03-12T04:13:46.100119+0000","last_peered":"2024-03-12T04:13:46.100119+0000","last_clean":"2024-03-12T04:13:46.100119+0000","last_became_active":"2024-03-12T04:12:53.777430+0000","last_became_peered":"2024-03-12T04:12:53.777430+0000","last_unstale":"2024-03-12T04:13:46.100119+0000","last_undegraded":"2024-03-12T04:13:46.100119+0000","last_fullsized":"2024-03-12T04:13:46.100119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:38:21.858857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.921622+0000","last_change":"2024-03-12T04:12:54.784710+0000","last_active":"2024-03-12T04:14:14.921622+0000","last_peered":"2024-03-12T04:14:14.921622+0000","last_clean":"2024-03-12T04:14:14.921622+0000","last_became_active":"2024-03-12T04:12:54.784342+0000","last_became_peered":"2024-03-12T04:12:54.784342+0000","last_unstale":"2024-03-12T04:14:14.921622+0000","last_undegraded":"2024-03-12T04:14:14.921622+0000","last_fullsized":"2024-03-12T04:14:14.921622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:21.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935890+0000","last_change":"2024-03-12T04:12:53.779289+0000","last_active":"2024-03-12T04:13:45.935890+0000","last_peered":"2024-03-12T04:13:45.935890+0000","last_clean":"2024-03-12T04:13:45.935890+0000","last_became_active":"2024-03-12T04:12:53.778894+0000","last_became_peered":"2024-03-12T04:12:53.778894+0000","last_unstale":"2024-03-12T04:13:45.935890+0000","last_undegraded":"2024-03-12T04:13:45.935890+0000","last_fullsized":"2024-03-12T04:13:45.935890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:13:33.328977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.952684+0000","last_change":"2024-03-12T04:12:54.794102+0000","last_active":"2024-03-12T04:14:14.952684+0000","last_peered":"2024-03-12T04:14:14.952684+0000","last_clean":"2024-03-12T04:14:14.952684+0000","last_became_active":"2024-03-12T04:12:54.793789+0000","last_became_peered":"2024-03-12T04:12:54.793789+0000","last_unstale":"2024-03-12T04:14:14.952684+0000","last_undegraded":"2024-03-12T04:14:14.952684+0000","last_fullsized":"2024-03-12T04:14:14.952684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:55:33.632297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:58.221649+0000","last_change":"2024-03-12T04:12:53.774847+0000","last_active":"2024-03-12T04:14:58.221649+0000","last_peered":"2024-03-12T04:14:58.221649+0000","last_clean":"2024-03-12T04:14:58.221649+0000","last_became_active":"2024-03-12T04:12:53.774457+0000","last_became_peered":"2024-03-12T04:12:53.774457+0000","last_unstale":"2024-03-12T04:14:58.221649+0000","last_undegraded":"2024-03-12T04:14:58.221649+0000","last_fullsized":"2024-03-12T04:14:58.221649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:17.350636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59508,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":181,"num_write":37,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.930949+0000","last_change":"2024-03-12T04:12:54.783671+0000","last_active":"2024-03-12T04:14:14.930949+0000","last_peered":"2024-03-12T04:14:14.930949+0000","last_clean":"2024-03-12T04:14:14.930949+0000","last_became_active":"2024-03-12T04:12:54.782919+0000","last_became_peered":"2024-03-12T04:12:54.782919+0000","last_unstale":"2024-03-12T04:14:14.930949+0000","last_undegraded":"2024-03-12T04:14:14.930949+0000","last_fullsized":"2024-03-12T04:14:14.930949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:38:19.445096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.948316+0000","last_change":"2024-03-12T04:13:45.947951+0000","last_active":"2024-03-12T04:13:45.948316+0000","last_peered":"2024-03-12T04:13:45.948316+0000","last_clean":"2024-03-12T04:13:45.948316+0000","last_became_active":"2024-03-12T04:13:45.938099+0000","last_became_peered":"2024-03-12T04:13:45.938099+0000","last_unstale":"2024-03-12T04:13:45.948316+0000","last_undegraded":"2024-03-12T04:13:45.948316+0000","last_fullsized":"2024-03-12T04:13:45.948316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:03.905076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.943052+0000","last_change":"2024-03-12T04:12:54.779319+0000","last_active":"2024-03-12T04:14:14.943052+0000","last_peered":"2024-03-12T04:14:14.943052+0000","last_clean":"2024-03-12T04:14:14.943052+0000","last_became_active":"2024-03-12T04:12:54.778927+0000","last_became_peered":"2024-03-12T04:12:54.778927+0000","last_unstale":"2024-03-12T04:14:14.943052+0000","last_undegraded":"2024-03-12T04:14:14.943052+0000","last_fullsized":"2024-03-12T04:14:14.943052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:20:53.947257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.934971+0000","last_change":"2024-03-12T04:12:53.774754+0000","last_active":"2024-03-12T04:13:45.934971+0000","last_peered":"2024-03-12T04:13:45.934971+0000","last_clean":"2024-03-12T04:13:45.934971+0000","last_became_active":"2024-03-12T04:12:53.774395+0000","last_became_peered":"2024-03-12T04:12:53.774395+0000","last_unstale":"2024-03-12T04:13:45.934971+0000","last_undegraded":"2024-03-12T04:13:45.934971+0000","last_fullsized":"2024-03-12T04:13:45.934971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:35:13.286204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946653+0000","last_change":"2024-03-12T04:12:54.786891+0000","last_active":"2024-03-12T04:14:14.946653+0000","last_peered":"2024-03-12T04:14:14.946653+0000","last_clean":"2024-03-12T04:14:14.946653+0000","last_became_active":"2024-03-12T04:12:54.786591+0000","last_became_peered":"2024-03-12T04:12:54.786591+0000","last_unstale":"2024-03-12T04:14:14.946653+0000","last_undegraded":"2024-03-12T04:14:14.946653+0000","last_fullsized":"2024-03-12T04:14:14.946653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:16:09.188557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.187545+0000","last_change":"2024-03-12T04:12:53.769839+0000","last_active":"2024-03-12T04:14:54.187545+0000","last_peered":"2024-03-12T04:14:54.187545+0000","last_clean":"2024-03-12T04:14:54.187545+0000","last_became_active":"2024-03-12T04:12:53.769486+0000","last_became_peered":"2024-03-12T04:12:53.769486+0000","last_unstale":"2024-03-12T04:14:54.187545+0000","last_undegraded":"2024-03-12T04:14:54.187545+0000","last_fullsized":"2024-03-12T04:14:54.187545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:46.050018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945596+0000","last_change":"2024-03-12T04:12:54.782447+0000","last_active":"2024-03-12T04:14:14.945596+0000","last_peered":"2024-03-12T04:14:14.945596+0000","last_clean":"2024-03-12T04:14:14.945596+0000","last_became_active":"2024-03-12T04:12:54.782091+0000","last_became_peered":"2024-03-12T04:12:54.782091+0000","last_unstale":"2024-03-12T04:14:14.945596+0000","last_undegraded":"2024-03-12T04:14:14.945596+0000","last_fullsized":"2024-03-12T04:14:14.945596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:22.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.959032+0000","last_change":"2024-03-12T04:13:45.958615+0000","last_active":"2024-03-12T04:13:45.959032+0000","last_peered":"2024-03-12T04:13:45.959032+0000","last_clean":"2024-03-12T04:13:45.959032+0000","last_became_active":"2024-03-12T04:13:45.938341+0000","last_became_peered":"2024-03-12T04:13:45.938341+0000","last_unstale":"2024-03-12T04:13:45.959032+0000","last_undegraded":"2024-03-12T04:13:45.959032+0000","last_fullsized":"2024-03-12T04:13:45.959032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:17:44.688344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.951853+0000","last_change":"2024-03-12T04:12:54.784282+0000","last_active":"2024-03-12T04:14:14.951853+0000","last_peered":"2024-03-12T04:14:14.951853+0000","last_clean":"2024-03-12T04:14:14.951853+0000","last_became_active":"2024-03-12T04:12:54.783953+0000","last_became_peered":"2024-03-12T04:12:54.783953+0000","last_unstale":"2024-03-12T04:14:14.951853+0000","last_undegraded":"2024-03-12T04:14:14.951853+0000","last_fullsized":"2024-03-12T04:14:14.951853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:14:02.182546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937316+0000","last_change":"2024-03-12T04:12:53.770230+0000","last_active":"2024-03-12T04:13:45.937316+0000","last_peered":"2024-03-12T04:13:45.937316+0000","last_clean":"2024-03-12T04:13:45.937316+0000","last_became_active":"2024-03-12T04:12:53.768817+0000","last_became_peered":"2024-03-12T04:12:53.768817+0000","last_unstale":"2024-03-12T04:13:45.937316+0000","last_undegraded":"2024-03-12T04:13:45.937316+0000","last_fullsized":"2024-03-12T04:13:45.937316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:09:12.656489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940483+0000","last_change":"2024-03-12T04:12:54.785654+0000","last_active":"2024-03-12T04:14:14.940483+0000","last_peered":"2024-03-12T04:14:14.940483+0000","last_clean":"2024-03-12T04:14:14.940483+0000","last_became_active":"2024-03-12T04:12:54.785244+0000","last_became_peered":"2024-03-12T04:12:54.785244+0000","last_unstale":"2024-03-12T04:14:14.940483+0000","last_undegraded":"2024-03-12T04:14:14.940483+0000","last_fullsized":"2024-03-12T04:14:14.940483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:00:41.961637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.930880+0000","last_change":"2024-03-12T04:12:53.780962+0000","last_active":"2024-03-12T04:13:45.930880+0000","last_peered":"2024-03-12T04:13:45.930880+0000","last_clean":"2024-03-12T04:13:45.930880+0000","last_became_active":"2024-03-12T04:12:53.780623+0000","last_became_peered":"2024-03-12T04:12:53.780623+0000","last_unstale":"2024-03-12T04:13:45.930880+0000","last_undegraded":"2024-03-12T04:13:45.930880+0000","last_fullsized":"2024-03-12T04:13:45.930880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946481+0000","last_change":"2024-03-12T04:12:54.775065+0000","last_active":"2024-03-12T04:14:14.946481+0000","last_peered":"2024-03-12T04:14:14.946481+0000","last_clean":"2024-03-12T04:14:14.946481+0000","last_became_active":"2024-03-12T04:12:54.774686+0000","last_became_peered":"2024-03-12T04:12:54.774686+0000","last_unstale":"2024-03-12T04:14:14.946481+0000","last_undegraded":"2024-03-12T04:14:14.946481+0000","last_fullsized":"2024-03-12T04:14:14.946481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:56:25.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:55.189811+0000","last_change":"2024-03-12T04:12:53.777702+0000","last_active":"2024-03-12T04:14:55.189811+0000","last_peered":"2024-03-12T04:14:55.189811+0000","last_clean":"2024-03-12T04:14:55.189811+0000","last_became_active":"2024-03-12T04:12:53.777346+0000","last_became_peered":"2024-03-12T04:12:53.777346+0000","last_unstale":"2024-03-12T04:14:55.189811+0000","last_undegraded":"2024-03-12T04:14:55.189811+0000","last_fullsized":"2024-03-12T04:14:55.189811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:10:09.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939910+0000","last_change":"2024-03-12T04:12:54.789087+0000","last_active":"2024-03-12T04:14:14.939910+0000","last_peered":"2024-03-12T04:14:14.939910+0000","last_clean":"2024-03-12T04:14:14.939910+0000","last_became_active":"2024-03-12T04:12:54.788735+0000","last_became_peered":"2024-03-12T04:12:54.788735+0000","last_unstale":"2024-03-12T04:14:14.939910+0000","last_undegraded":"2024-03-12T04:14:14.939910+0000","last_fullsized":"2024-03-12T04:14:14.939910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:14.947137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945695+0000","last_change":"2024-03-12T04:12:54.789974+0000","last_active":"2024-03-12T04:14:14.945695+0000","last_peered":"2024-03-12T04:14:14.945695+0000","last_clean":"2024-03-12T04:14:14.945695+0000","last_became_active":"2024-03-12T04:12:54.789578+0000","last_became_peered":"2024-03-12T04:12:54.789578+0000","last_unstale":"2024-03-12T04:14:14.945695+0000","last_undegraded":"2024-03-12T04:14:14.945695+0000","last_fullsized":"2024-03-12T04:14:14.945695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:22:25.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.224216+0000","last_change":"2024-03-12T04:12:53.773601+0000","last_active":"2024-03-12T04:14:54.224216+0000","last_peered":"2024-03-12T04:14:54.224216+0000","last_clean":"2024-03-12T04:14:54.224216+0000","last_became_active":"2024-03-12T04:12:53.773264+0000","last_became_peered":"2024-03-12T04:12:53.773264+0000","last_unstale":"2024-03-12T04:14:54.224216+0000","last_undegraded":"2024-03-12T04:14:54.224216+0000","last_fullsized":"2024-03-12T04:14:54.224216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:07:03.580157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948721+0000","last_change":"2024-03-12T04:12:54.782496+0000","last_active":"2024-03-12T04:14:14.948721+0000","last_peered":"2024-03-12T04:14:14.948721+0000","last_clean":"2024-03-12T04:14:14.948721+0000","last_became_active":"2024-03-12T04:12:54.782014+0000","last_became_peered":"2024-03-12T04:12:54.782014+0000","last_unstale":"2024-03-12T04:14:14.948721+0000","last_undegraded":"2024-03-12T04:14:14.948721+0000","last_fullsized":"2024-03-12T04:14:14.948721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:08.580655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099322+0000","last_change":"2024-03-12T04:12:53.771529+0000","last_active":"2024-03-12T04:13:46.099322+0000","last_peered":"2024-03-12T04:13:46.099322+0000","last_clean":"2024-03-12T04:13:46.099322+0000","last_became_active":"2024-03-12T04:12:53.770855+0000","last_became_peered":"2024-03-12T04:12:53.770855+0000","last_unstale":"2024-03-12T04:13:46.099322+0000","last_undegraded":"2024-03-12T04:13:46.099322+0000","last_fullsized":"2024-03-12T04:13:46.099322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:32:33.969240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100306+0000","last_change":"2024-03-12T04:11:37.642780+0000","last_active":"2024-03-12T04:13:46.100306+0000","last_peered":"2024-03-12T04:13:46.100306+0000","last_clean":"2024-03-12T04:13:46.100306+0000","last_became_active":"2024-03-12T04:11:37.642145+0000","last_became_peered":"2024-03-12T04:11:37.642145+0000","last_unstale":"2024-03-12T04:13:46.100306+0000","last_undegraded":"2024-03-12T04:13:46.100306+0000","last_fullsized":"2024-03-12T04:13:46.100306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:08:09.604393+0000","last_clean_scrub_stamp":"2024-03-12T04:08:09.604393+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:46:36.666483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935254+0000","last_change":"2024-03-12T04:12:54.790464+0000","last_active":"2024-03-12T04:14:14.935254+0000","last_peered":"2024-03-12T04:14:14.935254+0000","last_clean":"2024-03-12T04:14:14.935254+0000","last_became_active":"2024-03-12T04:12:54.788440+0000","last_became_peered":"2024-03-12T04:12:54.788440+0000","last_unstale":"2024-03-12T04:14:14.935254+0000","last_undegraded":"2024-03-12T04:14:14.935254+0000","last_fullsized":"2024-03-12T04:14:14.935254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:54.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.100152+0000","last_change":"2024-03-12T04:12:53.773334+0000","last_active":"2024-03-12T04:13:46.100152+0000","last_peered":"2024-03-12T04:13:46.100152+0000","last_clean":"2024-03-12T04:13:46.100152+0000","last_became_active":"2024-03-12T04:12:53.772759+0000","last_became_peered":"2024-03-12T04:12:53.772759+0000","last_unstale":"2024-03-12T04:13:46.100152+0000","last_undegraded":"2024-03-12T04:13:46.100152+0000","last_fullsized":"2024-03-12T04:13:46.100152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:03:17.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.937169+0000","last_change":"2024-03-12T04:12:53.777035+0000","last_active":"2024-03-12T04:13:45.937169+0000","last_peered":"2024-03-12T04:13:45.937169+0000","last_clean":"2024-03-12T04:13:45.937169+0000","last_became_active":"2024-03-12T04:12:53.776587+0000","last_became_peered":"2024-03-12T04:12:53.776587+0000","last_unstale":"2024-03-12T04:13:45.937169+0000","last_undegraded":"2024-03-12T04:13:45.937169+0000","last_fullsized":"2024-03-12T04:13:45.937169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:21.390525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.934802+0000","last_change":"2024-03-12T04:12:54.790835+0000","last_active":"2024-03-12T04:14:14.934802+0000","last_peered":"2024-03-12T04:14:14.934802+0000","last_clean":"2024-03-12T04:14:14.934802+0000","last_became_active":"2024-03-12T04:12:54.790512+0000","last_became_peered":"2024-03-12T04:12:54.790512+0000","last_unstale":"2024-03-12T04:14:14.934802+0000","last_undegraded":"2024-03-12T04:14:14.934802+0000","last_fullsized":"2024-03-12T04:14:14.934802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:37:58.029450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931069+0000","last_change":"2024-03-12T04:12:53.774604+0000","last_active":"2024-03-12T04:13:45.931069+0000","last_peered":"2024-03-12T04:13:45.931069+0000","last_clean":"2024-03-12T04:13:45.931069+0000","last_became_active":"2024-03-12T04:12:53.774270+0000","last_became_peered":"2024-03-12T04:12:53.774270+0000","last_unstale":"2024-03-12T04:13:45.931069+0000","last_undegraded":"2024-03-12T04:13:45.931069+0000","last_fullsized":"2024-03-12T04:13:45.931069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:23.635179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946179+0000","last_change":"2024-03-12T04:12:54.786259+0000","last_active":"2024-03-12T04:14:14.946179+0000","last_peered":"2024-03-12T04:14:14.946179+0000","last_clean":"2024-03-12T04:14:14.946179+0000","last_became_active":"2024-03-12T04:12:54.783114+0000","last_became_peered":"2024-03-12T04:12:54.783114+0000","last_unstale":"2024-03-12T04:14:14.946179+0000","last_undegraded":"2024-03-12T04:14:14.946179+0000","last_fullsized":"2024-03-12T04:14:14.946179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:53.036505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.099781+0000","last_change":"2024-03-12T04:12:53.779618+0000","last_active":"2024-03-12T04:13:46.099781+0000","last_peered":"2024-03-12T04:13:46.099781+0000","last_clean":"2024-03-12T04:13:46.099781+0000","last_became_active":"2024-03-12T04:12:53.779264+0000","last_became_peered":"2024-03-12T04:12:53.779264+0000","last_unstale":"2024-03-12T04:13:46.099781+0000","last_undegraded":"2024-03-12T04:13:46.099781+0000","last_fullsized":"2024-03-12T04:13:46.099781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:57.633014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.945120+0000","last_change":"2024-03-12T04:12:54.789012+0000","last_active":"2024-03-12T04:14:14.945120+0000","last_peered":"2024-03-12T04:14:14.945120+0000","last_clean":"2024-03-12T04:14:14.945120+0000","last_became_active":"2024-03-12T04:12:54.788441+0000","last_became_peered":"2024-03-12T04:12:54.788441+0000","last_unstale":"2024-03-12T04:14:14.945120+0000","last_undegraded":"2024-03-12T04:14:14.945120+0000","last_fullsized":"2024-03-12T04:14:14.945120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:55.858926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:46.102816+0000","last_change":"2024-03-12T04:12:53.776621+0000","last_active":"2024-03-12T04:13:46.102816+0000","last_peered":"2024-03-12T04:13:46.102816+0000","last_clean":"2024-03-12T04:13:46.102816+0000","last_became_active":"2024-03-12T04:12:53.776313+0000","last_became_peered":"2024-03-12T04:12:53.776313+0000","last_unstale":"2024-03-12T04:13:46.102816+0000","last_undegraded":"2024-03-12T04:13:46.102816+0000","last_fullsized":"2024-03-12T04:13:46.102816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:43.536159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.900148+0000","last_change":"2024-03-12T04:12:54.786929+0000","last_active":"2024-03-12T04:14:14.900148+0000","last_peered":"2024-03-12T04:14:14.900148+0000","last_clean":"2024-03-12T04:14:14.900148+0000","last_became_active":"2024-03-12T04:12:54.786580+0000","last_became_peered":"2024-03-12T04:12:54.786580+0000","last_unstale":"2024-03-12T04:14:14.900148+0000","last_undegraded":"2024-03-12T04:14:14.900148+0000","last_fullsized":"2024-03-12T04:14:14.900148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:58.955293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935801+0000","last_change":"2024-03-12T04:12:53.778577+0000","last_active":"2024-03-12T04:13:45.935801+0000","last_peered":"2024-03-12T04:13:45.935801+0000","last_clean":"2024-03-12T04:13:45.935801+0000","last_became_active":"2024-03-12T04:12:53.778033+0000","last_became_peered":"2024-03-12T04:12:53.778033+0000","last_unstale":"2024-03-12T04:13:45.935801+0000","last_undegraded":"2024-03-12T04:13:45.935801+0000","last_fullsized":"2024-03-12T04:13:45.935801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:29:02.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.939536+0000","last_change":"2024-03-12T04:12:54.785861+0000","last_active":"2024-03-12T04:14:14.939536+0000","last_peered":"2024-03-12T04:14:14.939536+0000","last_clean":"2024-03-12T04:14:14.939536+0000","last_became_active":"2024-03-12T04:12:54.784682+0000","last_became_peered":"2024-03-12T04:12:54.784682+0000","last_unstale":"2024-03-12T04:14:14.939536+0000","last_undegraded":"2024-03-12T04:14:14.939536+0000","last_fullsized":"2024-03-12T04:14:14.939536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:35:12.288309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.931757+0000","last_change":"2024-03-12T04:12:53.772865+0000","last_active":"2024-03-12T04:13:45.931757+0000","last_peered":"2024-03-12T04:13:45.931757+0000","last_clean":"2024-03-12T04:13:45.931757+0000","last_became_active":"2024-03-12T04:12:53.772502+0000","last_became_peered":"2024-03-12T04:12:53.772502+0000","last_unstale":"2024-03-12T04:13:45.931757+0000","last_undegraded":"2024-03-12T04:13:45.931757+0000","last_fullsized":"2024-03-12T04:13:45.931757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:40:24.112197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.942329+0000","last_change":"2024-03-12T04:12:54.780722+0000","last_active":"2024-03-12T04:14:14.942329+0000","last_peered":"2024-03-12T04:14:14.942329+0000","last_clean":"2024-03-12T04:14:14.942329+0000","last_became_active":"2024-03-12T04:12:54.780404+0000","last_became_peered":"2024-03-12T04:12:54.780404+0000","last_unstale":"2024-03-12T04:14:14.942329+0000","last_undegraded":"2024-03-12T04:14:14.942329+0000","last_fullsized":"2024-03-12T04:14:14.942329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.068094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.936882+0000","last_change":"2024-03-12T04:12:53.771471+0000","last_active":"2024-03-12T04:13:45.936882+0000","last_peered":"2024-03-12T04:13:45.936882+0000","last_clean":"2024-03-12T04:13:45.936882+0000","last_became_active":"2024-03-12T04:12:53.771147+0000","last_became_peered":"2024-03-12T04:12:53.771147+0000","last_unstale":"2024-03-12T04:13:45.936882+0000","last_undegraded":"2024-03-12T04:13:45.936882+0000","last_fullsized":"2024-03-12T04:13:45.936882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:46.159168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.949687+0000","last_change":"2024-03-12T04:12:54.796253+0000","last_active":"2024-03-12T04:14:14.949687+0000","last_peered":"2024-03-12T04:14:14.949687+0000","last_clean":"2024-03-12T04:14:14.949687+0000","last_became_active":"2024-03-12T04:12:54.795656+0000","last_became_peered":"2024-03-12T04:12:54.795656+0000","last_unstale":"2024-03-12T04:14:14.949687+0000","last_undegraded":"2024-03-12T04:14:14.949687+0000","last_fullsized":"2024-03-12T04:14:14.949687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:39:32.788874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:54.198832+0000","last_change":"2024-03-12T04:12:53.780257+0000","last_active":"2024-03-12T04:14:54.198832+0000","last_peered":"2024-03-12T04:14:54.198832+0000","last_clean":"2024-03-12T04:14:54.198832+0000","last_became_active":"2024-03-12T04:12:53.779891+0000","last_became_peered":"2024-03-12T04:12:53.779891+0000","last_unstale":"2024-03-12T04:14:54.198832+0000","last_undegraded":"2024-03-12T04:14:54.198832+0000","last_fullsized":"2024-03-12T04:14:54.198832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:49:30.218372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.950676+0000","last_change":"2024-03-12T04:12:54.792760+0000","last_active":"2024-03-12T04:14:14.950676+0000","last_peered":"2024-03-12T04:14:14.950676+0000","last_clean":"2024-03-12T04:14:14.950676+0000","last_became_active":"2024-03-12T04:12:54.792363+0000","last_became_peered":"2024-03-12T04:12:54.792363+0000","last_unstale":"2024-03-12T04:14:14.950676+0000","last_undegraded":"2024-03-12T04:14:14.950676+0000","last_fullsized":"2024-03-12T04:14:14.950676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:30:15.394318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.935702+0000","last_change":"2024-03-12T04:12:53.775556+0000","last_active":"2024-03-12T04:13:45.935702+0000","last_peered":"2024-03-12T04:13:45.935702+0000","last_clean":"2024-03-12T04:13:45.935702+0000","last_became_active":"2024-03-12T04:12:53.775182+0000","last_became_peered":"2024-03-12T04:12:53.775182+0000","last_unstale":"2024-03-12T04:13:45.935702+0000","last_undegraded":"2024-03-12T04:13:45.935702+0000","last_fullsized":"2024-03-12T04:13:45.935702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:13:50.076672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948544+0000","last_change":"2024-03-12T04:12:54.795876+0000","last_active":"2024-03-12T04:14:14.948544+0000","last_peered":"2024-03-12T04:14:14.948544+0000","last_clean":"2024-03-12T04:14:14.948544+0000","last_became_active":"2024-03-12T04:12:54.793874+0000","last_became_peered":"2024-03-12T04:12:54.793874+0000","last_unstale":"2024-03-12T04:14:14.948544+0000","last_undegraded":"2024-03-12T04:14:14.948544+0000","last_fullsized":"2024-03-12T04:14:14.948544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:33:31.165974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:13:45.933769+0000","last_change":"2024-03-12T04:12:53.774879+0000","last_active":"2024-03-12T04:13:45.933769+0000","last_peered":"2024-03-12T04:13:45.933769+0000","last_clean":"2024-03-12T04:13:45.933769+0000","last_became_active":"2024-03-12T04:12:53.774530+0000","last_became_peered":"2024-03-12T04:12:53.774530+0000","last_unstale":"2024-03-12T04:13:45.933769+0000","last_undegraded":"2024-03-12T04:13:45.933769+0000","last_fullsized":"2024-03-12T04:13:45.933769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:52.747203+0000","last_clean_scrub_stamp":"2024-03-12T04:12:52.747203+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:25.858165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946946+0000","last_change":"2024-03-12T04:12:54.790140+0000","last_active":"2024-03-12T04:14:14.946946+0000","last_peered":"2024-03-12T04:14:14.946946+0000","last_clean":"2024-03-12T04:14:14.946946+0000","last_became_active":"2024-03-12T04:12:54.789798+0000","last_became_peered":"2024-03-12T04:12:54.789798+0000","last_unstale":"2024-03-12T04:14:14.946946+0000","last_undegraded":"2024-03-12T04:14:14.946946+0000","last_fullsized":"2024-03-12T04:14:14.946946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:12:38.722638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.936334+0000","last_change":"2024-03-12T04:12:54.785029+0000","last_active":"2024-03-12T04:14:14.936334+0000","last_peered":"2024-03-12T04:14:14.936334+0000","last_clean":"2024-03-12T04:14:14.936334+0000","last_became_active":"2024-03-12T04:12:54.784197+0000","last_became_peered":"2024-03-12T04:12:54.784197+0000","last_unstale":"2024-03-12T04:14:14.936334+0000","last_undegraded":"2024-03-12T04:14:14.936334+0000","last_fullsized":"2024-03-12T04:14:14.936334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:22:41.352093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.937779+0000","last_change":"2024-03-12T04:12:54.793132+0000","last_active":"2024-03-12T04:14:14.937779+0000","last_peered":"2024-03-12T04:14:14.937779+0000","last_clean":"2024-03-12T04:14:14.937779+0000","last_became_active":"2024-03-12T04:12:54.792838+0000","last_became_peered":"2024-03-12T04:12:54.792838+0000","last_unstale":"2024-03-12T04:14:14.937779+0000","last_undegraded":"2024-03-12T04:14:14.937779+0000","last_fullsized":"2024-03-12T04:14:14.937779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:25:22.641963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.935192+0000","last_change":"2024-03-12T04:13:45.938126+0000","last_active":"2024-03-12T04:14:14.935192+0000","last_peered":"2024-03-12T04:14:14.935192+0000","last_clean":"2024-03-12T04:14:14.935192+0000","last_became_active":"2024-03-12T04:13:45.937698+0000","last_became_peered":"2024-03-12T04:13:45.937698+0000","last_unstale":"2024-03-12T04:14:14.935192+0000","last_undegraded":"2024-03-12T04:14:14.935192+0000","last_fullsized":"2024-03-12T04:14:14.935192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:10:25.378634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944252+0000","last_change":"2024-03-12T04:12:54.785830+0000","last_active":"2024-03-12T04:14:14.944252+0000","last_peered":"2024-03-12T04:14:14.944252+0000","last_clean":"2024-03-12T04:14:14.944252+0000","last_became_active":"2024-03-12T04:12:54.785087+0000","last_became_peered":"2024-03-12T04:12:54.785087+0000","last_unstale":"2024-03-12T04:14:14.944252+0000","last_undegraded":"2024-03-12T04:14:14.944252+0000","last_fullsized":"2024-03-12T04:14:14.944252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:08:09.860311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.944035+0000","last_change":"2024-03-12T04:13:45.937883+0000","last_active":"2024-03-12T04:14:14.944035+0000","last_peered":"2024-03-12T04:14:14.944035+0000","last_clean":"2024-03-12T04:14:14.944035+0000","last_became_active":"2024-03-12T04:13:45.937471+0000","last_became_peered":"2024-03-12T04:13:45.937471+0000","last_unstale":"2024-03-12T04:14:14.944035+0000","last_undegraded":"2024-03-12T04:14:14.944035+0000","last_fullsized":"2024-03-12T04:14:14.944035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:28:21.997033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.932702+0000","last_change":"2024-03-12T04:13:45.938864+0000","last_active":"2024-03-12T04:14:14.932702+0000","last_peered":"2024-03-12T04:14:14.932702+0000","last_clean":"2024-03-12T04:14:14.932702+0000","last_became_active":"2024-03-12T04:13:45.938336+0000","last_became_peered":"2024-03-12T04:13:45.938336+0000","last_unstale":"2024-03-12T04:14:14.932702+0000","last_undegraded":"2024-03-12T04:14:14.932702+0000","last_fullsized":"2024-03-12T04:14:14.932702+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:00:44.990282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.938228+0000","last_change":"2024-03-12T04:12:54.796733+0000","last_active":"2024-03-12T04:14:14.938228+0000","last_peered":"2024-03-12T04:14:14.938228+0000","last_clean":"2024-03-12T04:14:14.938228+0000","last_became_active":"2024-03-12T04:12:54.796388+0000","last_became_peered":"2024-03-12T04:12:54.796388+0000","last_unstale":"2024-03-12T04:14:14.938228+0000","last_undegraded":"2024-03-12T04:14:14.938228+0000","last_fullsized":"2024-03-12T04:14:14.938228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:01.580286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.947274+0000","last_change":"2024-03-12T04:12:54.787200+0000","last_active":"2024-03-12T04:14:14.947274+0000","last_peered":"2024-03-12T04:14:14.947274+0000","last_clean":"2024-03-12T04:14:14.947274+0000","last_became_active":"2024-03-12T04:12:54.786657+0000","last_became_peered":"2024-03-12T04:12:54.786657+0000","last_unstale":"2024-03-12T04:14:14.947274+0000","last_undegraded":"2024-03-12T04:14:14.947274+0000","last_fullsized":"2024-03-12T04:14:14.947274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:15:35.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.943388+0000","last_change":"2024-03-12T04:12:54.785132+0000","last_active":"2024-03-12T04:14:14.943388+0000","last_peered":"2024-03-12T04:14:14.943388+0000","last_clean":"2024-03-12T04:14:14.943388+0000","last_became_active":"2024-03-12T04:12:54.782548+0000","last_became_peered":"2024-03-12T04:12:54.782548+0000","last_unstale":"2024-03-12T04:14:14.943388+0000","last_undegraded":"2024-03-12T04:14:14.943388+0000","last_fullsized":"2024-03-12T04:14:14.943388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:40:49.284778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.948256+0000","last_change":"2024-03-12T04:12:54.790227+0000","last_active":"2024-03-12T04:14:14.948256+0000","last_peered":"2024-03-12T04:14:14.948256+0000","last_clean":"2024-03-12T04:14:14.948256+0000","last_became_active":"2024-03-12T04:12:54.789853+0000","last_became_peered":"2024-03-12T04:12:54.789853+0000","last_unstale":"2024-03-12T04:14:14.948256+0000","last_undegraded":"2024-03-12T04:14:14.948256+0000","last_fullsized":"2024-03-12T04:14:14.948256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:38:16.683661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.953923+0000","last_change":"2024-03-12T04:12:54.796039+0000","last_active":"2024-03-12T04:14:14.953923+0000","last_peered":"2024-03-12T04:14:14.953923+0000","last_clean":"2024-03-12T04:14:14.953923+0000","last_became_active":"2024-03-12T04:12:54.795568+0000","last_became_peered":"2024-03-12T04:12:54.795568+0000","last_unstale":"2024-03-12T04:14:14.953923+0000","last_undegraded":"2024-03-12T04:14:14.953923+0000","last_fullsized":"2024-03-12T04:14:14.953923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:46.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.946286+0000","last_change":"2024-03-12T04:12:54.785610+0000","last_active":"2024-03-12T04:14:14.946286+0000","last_peered":"2024-03-12T04:14:14.946286+0000","last_clean":"2024-03-12T04:14:14.946286+0000","last_became_active":"2024-03-12T04:12:54.785256+0000","last_became_peered":"2024-03-12T04:12:54.785256+0000","last_unstale":"2024-03-12T04:14:14.946286+0000","last_undegraded":"2024-03-12T04:14:14.946286+0000","last_fullsized":"2024-03-12T04:14:14.946286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:34.152361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-12T04:14:14.940458+0000","last_change":"2024-03-12T04:12:54.787064+0000","last_active":"2024-03-12T04:14:14.940458+0000","last_peered":"2024-03-12T04:14:14.940458+0000","last_clean":"2024-03-12T04:14:14.940458+0000","last_became_active":"2024-03-12T04:12:54.786701+0000","last_became_peered":"2024-03-12T04:12:54.786701+0000","last_unstale":"2024-03-12T04:14:14.940458+0000","last_undegraded":"2024-03-12T04:14:14.940458+0000","last_fullsized":"2024-03-12T04:14:14.940458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T04:12:53.752953+0000","last_clean_scrub_stamp":"2024-03-12T04:12:53.752953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:44:43.582171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250964,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1208,"num_read_kb":681,"num_write":257,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":206898,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":468541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":1119430,"data_compressed":38179,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":467798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":468064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1062846,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":468008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":528429,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":469347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":528127,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":467340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":468401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1060513,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34682,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36320,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61753,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62930,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62269,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T04:15:00.885 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:15:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 -- 172.21.15.42:0/79350676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102d50 msgr2=0x7f12341095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 --2- 172.21.15.42:0/79350676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102d50 0x7f12341095e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1224009ca0 tx=0x7f122402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 -- 172.21.15.42:0/79350676 shutdown_connections 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 --2- 172.21.15.42:0/79350676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102d50 0x7f12341095e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 --2- 172.21.15.42:0/79350676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1234102390 0x7f1234102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 --2- 172.21.15.42:0/79350676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12341011a0 0x7f12341015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 -- 172.21.15.42:0/79350676 >> 172.21.15.42:0/79350676 conn(0x7f12340fc730 msgr2=0x7f12340feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 -- 172.21.15.42:0/79350676 shutdown_connections 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 -- 172.21.15.42:0/79350676 wait complete. 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 Processor -- start 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.023+0000 7f123933a640 1 -- start start 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f123933a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12341011a0 0x7f1234197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f123933a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102390 0x7f12341976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f123933a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1234102d50 0x7f123419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f123933a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419ec60 con 0x7f1234102390 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f123933a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419edd0 con 0x7f1234102d50 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f123933a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419ef90 con 0x7f12341011a0 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102390 0x7f12341976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102390 0x7f12341976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50142/0 (socket says 172.21.15.42:50142) 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 -- 172.21.15.42:0/1179656737 learned_addr learned my addr 172.21.15.42:0/1179656737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12337fe640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1234102d50 0x7f123419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f1232ffd640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12341011a0 0x7f1234197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 -- 172.21.15.42:0/1179656737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12341011a0 msgr2=0x7f1234197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12341011a0 0x7f1234197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 -- 172.21.15.42:0/1179656737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1234102d50 msgr2=0x7f123419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1234102d50 0x7f123419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 -- 172.21.15.42:0/1179656737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1224009900 con 0x7f1234102390 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f12327fc640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102390 0x7f12341976b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f121c00b4a0 tx=0x7f121c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.024+0000 7f122bfff640 1 -- 172.21.15.42:0/1179656737 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c010610 con 0x7f1234102390 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.025+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123419f270 con 0x7f1234102390 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.025+0000 7f122bfff640 1 -- 172.21.15.42:0/1179656737 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f121c010c20 con 0x7f1234102390 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.025+0000 7f122bfff640 1 -- 172.21.15.42:0/1179656737 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c017d00 con 0x7f1234102390 2024-03-12T04:15:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.025+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123419f850 con 0x7f1234102390 2024-03-12T04:15:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.026+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12341084c0 con 0x7f1234102390 2024-03-12T04:15:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.027+0000 7f122bfff640 1 -- 172.21.15.42:0/1179656737 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f121c010770 con 0x7f1234102390 2024-03-12T04:15:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.027+0000 7f122bfff640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12200a1a30 0x7f12200a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.027+0000 7f1232ffd640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12200a1a30 0x7f12200a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.027+0000 7f122bfff640 1 -- 172.21.15.42:0/1179656737 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f121c0ccff0 con 0x7f1234102390 2024-03-12T04:15:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.028+0000 7f1232ffd640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12200a1a30 0x7f12200a3ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12341021f0 tx=0x7f122c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.031+0000 7f122bfff640 1 -- 172.21.15.42:0/1179656737 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f121c0962e0 con 0x7f1234102390 2024-03-12T04:15:01.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.214+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f123410ccc0 con 0x7f1234102390 2024-03-12T04:15:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.215+0000 7f122bfff640 1 -- 172.21.15.42:0/1179656737 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f121c0999a0 con 0x7f1234102390 2024-03-12T04:15:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:15:01.218 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:15:01.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12200a1a30 msgr2=0x7f12200a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12200a1a30 0x7f12200a3ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12341021f0 tx=0x7f122c00a2d0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102390 msgr2=0x7f12341976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102390 0x7f12341976b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f121c00b4a0 tx=0x7f121c00b960 comp rx=0 tx=0).stop 2024-03-12T04:15:01.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 shutdown_connections 2024-03-12T04:15:01.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f12200a1a30 0x7f12200a3ef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1234102d50 0x7f123419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1234102390 0x7f12341976b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 --2- 172.21.15.42:0/1179656737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12341011a0 0x7f1234197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.217+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 >> 172.21.15.42:0/1179656737 conn(0x7f12340fc730 msgr2=0x7f1234105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:01.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.218+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 shutdown_connections 2024-03-12T04:15:01.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.218+0000 7f123933a640 1 -- 172.21.15.42:0/1179656737 wait complete. 2024-03-12T04:15:01.230 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:01 smithi042 ceph-mon[32260]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.230 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:01 smithi042 ceph-mon[32260]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.230 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:01 smithi042 ceph-mon[32260]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:15:01.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:15:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 -- 172.21.15.42:0/566477209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218106e70 msgr2=0x7f52181092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 --2- 172.21.15.42:0/566477209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218106e70 0x7f52181092c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5208009ca0 tx=0x7f520802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 -- 172.21.15.42:0/566477209 shutdown_connections 2024-03-12T04:15:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 --2- 172.21.15.42:0/566477209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218106e70 0x7f52181092c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 --2- 172.21.15.42:0/566477209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5218104540 0x7f5218106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 --2- 172.21.15.42:0/566477209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5218101c10 0x7f5218104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 -- 172.21.15.42:0/566477209 >> 172.21.15.42:0/566477209 conn(0x7f52180fb620 msgr2=0x7f52180fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 -- 172.21.15.42:0/566477209 shutdown_connections 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.335+0000 7f521f2c9640 1 -- 172.21.15.42:0/566477209 wait complete. 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 Processor -- start 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 -- start start 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5218101c10 0x7f5218197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218104540 0x7f52181976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5218106e70 0x7f521819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521819ec60 con 0x7f5218104540 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521819edd0 con 0x7f5218106e70 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521f2c9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521819ef90 con 0x7f5218101c10 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218104540 0x7f52181976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218104540 0x7f52181976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50162/0 (socket says 172.21.15.42:50162) 2024-03-12T04:15:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 -- 172.21.15.42:0/477042617 learned_addr learned my addr 172.21.15.42:0/477042617 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521d866640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5218106e70 0x7f521819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521d065640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5218101c10 0x7f5218197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 -- 172.21.15.42:0/477042617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5218101c10 msgr2=0x7f5218197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5218101c10 0x7f5218197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 -- 172.21.15.42:0/477042617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5218106e70 msgr2=0x7f521819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5218106e70 0x7f521819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.336+0000 7f521c864640 1 -- 172.21.15.42:0/477042617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208009900 con 0x7f5218104540 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.337+0000 7f521d065640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5218101c10 0x7f5218197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.337+0000 7f521c864640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218104540 0x7f52181976b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f520c00d740 tx=0x7f520c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.337+0000 7f52167fc640 1 -- 172.21.15.42:0/477042617 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c009c30 con 0x7f5218104540 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.337+0000 7f52167fc640 1 -- 172.21.15.42:0/477042617 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f520c00de50 con 0x7f5218104540 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.337+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521819f270 con 0x7f5218104540 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.337+0000 7f52167fc640 1 -- 172.21.15.42:0/477042617 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c004e40 con 0x7f5218104540 2024-03-12T04:15:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.337+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521819f730 con 0x7f5218104540 2024-03-12T04:15:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.338+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52180fd110 con 0x7f5218104540 2024-03-12T04:15:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.339+0000 7f52167fc640 1 -- 172.21.15.42:0/477042617 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f520c004fa0 con 0x7f5218104540 2024-03-12T04:15:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.339+0000 7f52167fc640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52000a1a30 0x7f52000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.339+0000 7f521d065640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52000a1a30 0x7f52000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.339+0000 7f52167fc640 1 -- 172.21.15.42:0/477042617 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f520c0cce40 con 0x7f5218104540 2024-03-12T04:15:01.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.340+0000 7f521d065640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52000a1a30 0x7f52000a3ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5210009b40 tx=0x7f5210005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.344+0000 7f52167fc640 1 -- 172.21.15.42:0/477042617 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f520c099bb0 con 0x7f5218104540 2024-03-12T04:15:01.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:01 smithi119 ceph-mon[36119]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:01 smithi119 ceph-mon[36119]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:01 smithi119 ceph-mon[36119]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:01 smithi136 ceph-mon[35571]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:01 smithi136 ceph-mon[35571]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:01 smithi136 ceph-mon[35571]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T04:15:01.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.524+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f521802ce40 con 0x7f5218104540 2024-03-12T04:15:01.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.525+0000 7f52167fc640 1 -- 172.21.15.42:0/477042617 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f520c0170a0 con 0x7f5218104540 2024-03-12T04:15:01.527 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:15:01.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52000a1a30 msgr2=0x7f52000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52000a1a30 0x7f52000a3ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5210009b40 tx=0x7f5210005e50 comp rx=0 tx=0).stop 2024-03-12T04:15:01.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218104540 msgr2=0x7f52181976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218104540 0x7f52181976b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f520c00d740 tx=0x7f520c00dc00 comp rx=0 tx=0).stop 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 shutdown_connections 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f52000a1a30 0x7f52000a3ef0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5218106e70 0x7f521819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5218104540 0x7f52181976b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.526+0000 7f521f2c9640 1 --2- 172.21.15.42:0/477042617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5218101c10 0x7f5218197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.527+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 >> 172.21.15.42:0/477042617 conn(0x7f52180fb620 msgr2=0x7f5218102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.527+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 shutdown_connections 2024-03-12T04:15:01.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.527+0000 7f521f2c9640 1 -- 172.21.15.42:0/477042617 wait complete. 2024-03-12T04:15:01.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:15:01.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.643+0000 7f585530f640 1 -- 172.21.15.42:0/1436168777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 msgr2=0x7f58501004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.643+0000 7f585530f640 1 --2- 172.21.15.42:0/1436168777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501004a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5840009900 tx=0x7f584002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:01.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.643+0000 7f585530f640 1 -- 172.21.15.42:0/1436168777 shutdown_connections 2024-03-12T04:15:01.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.643+0000 7f585530f640 1 --2- 172.21.15.42:0/1436168777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5850101c30 0x7f58501084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.643+0000 7f585530f640 1 --2- 172.21.15.42:0/1436168777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5850101270 0x7f58501016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.643+0000 7f585530f640 1 --2- 172.21.15.42:0/1436168777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501004a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.643+0000 7f585530f640 1 -- 172.21.15.42:0/1436168777 >> 172.21.15.42:0/1436168777 conn(0x7f58500fb610 msgr2=0x7f58500fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 -- 172.21.15.42:0/1436168777 shutdown_connections 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 -- 172.21.15.42:0/1436168777 wait complete. 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 Processor -- start 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 -- start start 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5850101270 0x7f58501965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5850101c30 0x7f585019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585019dba0 con 0x7f5850100080 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585019dd10 con 0x7f5850101270 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.644+0000 7f585530f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585019ded0 con 0x7f5850101c30 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57474/0 (socket says 172.21.15.42:57474) 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 -- 172.21.15.42:0/1441141766 learned_addr learned my addr 172.21.15.42:0/1441141766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:01.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f58467fc640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5850101270 0x7f58501965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584f7fe640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5850101c30 0x7f585019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 -- 172.21.15.42:0/1441141766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5850101c30 msgr2=0x7f585019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5850101c30 0x7f585019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 -- 172.21.15.42:0/1441141766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5850101270 msgr2=0x7f58501965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5850101270 0x7f58501965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 -- 172.21.15.42:0/1441141766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5840009560 con 0x7f5850100080 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584f7fe640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5850101c30 0x7f585019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584effd640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501960b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5840000c00 tx=0x7f58400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584cff9640 1 -- 172.21.15.42:0/1441141766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400044e0 con 0x7f5850100080 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584cff9640 1 -- 172.21.15.42:0/1441141766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f584002de90 con 0x7f5850100080 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585019e150 con 0x7f5850100080 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f584cff9640 1 -- 172.21.15.42:0/1441141766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840005410 con 0x7f5850100080 2024-03-12T04:15:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.645+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585019e660 con 0x7f5850100080 2024-03-12T04:15:01.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.646+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5850107510 con 0x7f5850100080 2024-03-12T04:15:01.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.647+0000 7f584cff9640 1 -- 172.21.15.42:0/1441141766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5840005570 con 0x7f5850100080 2024-03-12T04:15:01.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.648+0000 7f584cff9640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f583c0a1a30 0x7f583c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.648+0000 7f58467fc640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f583c0a1a30 0x7f583c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.648+0000 7f584cff9640 1 -- 172.21.15.42:0/1441141766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f58400ef670 con 0x7f5850100080 2024-03-12T04:15:01.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.648+0000 7f58467fc640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f583c0a1a30 0x7f583c0a3ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f58501975c0 tx=0x7f583800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.653+0000 7f584cff9640 1 -- 172.21.15.42:0/1441141766 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58400b8850 con 0x7f5850100080 2024-03-12T04:15:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.835+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f585002cda0 con 0x7f5850100080 2024-03-12T04:15:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.836+0000 7f584cff9640 1 -- 172.21.15.42:0/1441141766 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58400bbf10 con 0x7f5850100080 2024-03-12T04:15:01.838 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f583c0a1a30 msgr2=0x7f583c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f583c0a1a30 0x7f583c0a3ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f58501975c0 tx=0x7f583800b390 comp rx=0 tx=0).stop 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 msgr2=0x7f58501960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501960b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5840000c00 tx=0x7f58400042f0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 shutdown_connections 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5850101c30 0x7f585019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f583c0a1a30 0x7f583c0a3ef0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5850101270 0x7f58501965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 --2- 172.21.15.42:0/1441141766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5850100080 0x7f58501960b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 >> 172.21.15.42:0/1441141766 conn(0x7f58500fb610 msgr2=0x7f58501046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 shutdown_connections 2024-03-12T04:15:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.838+0000 7f585530f640 1 -- 172.21.15.42:0/1441141766 wait complete. 2024-03-12T04:15:01.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:15:01.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 -- 172.21.15.42:0/3863496995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 msgr2=0x7f3274106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 --2- 172.21.15.42:0/3863496995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f3274106930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3264009900 tx=0x7f326402bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:01.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 -- 172.21.15.42:0/3863496995 shutdown_connections 2024-03-12T04:15:01.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 --2- 172.21.15.42:0/3863496995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3274106e70 0x7f32741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 --2- 172.21.15.42:0/3863496995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f3274106930 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 --2- 172.21.15.42:0/3863496995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3274101c10 0x7f3274104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 -- 172.21.15.42:0/3863496995 >> 172.21.15.42:0/3863496995 conn(0x7f32740fb620 msgr2=0x7f32740fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 -- 172.21.15.42:0/3863496995 shutdown_connections 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 -- 172.21.15.42:0/3863496995 wait complete. 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.954+0000 7f327af4f640 1 Processor -- start 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 -- start start 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3274101c10 0x7f32741971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f32741976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3274106e70 0x7f327419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327419eca0 con 0x7f3274104540 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327419ee10 con 0x7f3274106e70 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327419efd0 con 0x7f3274101c10 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f32741976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f32741976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57490/0 (socket says 172.21.15.42:57490) 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 -- 172.21.15.42:0/323119958 learned_addr learned my addr 172.21.15.42:0/323119958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f3278ceb640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3274101c10 0x7f32741971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 -- 172.21.15.42:0/323119958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3274101c10 msgr2=0x7f32741971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3274101c10 0x7f32741971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f32794ec640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3274106e70 0x7f327419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 -- 172.21.15.42:0/323119958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3274106e70 msgr2=0x7f327419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3274106e70 0x7f327419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 -- 172.21.15.42:0/323119958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3264009560 con 0x7f3274104540 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f326bfff640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f32741976f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3264000c00 tx=0x7f32640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f3269ffb640 1 -- 172.21.15.42:0/323119958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640044e0 con 0x7f3274104540 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327419f250 con 0x7f3274104540 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f3269ffb640 1 -- 172.21.15.42:0/323119958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f326402de90 con 0x7f3274104540 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.955+0000 7f3269ffb640 1 -- 172.21.15.42:0/323119958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264005410 con 0x7f3274104540 2024-03-12T04:15:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.956+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327419f760 con 0x7f3274104540 2024-03-12T04:15:01.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.957+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32740fd110 con 0x7f3274104540 2024-03-12T04:15:01.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.958+0000 7f3269ffb640 1 -- 172.21.15.42:0/323119958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3264005570 con 0x7f3274104540 2024-03-12T04:15:01.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.958+0000 7f3269ffb640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f325c0a1a30 0x7f325c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:01.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.958+0000 7f3269ffb640 1 -- 172.21.15.42:0/323119958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f32640ef6d0 con 0x7f3274104540 2024-03-12T04:15:01.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.958+0000 7f3278ceb640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f325c0a1a30 0x7f325c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:01.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.959+0000 7f3278ceb640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f325c0a1a30 0x7f325c0a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f32700058c0 tx=0x7f3270005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:01.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:01.963+0000 7f3269ffb640 1 -- 172.21.15.42:0/323119958 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32640b8910 con 0x7f3274104540 2024-03-12T04:15:02.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.143+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f327402ce40 con 0x7f3274104540 2024-03-12T04:15:02.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.143+0000 7f3269ffb640 1 -- 172.21.15.42:0/323119958 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32640bbfd0 con 0x7f3274104540 2024-03-12T04:15:02.146 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:15:02.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f325c0a1a30 msgr2=0x7f325c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f325c0a1a30 0x7f325c0a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f32700058c0 tx=0x7f3270005c60 comp rx=0 tx=0).stop 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 msgr2=0x7f32741976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f32741976f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3264000c00 tx=0x7f32640042f0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 shutdown_connections 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f325c0a1a30 0x7f325c0a3ef0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3274106e70 0x7f327419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3274104540 0x7f32741976f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 --2- 172.21.15.42:0/323119958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3274101c10 0x7f32741971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.145+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 >> 172.21.15.42:0/323119958 conn(0x7f32740fb620 msgr2=0x7f3274103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.146+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 shutdown_connections 2024-03-12T04:15:02.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.146+0000 7f327af4f640 1 -- 172.21.15.42:0/323119958 wait complete. 2024-03-12T04:15:02.158 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:02 smithi042 ceph-mon[32260]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-12T04:15:02.158 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1179656737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:15:02.158 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/477042617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:15:02.158 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1441141766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:15:02.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:15:02.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.259+0000 7fa894617640 1 -- 172.21.15.42:0/2055822300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101290 msgr2=0x7fa88c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.259+0000 7fa894617640 1 --2- 172.21.15.42:0/2055822300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101290 0x7fa88c101710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa880009900 tx=0x7fa88002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- 172.21.15.42:0/2055822300 shutdown_connections 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 --2- 172.21.15.42:0/2055822300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa88c101c50 0x7fa88c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 --2- 172.21.15.42:0/2055822300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101290 0x7fa88c101710 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 --2- 172.21.15.42:0/2055822300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa88c1000a0 0x7fa88c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- 172.21.15.42:0/2055822300 >> 172.21.15.42:0/2055822300 conn(0x7fa88c0fb610 msgr2=0x7fa88c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- 172.21.15.42:0/2055822300 shutdown_connections 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- 172.21.15.42:0/2055822300 wait complete. 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 Processor -- start 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- start start 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa88c1000a0 0x7fa88c1960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa88c101290 0x7fa88c1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101c50 0x7fa88c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c19db90 con 0x7fa88c101c50 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c19dd00 con 0x7fa88c1000a0 2024-03-12T04:15:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa894617640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c19dec0 con 0x7fa88c101290 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa892bb4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101c50 0x7fa88c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa892bb4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101c50 0x7fa88c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57518/0 (socket says 172.21.15.42:57518) 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa892bb4640 1 -- 172.21.15.42:0/865463661 learned_addr learned my addr 172.21.15.42:0/865463661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.260+0000 7fa8923b3640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa88c1000a0 0x7fa88c1960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa891bb2640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa88c101290 0x7fa88c1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa892bb4640 1 -- 172.21.15.42:0/865463661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa88c101290 msgr2=0x7fa88c1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa892bb4640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa88c101290 0x7fa88c1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa892bb4640 1 -- 172.21.15.42:0/865463661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa88c1000a0 msgr2=0x7fa88c1960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa892bb4640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa88c1000a0 0x7fa88c1960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa892bb4640 1 -- 172.21.15.42:0/865463661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009560 con 0x7fa88c101c50 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa892bb4640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101c50 0x7fa88c19d650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa88400eaa0 tx=0x7fa88400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa87f7fe640 1 -- 172.21.15.42:0/865463661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400cdb0 con 0x7fa88c101c50 2024-03-12T04:15:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa87f7fe640 1 -- 172.21.15.42:0/865463661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa884005020 con 0x7fa88c101c50 2024-03-12T04:15:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c19e1a0 con 0x7fa88c101c50 2024-03-12T04:15:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa87f7fe640 1 -- 172.21.15.42:0/865463661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840116b0 con 0x7fa88c101c50 2024-03-12T04:15:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.261+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c19e660 con 0x7fa88c101c50 2024-03-12T04:15:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.262+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c107490 con 0x7fa88c101c50 2024-03-12T04:15:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.263+0000 7fa87f7fe640 1 -- 172.21.15.42:0/865463661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa884005190 con 0x7fa88c101c50 2024-03-12T04:15:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.264+0000 7fa87f7fe640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa8780a1a30 0x7fa8780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.264+0000 7fa8923b3640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa8780a1a30 0x7fa8780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.264+0000 7fa87f7fe640 1 -- 172.21.15.42:0/865463661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa8840cd600 con 0x7fa88c101c50 2024-03-12T04:15:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.264+0000 7fa8923b3640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa8780a1a30 0x7fa8780a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa8880058c0 tx=0x7fa88800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.268+0000 7fa87f7fe640 1 -- 172.21.15.42:0/865463661 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa884096840 con 0x7fa88c101c50 2024-03-12T04:15:02.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:02 smithi119 ceph-mon[36119]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-12T04:15:02.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1179656737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:15:02.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/477042617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:15:02.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1441141766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:15:02.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:02 smithi136 ceph-mon[35571]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-12T04:15:02.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1179656737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:15:02.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/477042617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:15:02.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1441141766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:15:02.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.490+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa88c10b960 con 0x7fa88c101c50 2024-03-12T04:15:02.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.491+0000 7fa87f7fe640 1 -- 172.21.15.42:0/865463661 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fa884099f00 con 0x7fa88c101c50 2024-03-12T04:15:02.493 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:15:02.494 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:15:02.494 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:15:02.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa8780a1a30 msgr2=0x7fa8780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa8780a1a30 0x7fa8780a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa8880058c0 tx=0x7fa88800a2d0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101c50 msgr2=0x7fa88c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101c50 0x7fa88c19d650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa88400eaa0 tx=0x7fa88400ef60 comp rx=0 tx=0).stop 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 shutdown_connections 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c101c50 0x7fa88c19d650 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa88c101290 0x7fa88c1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fa8780a1a30 0x7fa8780a3ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 --2- 172.21.15.42:0/865463661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa88c1000a0 0x7fa88c1960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 >> 172.21.15.42:0/865463661 conn(0x7fa88c0fb610 msgr2=0x7fa88c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 shutdown_connections 2024-03-12T04:15:02.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.493+0000 7fa894617640 1 -- 172.21.15.42:0/865463661 wait complete. 2024-03-12T04:15:02.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.607+0000 7f72686aa640 1 -- 172.21.15.42:0/4194410013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 msgr2=0x7f72601004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.607+0000 7f72686aa640 1 --2- 172.21.15.42:0/4194410013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f72601004c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7258009900 tx=0x7f725802bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- 172.21.15.42:0/4194410013 shutdown_connections 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 --2- 172.21.15.42:0/4194410013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7260101c50 0x7f72601084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 --2- 172.21.15.42:0/4194410013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7260101290 0x7f7260101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 --2- 172.21.15.42:0/4194410013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f72601004c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- 172.21.15.42:0/4194410013 >> 172.21.15.42:0/4194410013 conn(0x7f72600fb610 msgr2=0x7f72600fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- 172.21.15.42:0/4194410013 shutdown_connections 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- 172.21.15.42:0/4194410013 wait complete. 2024-03-12T04:15:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 Processor -- start 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- start start 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f7260197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7260101290 0x7f7260197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7260101c50 0x7f726019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726019ec20 con 0x7f72601000a0 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726019ed90 con 0x7f7260101c50 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f72686aa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726019ef50 con 0x7f7260101290 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f7266446640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f7260197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f7266446640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f7260197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57534/0 (socket says 172.21.15.42:57534) 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f7266446640 1 -- 172.21.15.42:0/2102987711 learned_addr learned my addr 172.21.15.42:0/2102987711 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f7265c45640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7260101290 0x7f7260197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.608+0000 7f7266c47640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7260101c50 0x7f726019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f7266446640 1 -- 172.21.15.42:0/2102987711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7260101290 msgr2=0x7f7260197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f7266446640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7260101290 0x7f7260197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f7266446640 1 -- 172.21.15.42:0/2102987711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7260101c50 msgr2=0x7f726019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f7266446640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7260101c50 0x7f726019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f7266446640 1 -- 172.21.15.42:0/2102987711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7258009560 con 0x7f72601000a0 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f7266c47640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7260101c50 0x7f726019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f7266446640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f7260197130 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f72580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f725f7fe640 1 -- 172.21.15.42:0/2102987711 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72580044e0 con 0x7f72601000a0 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f725f7fe640 1 -- 172.21.15.42:0/2102987711 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f725802de90 con 0x7f72601000a0 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726019f1d0 con 0x7f72601000a0 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f725f7fe640 1 -- 172.21.15.42:0/2102987711 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258005430 con 0x7f72601000a0 2024-03-12T04:15:02.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.609+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726019f6e0 con 0x7f72601000a0 2024-03-12T04:15:02.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.610+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7260107440 con 0x7f72601000a0 2024-03-12T04:15:02.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.611+0000 7f725f7fe640 1 -- 172.21.15.42:0/2102987711 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7258005590 con 0x7f72601000a0 2024-03-12T04:15:02.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.612+0000 7f725f7fe640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f72480a1a30 0x7f72480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.612+0000 7f7265c45640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f72480a1a30 0x7f72480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.612+0000 7f725f7fe640 1 -- 172.21.15.42:0/2102987711 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f72580efdb0 con 0x7f72601000a0 2024-03-12T04:15:02.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.612+0000 7f7265c45640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f72480a1a30 0x7f72480a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7260198640 tx=0x7f725400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.615+0000 7f725f7fe640 1 -- 172.21.15.42:0/2102987711 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72580b8f90 con 0x7f72601000a0 2024-03-12T04:15:02.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.799+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f726002cdd0 con 0x7f72601000a0 2024-03-12T04:15:02.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.801+0000 7f725f7fe640 1 -- 172.21.15.42:0/2102987711 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22608 (secure 0 0 0) 0x7f72580bc650 con 0x7f72601000a0 2024-03-12T04:15:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:15:02.804 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:13:45.923160+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f72480a1a30 msgr2=0x7f72480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f72480a1a30 0x7f72480a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7260198640 tx=0x7f725400a2d0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 msgr2=0x7f7260197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f7260197130 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f72580042f0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 shutdown_connections 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f72480a1a30 0x7f72480a3ef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7260101c50 0x7f726019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7260101290 0x7f7260197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 --2- 172.21.15.42:0/2102987711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72601000a0 0x7f7260197130 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 >> 172.21.15.42:0/2102987711 conn(0x7f72600fb610 msgr2=0x7f7260104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 shutdown_connections 2024-03-12T04:15:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.803+0000 7f72686aa640 1 -- 172.21.15.42:0/2102987711 wait complete. 2024-03-12T04:15:02.817 DEBUG:tasks.fwd_scrub:Yielding 2024-03-12T04:15:02.817 INFO:teuthology.run_tasks:Running task exec... 2024-03-12T04:15:02.827 INFO:teuthology.task.exec:Executing custom commands... 2024-03-12T04:15:02.827 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T04:15:02.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-12T04:15:02.828 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-12T04:15:02.828 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:15:02.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:15:02.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 -- 172.21.15.42:0/1325114114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1027d0 msgr2=0x7f79ec102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 --2- 172.21.15.42:0/1325114114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1027d0 0x7f79ec102bf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009ca0 tx=0x7f79dc02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 -- 172.21.15.42:0/1325114114 shutdown_connections 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 --2- 172.21.15.42:0/1325114114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0ffc00 0x7f79ec1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 --2- 172.21.15.42:0/1325114114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec0ff240 0x7f79ec0ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 --2- 172.21.15.42:0/1325114114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1027d0 0x7f79ec102bf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 -- 172.21.15.42:0/1325114114 >> 172.21.15.42:0/1325114114 conn(0x7f79ec0fada0 msgr2=0x7f79ec0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 -- 172.21.15.42:0/1325114114 shutdown_connections 2024-03-12T04:15:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.929+0000 7f79f4725640 1 -- 172.21.15.42:0/1325114114 wait complete. 2024-03-12T04:15:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 Processor -- start 2024-03-12T04:15:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 -- start start 2024-03-12T04:15:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0ff240 0x7f79ec101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec0ffc00 0x7f79ec100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec1027d0 0x7f79ec100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1011b0 con 0x7f79ec0ffc00 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec101320 con 0x7f79ec0ff240 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f4725640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1a11b0 con 0x7f79ec1027d0 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec0ffc00 0x7f79ec100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec0ffc00 0x7f79ec100510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57556/0 (socket says 172.21.15.42:57556) 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f24c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0ff240 0x7f79ec101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 -- 172.21.15.42:0/3693835430 learned_addr learned my addr 172.21.15.42:0/3693835430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f2cc2640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec1027d0 0x7f79ec100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 -- 172.21.15.42:0/3693835430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec1027d0 msgr2=0x7f79ec100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec1027d0 0x7f79ec100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 -- 172.21.15.42:0/3693835430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0ff240 msgr2=0x7f79ec101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0ff240 0x7f79ec101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f1cc0640 1 -- 172.21.15.42:0/3693835430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009900 con 0x7f79ec0ffc00 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.930+0000 7f79f24c1640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0ff240 0x7f79ec101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:15:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.931+0000 7f79f1cc0640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec0ffc00 0x7f79ec100510 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f79e000d980 tx=0x7f79e000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.931+0000 7f79eb7fe640 1 -- 172.21.15.42:0/3693835430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0009c30 con 0x7f79ec0ffc00 2024-03-12T04:15:02.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.931+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec1a1460 con 0x7f79ec0ffc00 2024-03-12T04:15:02.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.931+0000 7f79eb7fe640 1 -- 172.21.15.42:0/3693835430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f79e000b660 con 0x7f79ec0ffc00 2024-03-12T04:15:02.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.931+0000 7f79eb7fe640 1 -- 172.21.15.42:0/3693835430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000f620 con 0x7f79ec0ffc00 2024-03-12T04:15:02.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.931+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec1a19c0 con 0x7f79ec0ffc00 2024-03-12T04:15:02.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 -- 172.21.15.42:0/2512196307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854106ea0 msgr2=0x7f28541092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 --2- 172.21.15.42:0/2512196307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854106ea0 0x7f28541092b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 -- 172.21.15.42:0/2512196307 shutdown_connections 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 --2- 172.21.15.42:0/2512196307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854106ea0 0x7f28541092b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 --2- 172.21.15.42:0/2512196307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2854104570 0x7f2854106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 --2- 172.21.15.42:0/2512196307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2854101c40 0x7f2854104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 -- 172.21.15.42:0/2512196307 >> 172.21.15.42:0/2512196307 conn(0x7f28540fb610 msgr2=0x7f28540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 -- 172.21.15.42:0/2512196307 shutdown_connections 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f2859d56640 1 -- 172.21.15.42:0/2512196307 wait complete. 2024-03-12T04:15:02.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.932+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec108710 con 0x7f79ec0ffc00 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 Processor -- start 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 -- start start 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2854101c40 0x7f2854197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854104570 0x7f2854197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2854106ea0 0x7f285419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285419ec40 con 0x7f2854104570 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285419edb0 con 0x7f2854106ea0 2024-03-12T04:15:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2859d56640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285419ef70 con 0x7f2854101c40 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854104570 0x7f2854197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854104570 0x7f2854197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57560/0 (socket says 172.21.15.42:57560) 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 -- 172.21.15.42:0/1361134793 learned_addr learned my addr 172.21.15.42:0/1361134793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f28537fe640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2854101c40 0x7f2854197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2853fff640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2854106ea0 0x7f285419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 -- 172.21.15.42:0/1361134793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2854101c40 msgr2=0x7f2854197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2854101c40 0x7f2854197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 -- 172.21.15.42:0/1361134793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2854106ea0 msgr2=0x7f285419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2854106ea0 0x7f285419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.933+0000 7f2852ffd640 1 -- 172.21.15.42:0/1361134793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c009900 con 0x7f2854104570 2024-03-12T04:15:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.934+0000 7f79eb7fe640 1 -- 172.21.15.42:0/3693835430 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f79e0010460 con 0x7f79ec0ffc00 2024-03-12T04:15:02.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.934+0000 7f2852ffd640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854104570 0x7f2854197690 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f284800e9c0 tx=0x7f284800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.934+0000 7f79eb7fe640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79d40a1b50 0x7f79d40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.934+0000 7f79f24c1640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79d40a1b50 0x7f79d40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.934+0000 7f79eb7fe640 1 -- 172.21.15.42:0/3693835430 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f79e00cced0 con 0x7f79ec0ffc00 2024-03-12T04:15:02.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.935+0000 7f79f24c1640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79d40a1b50 0x7f79d40a4010 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f79dc02c700 tx=0x7f79dc0325d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.939+0000 7f2850ff9640 1 -- 172.21.15.42:0/1361134793 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848009700 con 0x7f2854104570 2024-03-12T04:15:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.939+0000 7f2850ff9640 1 -- 172.21.15.42:0/1361134793 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f284800cd80 con 0x7f2854104570 2024-03-12T04:15:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.939+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285419f250 con 0x7f2854104570 2024-03-12T04:15:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.939+0000 7f2850ff9640 1 -- 172.21.15.42:0/1361134793 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848010430 con 0x7f2854104570 2024-03-12T04:15:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.939+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285419f7b0 con 0x7f2854104570 2024-03-12T04:15:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.939+0000 7f79eb7fe640 1 -- 172.21.15.42:0/3693835430 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79e00961c0 con 0x7f79ec0ffc00 2024-03-12T04:15:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.941+0000 7f2850ff9640 1 -- 172.21.15.42:0/1361134793 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2848004f50 con 0x7f2854104570 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.941+0000 7f285ca08740 1 Processor -- start 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.941+0000 7f285ca08740 1 -- start start 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.941+0000 7f2850ff9640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f283c0a1ab0 0x7f283c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f285ca08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a304c0 0x559725a308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f285ca08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a31b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f285ca08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 0x559725a388e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f285ca08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972599ff90 con 0x559725a31690 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f285ca08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972599d4c0 con 0x559725a32050 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f285ca08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559725998ca0 con 0x559725a304c0 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2853fff640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f283c0a1ab0 0x7f283c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a31b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a31b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57564/0 (socket says 172.21.15.42:57564) 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 -- 172.21.15.42:0/1780332610 learned_addr learned my addr 172.21.15.42:0/1780332610 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2850ff9640 1 -- 172.21.15.42:0/1361134793 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2848014070 con 0x7f2854104570 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f28537fe640 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a304c0 0x559725a308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 -- 172.21.15.42:0/1780332610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a304c0 msgr2=0x559725a308e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a304c0 0x559725a308e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 -- 172.21.15.42:0/1780332610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 msgr2=0x559725a388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 0x559725a388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 -- 172.21.15.42:0/1780332610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972595b0b0 con 0x559725a31690 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2852ffd640 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a31b10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f284809a6a0 tx=0x7f2848005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddf94eacbd60fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2853fff640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f283c0a1ab0 0x7f283c0a3f70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f284c02c630 tx=0x7f284c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2839ffb640 1 -- 172.21.15.42:0/1780332610 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284809a070 con 0x559725a31690 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2839ffb640 1 -- 172.21.15.42:0/1780332610 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f284809ae10 con 0x559725a31690 2024-03-12T04:15:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.942+0000 7f2839ffb640 1 -- 172.21.15.42:0/1780332610 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848096d80 con 0x559725a31690 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.943+0000 7f285ca08740 1 -- 172.21.15.42:0/1780332610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 msgr2=0x559725a31b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.943+0000 7f285ca08740 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a31b10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f284809a6a0 tx=0x7f2848005720 comp rx=0 tx=0).stop 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.943+0000 7f285ca08740 1 -- 172.21.15.42:0/1780332610 shutdown_connections 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.943+0000 7f285ca08740 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 0x559725a388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.943+0000 7f285ca08740 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a31b10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.943+0000 7f285ca08740 1 --2- 172.21.15.42:0/1780332610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a304c0 0x559725a308e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.943+0000 7f285ca08740 1 -- 172.21.15.42:0/1780332610 >> 172.21.15.42:0/1780332610 conn(0x559725a2b6a0 msgr2=0x559725a2baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.944+0000 7f285ca08740 1 -- 172.21.15.42:0/1780332610 shutdown_connections 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.944+0000 7f285ca08740 1 -- 172.21.15.42:0/1780332610 wait complete. 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.944+0000 7f285ca08740 1 -- start start 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.944+0000 7f285ca08740 1 Processor -- start 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.944+0000 7f285ca08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a304c0 0x559725a3a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.944+0000 7f285ca08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a3ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f285ca08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f285ca08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972593df40 con 0x559725a304c0 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f285ca08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972566c4b0 con 0x559725a31690 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f285ca08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972591e670 con 0x559725a32050 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a304c0 0x559725a3a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a304c0 0x559725a3a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57578/0 (socket says 172.21.15.42:57578) 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 -- 172.21.15.42:0/953605720 learned_addr learned my addr 172.21.15.42:0/953605720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f285ca08740 10 client.0 _ll_register_callbacks cb 0x55972598bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f2852ffd640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a3ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f2853fff640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 -- 172.21.15.42:0/953605720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 msgr2=0x559725a41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a41c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 -- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 msgr2=0x559725a3ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a3ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 -- 172.21.15.42:0/953605720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285419f7b0 con 0x559725a304c0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f2852ffd640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a3ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f28537fe640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a304c0 0x559725a3a6d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2844012700 tx=0x7f2844012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c274c221751a587d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f2832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f2832ffd640 1 -- 172.21.15.42:0/953605720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844017070 con 0x559725a304c0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f2832ffd640 1 -- 172.21.15.42:0/953605720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f284400f8c0 con 0x559725a304c0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559725972f40 con 0x559725a304c0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f2832ffd640 1 -- 172.21.15.42:0/953605720 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28440122e0 con 0x559725a304c0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.945+0000 7f285ca08740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559725997320 con 0x559725a304c0 2024-03-12T04:15:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f2832ffd640 1 -- 172.21.15.42:0/953605720 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x559725972f40 con 0x559725a304c0 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f2832ffd640 1 -- 172.21.15.42:0/953605720 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2844004830 con 0x559725a304c0 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f285ca08740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f285ca08740 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f285ca08740 20 client.15480 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f285ca08740 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a46980 0x559725a48da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f285ca08740 4 client.15480 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55972598f310 con 0x559725a46980 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.946+0000 7f2852ffd640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a46980 0x559725a48da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.947+0000 7f2852ffd640 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a46980 0x559725a48da0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0844eac9ab90c2 server_cookie=3507f59266f7ad8b in_seq=0 out_seq=0 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.947+0000 7f2832ffd640 10 client.15480 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.947+0000 7f2832ffd640 1 -- 172.21.15.42:0/953605720 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55972598f310 con 0x559725a46980 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.947+0000 7f2832ffd640 10 client.15480 handle_command_reply: tid=0 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.947+0000 7f285ca08740 1 client.15480 shutdown 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a304c0 msgr2=0x559725a3a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a304c0 0x559725a3a6d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2844012700 tx=0x7f2844012bc0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 shutdown_connections 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a46980 0x559725a48da0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a3ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 --2- 172.21.15.42:0/953605720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a304c0 0x559725a3a6d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 >> 172.21.15.42:0/953605720 conn(0x559725a2b6a0 msgr2=0x559725a2a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f2832ffd640 0 client.15480 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 shutdown_connections 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 1 -- 172.21.15.42:0/953605720 wait complete. 2024-03-12T04:15:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.948+0000 7f285ca08740 20 client.15480 trim_cache size 0 max 16384 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 Processor -- start 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 -- start start 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a35240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a35780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a41c10 0x559725a39d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972598f310 con 0x559725a41c10 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559725972f40 con 0x559725a31690 2024-03-12T04:15:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f285ca08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559725a44f50 con 0x559725a32050 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a41c10 0x559725a39d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a41c10 0x559725a39d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57588/0 (socket says 172.21.15.42:57588) 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 -- 172.21.15.42:0/3761475289 learned_addr learned my addr 172.21.15.42:0/3761475289 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f28537fe640 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a35240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2852ffd640 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a35780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 -- 172.21.15.42:0/3761475289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 msgr2=0x559725a35780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a35780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 -- 172.21.15.42:0/3761475289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 msgr2=0x559725a35240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a35240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f2853fff640 1 -- 172.21.15.42:0/3761475289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559725933c00 con 0x559725a41c10 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.963+0000 7f28537fe640 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.964+0000 7f2853fff640 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a41c10 0x559725a39d20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662e124a1caed872 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.964+0000 7f28397fa640 1 -- 172.21.15.42:0/3761475289 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c034790 con 0x559725a41c10 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.964+0000 7f28397fa640 1 -- 172.21.15.42:0/3761475289 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f284c003e20 con 0x559725a41c10 2024-03-12T04:15:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.964+0000 7f28397fa640 1 -- 172.21.15.42:0/3761475289 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c004790 con 0x559725a41c10 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- 172.21.15.42:0/3761475289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a41c10 msgr2=0x559725a39d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a41c10 0x559725a39d20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c040000 comp rx=0 tx=0).stop 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- 172.21.15.42:0/3761475289 shutdown_connections 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a41c10 0x559725a39d20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a32050 0x559725a35780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 --2- 172.21.15.42:0/3761475289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a31690 0x559725a35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- 172.21.15.42:0/3761475289 >> 172.21.15.42:0/3761475289 conn(0x559725a2b6a0 msgr2=0x559725a39280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- 172.21.15.42:0/3761475289 shutdown_connections 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- 172.21.15.42:0/3761475289 wait complete. 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- start start 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 Processor -- start 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a41600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 0x559725a3e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a41c10 0x559725a3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559725a3ee50 con 0x559725a31690 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559725a9cee0 con 0x559725a32050 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559725a9d0a0 con 0x559725a41c10 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f285ca08740 10 client.0 _ll_register_callbacks cb 0x7f2848004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f28537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f28537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a41600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57596/0 (socket says 172.21.15.42:57596) 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.965+0000 7f28537fe640 1 -- 172.21.15.42:0/728900823 learned_addr learned my addr 172.21.15.42:0/728900823 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f2852ffd640 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 0x559725a3e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f2853fff640 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a41c10 0x559725a3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f28537fe640 1 -- 172.21.15.42:0/728900823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a41c10 msgr2=0x559725a3e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f28537fe640 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a41c10 0x559725a3e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f28537fe640 1 -- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 msgr2=0x559725a3e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f28537fe640 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 0x559725a3e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f28537fe640 1 -- 172.21.15.42:0/728900823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2854109e90 con 0x559725a31690 2024-03-12T04:15:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f28537fe640 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a41600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2844000c00 tx=0x7f28440052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b803fdfabc21a580 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f2832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f2832ffd640 1 -- 172.21.15.42:0/728900823 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844017040 con 0x559725a31690 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f2832ffd640 1 -- 172.21.15.42:0/728900823 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f284400f990 con 0x559725a31690 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559725a26f50 con 0x559725a31690 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f2832ffd640 1 -- 172.21.15.42:0/728900823 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f284400fb70 con 0x559725a31690 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f285ca08740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559725959910 con 0x559725a31690 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.966+0000 7f2832ffd640 1 -- 172.21.15.42:0/728900823 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x559725a26f50 con 0x559725a31690 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f2832ffd640 1 -- 172.21.15.42:0/728900823 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2844004020 con 0x559725a31690 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f285ca08740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f285ca08740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f285ca08740 20 client.15486 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f285ca08740 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a9eb50 0x559725aa0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f285ca08740 4 client.15486 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:15:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5597259be900 con 0x559725a9eb50 2024-03-12T04:15:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f2852ffd640 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a9eb50 0x559725aa0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f2852ffd640 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a9eb50 0x559725aa0f70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe71319af94f383 server_cookie=1140d533a9d74ebc in_seq=0 out_seq=0 2024-03-12T04:15:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.967+0000 7f2832ffd640 10 client.15486 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f2832ffd640 1 -- 172.21.15.42:0/728900823 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5597259be900 con 0x559725a9eb50 2024-03-12T04:15:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f2832ffd640 10 client.15486 handle_command_reply: tid=0 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 client.15486 shutdown 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 msgr2=0x559725a41600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a41600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2844000c00 tx=0x7f28440052b0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 shutdown_connections 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559725a9eb50 0x559725aa0f70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559725a41c10 0x559725a3e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559725a32050 0x559725a3e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f2832ffd640 0 client.15486 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 --2- 172.21.15.42:0/728900823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559725a31690 0x559725a41600 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 >> 172.21.15.42:0/728900823 conn(0x559725a2b6a0 msgr2=0x559725a35240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 shutdown_connections 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 1 -- 172.21.15.42:0/728900823 wait complete. 2024-03-12T04:15:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.969+0000 7f285ca08740 20 client.15486 trim_cache size 0 max 16384 2024-03-12T04:15:02.978 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:15:02.978 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:15:02.978 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "ab7ea0ed-ce23-4e22-b2d1-4befa62d5945", 2024-03-12T04:15:02.978 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:15:02.978 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f283c0a1ab0 msgr2=0x7f283c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f283c0a1ab0 0x7f283c0a3f70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f284c02c630 tx=0x7f284c005f90 comp rx=0 tx=0).stop 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854104570 msgr2=0x7f2854197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854104570 0x7f2854197690 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f284800e9c0 tx=0x7f284800ee80 comp rx=0 tx=0).stop 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 shutdown_connections 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f283c0a1ab0 0x7f283c0a3f70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2854106ea0 0x7f285419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2854104570 0x7f2854197690 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 --2- 172.21.15.42:0/1361134793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2854101c40 0x7f2854197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 >> 172.21.15.42:0/1361134793 conn(0x7f28540fb610 msgr2=0x7f28541024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 shutdown_connections 2024-03-12T04:15:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:02.970+0000 7f2859d56640 1 -- 172.21.15.42:0/1361134793 wait complete. 2024-03-12T04:15:02.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:15:02.983 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:15:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:03 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/323119958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:15:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:03 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/865463661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:03 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2102987711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:15:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:03 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:15:03.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:03 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:03.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:03 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:15:03.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:03 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 -- 172.21.15.42:0/353422630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80104530 msgr2=0x7fba80106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 --2- 172.21.15.42:0/353422630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80104530 0x7fba80106920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fba70009900 tx=0x7fba7002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 -- 172.21.15.42:0/353422630 shutdown_connections 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 --2- 172.21.15.42:0/353422630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba80106e60 0x7fba801092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 --2- 172.21.15.42:0/353422630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80104530 0x7fba80106920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 --2- 172.21.15.42:0/353422630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba80101c00 0x7fba80103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 -- 172.21.15.42:0/353422630 >> 172.21.15.42:0/353422630 conn(0x7fba800fb5f0 msgr2=0x7fba800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 -- 172.21.15.42:0/353422630 shutdown_connections 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.092+0000 7fba85b8a640 1 -- 172.21.15.42:0/353422630 wait complete. 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 Processor -- start 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 -- start start 2024-03-12T04:15:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80101c00 0x7fba80197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba80104530 0x7fba80197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba80106e60 0x7fba8019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8019ec20 con 0x7fba80101c00 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8019ed90 con 0x7fba80106e60 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba85b8a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8019ef50 con 0x7fba80104530 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba7f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80101c00 0x7fba80197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba7f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80101c00 0x7fba80197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57614/0 (socket says 172.21.15.42:57614) 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba7f7fe640 1 -- 172.21.15.42:0/1457140849 learned_addr learned my addr 172.21.15.42:0/1457140849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba7ffff640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba80106e60 0x7fba8019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.093+0000 7fba7effd640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba80104530 0x7fba80197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7f7fe640 1 -- 172.21.15.42:0/1457140849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba80104530 msgr2=0x7fba80197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba80104530 0x7fba80197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7f7fe640 1 -- 172.21.15.42:0/1457140849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba80106e60 msgr2=0x7fba8019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba80106e60 0x7fba8019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7f7fe640 1 -- 172.21.15.42:0/1457140849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70009560 con 0x7fba80101c00 2024-03-12T04:15:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80101c00 0x7fba80197130 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d950 tx=0x7fba7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7cff9640 1 -- 172.21.15.42:0/1457140849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74009c30 con 0x7fba80101c00 2024-03-12T04:15:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8019f230 con 0x7fba80101c00 2024-03-12T04:15:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7cff9640 1 -- 172.21.15.42:0/1457140849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba74009d90 con 0x7fba80101c00 2024-03-12T04:15:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba7cff9640 1 -- 172.21.15.42:0/1457140849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7400f6b0 con 0x7fba80101c00 2024-03-12T04:15:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.094+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8019f710 con 0x7fba80101c00 2024-03-12T04:15:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7cff9640 1 -- 172.21.15.42:0/1457140849 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fba74010aa0 con 0x7fba80101c00 2024-03-12T04:15:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 Processor -- start 2024-03-12T04:15:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 -- start start 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de13129cc0 0x55de1312e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de131312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de13099280 con 0x55de1312ee30 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de130967b0 con 0x55de131292d0 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba88837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de13091f90 con 0x55de13129cc0 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de131312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de131312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57618/0 (socket says 172.21.15.42:57618) 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 -- 172.21.15.42:0/4145555330 learned_addr learned my addr 172.21.15.42:0/4145555330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de13129cc0 0x55de1312e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7ffff640 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7cff9640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fba680a1a30 0x7fba680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 -- 172.21.15.42:0/4145555330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de13129cc0 msgr2=0x55de1312e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de13129cc0 0x55de1312e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 -- 172.21.15.42:0/4145555330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 msgr2=0x55de131296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7effd640 1 -- 172.21.15.42:0/4145555330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de13050570 con 0x55de1312ee30 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.097+0000 7fba7ffff640 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.098+0000 7fba7ffff640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fba680a1a30 0x7fba680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.098+0000 7fba7cff9640 1 -- 172.21.15.42:0/1457140849 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fba740cd0f0 con 0x7fba80101c00 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.098+0000 7fba7effd640 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de131312c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fba70009900 tx=0x7fba70005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf494089c757f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.098+0000 7fba65ffb640 1 -- 172.21.15.42:0/4145555330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7002cbf0 con 0x55de1312ee30 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.098+0000 7fba65ffb640 1 -- 172.21.15.42:0/4145555330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba7002de90 con 0x55de1312ee30 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.098+0000 7fba7ffff640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fba680a1a30 0x7fba680a3ef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fba780098b0 tx=0x7fba78004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.098+0000 7fba65ffb640 1 -- 172.21.15.42:0/4145555330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70003a30 con 0x55de1312ee30 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 -- 172.21.15.42:0/4145555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 msgr2=0x55de131312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de131312c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fba70009900 tx=0x7fba70005e80 comp rx=0 tx=0).stop 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 -- 172.21.15.42:0/4145555330 shutdown_connections 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de131312c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de13129cc0 0x55de1312e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 --2- 172.21.15.42:0/4145555330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 -- 172.21.15.42:0/4145555330 >> 172.21.15.42:0/4145555330 conn(0x55de131247f0 msgr2=0x55de13124bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 -- 172.21.15.42:0/4145555330 shutdown_connections 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 -- 172.21.15.42:0/4145555330 wait complete. 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 -- start start 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 Processor -- start 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de13137b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.099+0000 7fba88837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de1312ee30 0x55de1313eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba88837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de12d65430 con 0x55de13129cc0 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba88837740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de12d21e00 con 0x55de131292d0 2024-03-12T04:15:03.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba88837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de13016fd0 con 0x55de1312ee30 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba88837740 10 client.0 _ll_register_callbacks cb 0x55de1306cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de13137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de13137b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57630/0 (socket says 172.21.15.42:57630) 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 -- 172.21.15.42:0/3248605985 learned_addr learned my addr 172.21.15.42:0/3248605985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7ffff640 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de1312ee30 0x55de1313eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 -- 172.21.15.42:0/3248605985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de1312ee30 msgr2=0x55de1313eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de1312ee30 0x55de1313eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 -- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 msgr2=0x55de131375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 -- 172.21.15.42:0/3248605985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba80109e90 con 0x55de13129cc0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba7effd640 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de13137b30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba70004570 tx=0x7fba70004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2169eb38dbfc8d67 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba5effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba5effd640 1 -- 172.21.15.42:0/3248605985 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70005350 con 0x55de13129cc0 2024-03-12T04:15:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de12d917e0 con 0x55de13129cc0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.100+0000 7fba5effd640 1 -- 172.21.15.42:0/3248605985 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fba700054b0 con 0x55de13129cc0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba5effd640 1 -- 172.21.15.42:0/3248605985 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba70005690 con 0x55de13129cc0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55de13090610 con 0x55de13129cc0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba5effd640 1 -- 172.21.15.42:0/3248605985 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55de12d917e0 con 0x55de13129cc0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba5effd640 1 -- 172.21.15.42:0/3248605985 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fba7003c040 con 0x55de13129cc0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 20 client.15498 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de13143350 0x55de13145770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 4 client.15498 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de12dce720 con 0x55de13143350 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.101+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de13143350 0x55de13145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.102+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de13143350 0x55de13145770 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5718091e79d5c0b server_cookie=e4690af146fa3917 in_seq=0 out_seq=0 2024-03-12T04:15:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.102+0000 7fba5effd640 10 client.15498 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.102+0000 7fba5effd640 1 -- 172.21.15.42:0/3248605985 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fba7407ea90 con 0x55de13143350 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.102+0000 7fba5effd640 10 client.15498 handle_command_reply: tid=0 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.102+0000 7fba88837740 1 client.15498 shutdown 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 msgr2=0x55de13137b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de13137b30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba70004570 tx=0x7fba70004bb0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 shutdown_connections 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de13143350 0x55de13145770 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de1312ee30 0x55de1313eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de13137b30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba5effd640 0 client.15498 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 --2- 172.21.15.42:0/3248605985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de131292d0 0x55de131375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 >> 172.21.15.42:0/3248605985 conn(0x55de131247f0 msgr2=0x55de13123a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 shutdown_connections 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 1 -- 172.21.15.42:0/3248605985 wait complete. 2024-03-12T04:15:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.103+0000 7fba88837740 20 client.15498 trim_cache size 0 max 16384 2024-03-12T04:15:03.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 Processor -- start 2024-03-12T04:15:03.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 -- start start 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1312af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de1312b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1312ee30 0x55de1313eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de13128e90 con 0x55de13129cc0 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de12d917e0 con 0x55de1312ee30 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba88837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1312a3b0 con 0x55de131292d0 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba7effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de1312b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba7effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de1312b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57642/0 (socket says 172.21.15.42:57642) 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.118+0000 7fba7effd640 1 -- 172.21.15.42:0/863640117 learned_addr learned my addr 172.21.15.42:0/863640117 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7ffff640 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1312ee30 0x55de1313eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1312af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7effd640 1 -- 172.21.15.42:0/863640117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 msgr2=0x55de1312af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7effd640 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1312af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7effd640 1 -- 172.21.15.42:0/863640117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1312ee30 msgr2=0x55de1313eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7effd640 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1312ee30 0x55de1313eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7effd640 1 -- 172.21.15.42:0/863640117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1302cac0 con 0x55de13129cc0 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba7effd640 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de1312b480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fba70005dd0 tx=0x7fba70005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b8b1734231ca21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba657fa640 1 -- 172.21.15.42:0/863640117 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70004dc0 con 0x55de13129cc0 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba657fa640 1 -- 172.21.15.42:0/863640117 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba7003e430 con 0x55de13129cc0 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.119+0000 7fba657fa640 1 -- 172.21.15.42:0/863640117 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba700346a0 con 0x55de13129cc0 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 -- 172.21.15.42:0/863640117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 msgr2=0x55de1312b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de1312b480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fba70005dd0 tx=0x7fba70005e80 comp rx=0 tx=0).stop 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 -- 172.21.15.42:0/863640117 shutdown_connections 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1312ee30 0x55de1313eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de13129cc0 0x55de1312b480 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 --2- 172.21.15.42:0/863640117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1312af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 -- 172.21.15.42:0/863640117 >> 172.21.15.42:0/863640117 conn(0x55de131247f0 msgr2=0x55de131375f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 -- 172.21.15.42:0/863640117 shutdown_connections 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 -- 172.21.15.42:0/863640117 wait complete. 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 -- start start 2024-03-12T04:15:03.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 Processor -- start 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1313d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de13129cc0 0x55de1313dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de1313e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.120+0000 7fba88837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de13199980 con 0x55de1312ee30 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba88837740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de13199af0 con 0x55de13129cc0 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba88837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de13199cb0 con 0x55de131292d0 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba88837740 10 client.0 _ll_register_callbacks cb 0x7fba70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de1313e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de1313e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57652/0 (socket says 172.21.15.42:57652) 2024-03-12T04:15:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 -- 172.21.15.42:0/1545426661 learned_addr learned my addr 172.21.15.42:0/1545426661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7effd640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de13129cc0 0x55de1313dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1313d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 -- 172.21.15.42:0/1545426661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 msgr2=0x55de1313d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1313d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 -- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de13129cc0 msgr2=0x55de1313dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de13129cc0 0x55de1313dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 -- 172.21.15.42:0/1545426661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de13090610 con 0x55de1312ee30 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1313d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba7ffff640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de1313e150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fba78009830 tx=0x7fba78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4c590e93a84252 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba5effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba5effd640 1 -- 172.21.15.42:0/1545426661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78024070 con 0x55de1312ee30 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba5effd640 1 -- 172.21.15.42:0/1545426661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fba7801fb60 con 0x55de1312ee30 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba7002de90 con 0x55de1312ee30 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.121+0000 7fba5effd640 1 -- 172.21.15.42:0/1545426661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba7801fd20 con 0x55de1312ee30 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55de130526a0 con 0x55de1312ee30 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba5effd640 1 -- 172.21.15.42:0/1545426661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba7002de90 con 0x55de1312ee30 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba5effd640 1 -- 172.21.15.42:0/1545426661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fba78032050 con 0x55de1312ee30 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 20 client.15504 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de1319b8a0 0x55de1319dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 4 client.15504 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de12e1d890 con 0x55de1319b8a0 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.122+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de1319b8a0 0x55de1319dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba7f7fe640 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de1319b8a0 0x55de1319dcc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ead28b1d9efc82 server_cookie=2333f3c9c2c53c27 in_seq=0 out_seq=0 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba5effd640 10 client.15504 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba5effd640 1 -- 172.21.15.42:0/1545426661 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55de12e1d890 con 0x55de1319b8a0 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba5effd640 10 client.15504 handle_command_reply: tid=0 2024-03-12T04:15:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 client.15504 shutdown 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 msgr2=0x55de1313e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de1313e150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fba78009830 tx=0x7fba78004ab0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 shutdown_connections 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55de1319b8a0 0x55de1319dcc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de1312ee30 0x55de1313e150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de13129cc0 0x55de1313dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba5effd640 0 client.15504 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 --2- 172.21.15.42:0/1545426661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de131292d0 0x55de1313d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 >> 172.21.15.42:0/1545426661 conn(0x55de131247f0 msgr2=0x55de13137590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 shutdown_connections 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 1 -- 172.21.15.42:0/1545426661 wait complete. 2024-03-12T04:15:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.123+0000 7fba88837740 20 client.15504 trim_cache size 0 max 16384 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "ab7ea0ed-ce23-4e22-b2d1-4befa62d5945": { 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "ab7ea0ed-ce23-4e22-b2d1-4befa62d5945", 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:15:03.132 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fba680a1a30 msgr2=0x7fba680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fba680a1a30 0x7fba680a3ef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fba780098b0 tx=0x7fba78004a40 comp rx=0 tx=0).stop 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80101c00 msgr2=0x7fba80197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80101c00 0x7fba80197130 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d950 tx=0x7fba7400de10 comp rx=0 tx=0).stop 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 shutdown_connections 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7fba680a1a30 0x7fba680a3ef0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba80106e60 0x7fba8019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba80104530 0x7fba80197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 --2- 172.21.15.42:0/1457140849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba80101c00 0x7fba80197130 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 >> 172.21.15.42:0/1457140849 conn(0x7fba800fb5f0 msgr2=0x7fba80102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 shutdown_connections 2024-03-12T04:15:03.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.124+0000 7fba85b8a640 1 -- 172.21.15.42:0/1457140849 wait complete. 2024-03-12T04:15:03.137 INFO:tasks.cephfs.filesystem:scrub status for tag:ab7ea0ed-ce23-4e22-b2d1-4befa62d5945 - {'path': '/', 'tag': 'ab7ea0ed-ce23-4e22-b2d1-4befa62d5945', 'options': 'recursive,force'} 2024-03-12T04:15:03.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:03.180+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f79ec02cdd0 con 0x7f79ec0ffc00 2024-03-12T04:15:03.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:03 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/323119958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:15:03.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:03 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/865463661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:03.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:03 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2102987711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:15:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:03 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:15:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:03 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:03 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:15:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:03 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:03 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/323119958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:15:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:03 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/865463661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:03 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2102987711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:15:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:03 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:15:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:03 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:03 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:15:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:03 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.028+0000 7f79eb7fe640 1 -- 172.21.15.42:0/3693835430 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f79e00200a0 con 0x7f79ec0ffc00 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79d40a1b50 msgr2=0x7f79d40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79d40a1b50 0x7f79d40a4010 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f79dc02c700 tx=0x7f79dc0325d0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec0ffc00 msgr2=0x7f79ec100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec0ffc00 0x7f79ec100510 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f79e000d980 tx=0x7f79e000de40 comp rx=0 tx=0).stop 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 shutdown_connections 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec1027d0 0x7f79ec100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec0ffc00 0x7f79ec100510 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79d40a1b50 0x7f79d40a4010 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 --2- 172.21.15.42:0/3693835430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0ff240 0x7f79ec101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 >> 172.21.15.42:0/3693835430 conn(0x7f79ec0fada0 msgr2=0x7f79ec1058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 shutdown_connections 2024-03-12T04:15:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.031+0000 7f79f4725640 1 -- 172.21.15.42:0/3693835430 wait complete. 2024-03-12T04:15:04.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-12T04:15:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- 172.21.15.42:0/22295873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 msgr2=0x7f58cc1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 --2- 172.21.15.42:0/22295873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc1004c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009900 tx=0x7f58c402bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- 172.21.15.42:0/22295873 shutdown_connections 2024-03-12T04:15:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 --2- 172.21.15.42:0/22295873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58cc101c50 0x7f58cc1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 --2- 172.21.15.42:0/22295873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58cc101290 0x7f58cc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 --2- 172.21.15.42:0/22295873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc1004c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- 172.21.15.42:0/22295873 >> 172.21.15.42:0/22295873 conn(0x7f58cc0fb610 msgr2=0x7f58cc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- 172.21.15.42:0/22295873 shutdown_connections 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- 172.21.15.42:0/22295873 wait complete. 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 Processor -- start 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- start start 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58cc101290 0x7f58cc19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58cc101c50 0x7f58cc19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc19ec10 con 0x7f58cc1000a0 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc19ed80 con 0x7f58cc101290 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.165+0000 7f58d3377640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc19ef40 con 0x7f58cc101c50 2024-03-12T04:15:04.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc069390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57700/0 (socket says 172.21.15.42:57700) 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 -- 172.21.15.42:0/1076612256 learned_addr learned my addr 172.21.15.42:0/1076612256 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d0912640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58cc101290 0x7f58cc19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1914640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58cc101c50 0x7f58cc19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 -- 172.21.15.42:0/1076612256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58cc101c50 msgr2=0x7f58cc19e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58cc101c50 0x7f58cc19e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 -- 172.21.15.42:0/1076612256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58cc101290 msgr2=0x7f58cc19be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58cc101290 0x7f58cc19be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 -- 172.21.15.42:0/1076612256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c4009560 con 0x7f58cc1000a0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d1113640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc069390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f58c402c7d0 tx=0x7f58c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58ca7fc640 1 -- 172.21.15.42:0/1076612256 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4004500 con 0x7f58cc1000a0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc19f1c0 con 0x7f58cc1000a0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58ca7fc640 1 -- 172.21.15.42:0/1076612256 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58c402de90 con 0x7f58cc1000a0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58ca7fc640 1 -- 172.21.15.42:0/1076612256 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4005350 con 0x7f58cc1000a0 2024-03-12T04:15:04.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.166+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc19f6d0 con 0x7f58cc1000a0 2024-03-12T04:15:04.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.168+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58cc04eee0 con 0x7f58cc1000a0 2024-03-12T04:15:04.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.168+0000 7f58ca7fc640 1 -- 172.21.15.42:0/1076612256 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 136296+0+0 (secure 0 0 0) 0x7f58c40054b0 con 0x7f58cc1000a0 2024-03-12T04:15:04.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.169+0000 7f58ca7fc640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f58b40a1b20 0x7f58b40a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.169+0000 7f58d0912640 1 -- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f58b40a1b20 msgr2=0x7f58b40a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:04.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.169+0000 7f58ca7fc640 1 -- 172.21.15.42:0/1076612256 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f58c40ef490 con 0x7f58cc1000a0 2024-03-12T04:15:04.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.169+0000 7f58d0912640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f58b40a1b20 0x7f58b40a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:15:04.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.173+0000 7f58ca7fc640 1 -- 172.21.15.42:0/1076612256 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58c40b86d0 con 0x7f58cc1000a0 2024-03-12T04:15:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:04 smithi042 ceph-mon[32260]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-12T04:15:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3693835430' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-12T04:15:04.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:04 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:15:04.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: ignoring --setuser ceph since I am not root 2024-03-12T04:15:04.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: ignoring --setgroup ceph since I am not root 2024-03-12T04:15:04.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.356+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f58cc1074e0 con 0x7f58cc1000a0 2024-03-12T04:15:04.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.360+0000 7f58ca7fc640 1 -- 172.21.15.42:0/1076612256 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f58c40bbd90 con 0x7f58cc1000a0 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f58b40a1b20 msgr2=0x7f58b40a3fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f58b40a1b20 0x7f58b40a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 msgr2=0x7f58cc069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc069390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f58c402c7d0 tx=0x7f58c4004040 comp rx=0 tx=0).stop 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 shutdown_connections 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58cc101c50 0x7f58cc19e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f58b40a1b20 0x7f58b40a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58cc101290 0x7f58cc19be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 --2- 172.21.15.42:0/1076612256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc1000a0 0x7f58cc069390 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 >> 172.21.15.42:0/1076612256 conn(0x7f58cc0fb610 msgr2=0x7f58cc1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:04.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 shutdown_connections 2024-03-12T04:15:04.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.362+0000 7f58d3377640 1 -- 172.21.15.42:0/1076612256 wait complete. 2024-03-12T04:15:04.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-12T04:15:04.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-mon[36119]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-12T04:15:04.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3693835430' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-12T04:15:04.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:15:04.446 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: ignoring --setuser ceph since I am not root 2024-03-12T04:15:04.446 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: ignoring --setgroup ceph since I am not root 2024-03-12T04:15:04.446 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-mgr[37499]: -- 172.21.15.119:0/3150165809 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56342f6fe1a0 con 0x56342f6d4800 2024-03-12T04:15:04.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-mon[35571]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-12T04:15:04.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3693835430' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-12T04:15:04.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:15:04.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: ignoring --setuser ceph since I am not root 2024-03-12T04:15:04.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: ignoring --setgroup ceph since I am not root 2024-03-12T04:15:04.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-mgr[37427]: -- 172.21.15.136:0/3401776512 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e4708e1a0 con 0x556e47064800 2024-03-12T04:15:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 -- 172.21.15.42:0/3511636065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 msgr2=0x7f7a0c0ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 --2- 172.21.15.42:0/3511636065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c0ffe90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009900 tx=0x7f7a0002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 -- 172.21.15.42:0/3511636065 shutdown_connections 2024-03-12T04:15:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 --2- 172.21.15.42:0/3511636065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a0c100d70 0x7f7a0c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 --2- 172.21.15.42:0/3511636065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a0c1003d0 0x7f7a0c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 --2- 172.21.15.42:0/3511636065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c0ffe90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 -- 172.21.15.42:0/3511636065 >> 172.21.15.42:0/3511636065 conn(0x7f7a0c0fb610 msgr2=0x7f7a0c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 -- 172.21.15.42:0/3511636065 shutdown_connections 2024-03-12T04:15:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.479+0000 7f7a12664640 1 -- 172.21.15.42:0/3511636065 wait complete. 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 Processor -- start 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 -- start start 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a0c1003d0 0x7f7a0c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a0c100d70 0x7f7a0c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c1a0d30 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c1a0ea0 con 0x7f7a0c1003d0 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a12664640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c1a1060 con 0x7f7a0c100d70 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57714/0 (socket says 172.21.15.42:57714) 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 -- 172.21.15.42:0/603831748 learned_addr learned my addr 172.21.15.42:0/603831748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a10c01640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a0c100d70 0x7f7a0c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0b7fe640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a0c1003d0 0x7f7a0c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 -- 172.21.15.42:0/603831748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a0c100d70 msgr2=0x7f7a0c1a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a0c100d70 0x7f7a0c1a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 -- 172.21.15.42:0/603831748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a0c1003d0 msgr2=0x7f7a0c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a0c1003d0 0x7f7a0c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0bfff640 1 -- 172.21.15.42:0/603831748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00009560 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.480+0000 7f7a0b7fe640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a0c1003d0 0x7f7a0c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.481+0000 7f7a0bfff640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c199240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7a00036040 tx=0x7f7a00004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.481+0000 7f7a097fa640 1 -- 172.21.15.42:0/603831748 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00003800 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.481+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c1a12e0 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.481+0000 7f7a097fa640 1 -- 172.21.15.42:0/603831748 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a0002de90 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.481+0000 7f7a097fa640 1 -- 172.21.15.42:0/603831748 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00005600 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.481+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c1a17f0 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.482+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c10a7c0 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.484+0000 7f7a097fa640 1 -- 172.21.15.42:0/603831748 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 136296+0+0 (secure 0 0 0) 0x7f7a00004dc0 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.484+0000 7f7a097fa640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79f40a1b20 0x7f79f40a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.484+0000 7f7a0b7fe640 1 -- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79f40a1b20 msgr2=0x7f79f40a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.484+0000 7f7a097fa640 1 -- 172.21.15.42:0/603831748 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7a000eff80 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.484+0000 7f7a0b7fe640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79f40a1b20 0x7f79f40a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:15:04.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.487+0000 7f7a097fa640 1 -- 172.21.15.42:0/603831748 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a000b9380 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.672+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f7a0c02ceb0 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.676+0000 7f7a097fa640 1 -- 172.21.15.42:0/603831748 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7a000430c0 con 0x7f7a0c0ffa70 2024-03-12T04:15:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.678+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79f40a1b20 msgr2=0x7f79f40a3fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:15:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.678+0000 7f7a12664640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79f40a1b20 0x7f79f40a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 msgr2=0x7f7a0c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c199240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7a00036040 tx=0x7f7a00004370 comp rx=0 tx=0).stop 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 shutdown_connections 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a0c100d70 0x7f7a0c1a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f79f40a1b20 0x7f79f40a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a0c1003d0 0x7f7a0c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 --2- 172.21.15.42:0/603831748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a0c0ffa70 0x7f7a0c199240 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 >> 172.21.15.42:0/603831748 conn(0x7f7a0c0fb610 msgr2=0x7f7a0c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 shutdown_connections 2024-03-12T04:15:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.679+0000 7f7a12664640 1 -- 172.21.15.42:0/603831748 wait complete. 2024-03-12T04:15:04.692 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 -- 172.21.15.42:0/1932191160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39840ffab0 msgr2=0x7f39840ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 --2- 172.21.15.42:0/1932191160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39840ffab0 0x7f39840ffed0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3978009900 tx=0x7f397802bcd0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 -- 172.21.15.42:0/1932191160 shutdown_connections 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 --2- 172.21.15.42:0/1932191160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3984100db0 0x7f3984101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 --2- 172.21.15.42:0/1932191160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3984100410 0x7f3984100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 --2- 172.21.15.42:0/1932191160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39840ffab0 0x7f39840ffed0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 -- 172.21.15.42:0/1932191160 >> 172.21.15.42:0/1932191160 conn(0x7f39840fb650 msgr2=0x7f39840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 -- 172.21.15.42:0/1932191160 shutdown_connections 2024-03-12T04:15:04.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.795+0000 7f398aaec640 1 -- 172.21.15.42:0/1932191160 wait complete. 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 Processor -- start 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 -- start start 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39840ffab0 0x7f3984199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3984100410 0x7f39841998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3984100db0 0x7f39841a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39841a0e30 con 0x7f3984100410 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39841a0fa0 con 0x7f39840ffab0 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f398aaec640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39841a1160 con 0x7f3984100db0 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f3983fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3984100410 0x7f39841998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f3983fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3984100410 0x7f39841998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57732/0 (socket says 172.21.15.42:57732) 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f3983fff640 1 -- 172.21.15.42:0/4150328713 learned_addr learned my addr 172.21.15.42:0/4150328713 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39840ffab0 0x7f3984199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f3989089640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3984100db0 0x7f39841a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f3983fff640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3984100db0 msgr2=0x7f39841a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.796+0000 7f3983fff640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3984100db0 0x7f39841a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f3983fff640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39840ffab0 msgr2=0x7f3984199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f3983fff640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39840ffab0 0x7f3984199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f3983fff640 1 -- 172.21.15.42:0/4150328713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3978009560 con 0x7f3984100410 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f3983fff640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3984100410 0x7f39841998d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f397000d770 tx=0x7f397000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970009c30 con 0x7f3984100410 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f397000de80 con 0x7f3984100410 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39841a1440 con 0x7f3984100410 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970004e00 con 0x7f3984100410 2024-03-12T04:15:04.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.797+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39841a1900 con 0x7f3984100410 2024-03-12T04:15:04.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.798+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398410a800 con 0x7f3984100410 2024-03-12T04:15:04.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.799+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 136296+0+0 (secure 0 0 0) 0x7f3970004f60 con 0x7f3984100410 2024-03-12T04:15:04.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.799+0000 7f3981ffb640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:04.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.800+0000 7f3988888640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 msgr2=0x7f39740a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:04.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.800+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f39700cce10 con 0x7f3984100410 2024-03-12T04:15:04.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.800+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T04:15:04.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:04.804+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3970096100 con 0x7f3984100410 2024-03-12T04:15:04.945 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:04.600+0000 7f61715f0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:15:04.945 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:04.694+0000 7f61715f0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:15:04.957 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:04.595+0000 7f3c6069c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:15:04.957 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:04.689+0000 7f3c6069c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:15:05.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:05.001+0000 7f3988888640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 msgr2=0x7f39740a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:05.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:05.001+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T04:15:05.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:05.007+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 --> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f398402ce10 con 0x7f39740a1b20 2024-03-12T04:15:05.031 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:04.633+0000 7ff7eb029200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T04:15:05.032 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:04.734+0000 7ff7eb029200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T04:15:05.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3693835430' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-12T04:15:05.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-mon[32260]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-12T04:15:05.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1076612256' entity='client.admin' 2024-03-12T04:15:05.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/603831748' entity='client.admin' 2024-03-12T04:15:05.329 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3693835430' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-12T04:15:05.329 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-mon[35571]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-12T04:15:05.329 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1076612256' entity='client.admin' 2024-03-12T04:15:05.329 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/603831748' entity='client.admin' 2024-03-12T04:15:05.334 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3693835430' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-12T04:15:05.335 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-mon[36119]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-12T04:15:05.335 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1076612256' entity='client.admin' 2024-03-12T04:15:05.335 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/603831748' entity='client.admin' 2024-03-12T04:15:05.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:05.402+0000 7f3988888640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 msgr2=0x7f39740a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:05.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:05.402+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-12T04:15:05.659 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:05.327+0000 7f3c6069c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:15:05.659 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:05.419+0000 7f3c6069c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:15:05.659 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:05.561+0000 7f3c6069c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:15:05.659 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:05.657+0000 7f3c6069c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:15:05.663 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:05.332+0000 7f61715f0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:15:05.663 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:05.426+0000 7f61715f0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:15:05.663 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:05.567+0000 7f61715f0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:15:05.663 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:05.660+0000 7f61715f0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:15:05.742 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:05.401+0000 7ff7eb029200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T04:15:05.743 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:05.497+0000 7ff7eb029200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T04:15:05.743 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:05.641+0000 7ff7eb029200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T04:15:05.743 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:05.739+0000 7ff7eb029200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T04:15:05.945 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:05.742+0000 7f61715f0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:15:05.957 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:05.739+0000 7f3c6069c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:15:06.065 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:05.825+0000 7ff7eb029200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T04:15:06.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:06.203+0000 7f3988888640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 msgr2=0x7f39740a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:06.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:06.203+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-12T04:15:06.295 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:06.003+0000 7f3c6069c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:15:06.295 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:06.112+0000 7f3c6069c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:15:06.295 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:06.198+0000 7f3c6069c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:15:06.295 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:06.293+0000 7f3c6069c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:15:06.298 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:06 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:06.005+0000 7f61715f0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:15:06.298 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:06 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:06.115+0000 7f61715f0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:15:06.298 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:06 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:06.200+0000 7f61715f0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:15:06.298 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:06 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:06.295+0000 7f61715f0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:15:06.397 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:06.095+0000 7ff7eb029200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T04:15:06.398 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:06.208+0000 7ff7eb029200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T04:15:06.398 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:06.297+0000 7ff7eb029200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T04:15:06.695 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:06 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:06.471+0000 7f61715f0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:15:06.707 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:06.468+0000 7f3c6069c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:15:06.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:06.395+0000 7ff7eb029200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T04:15:06.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:06.575+0000 7ff7eb029200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T04:15:07.695 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:07.423+0000 7f61715f0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:15:07.695 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:07.553+0000 7f61715f0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:15:07.707 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:07.423+0000 7f3c6069c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:15:07.707 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:07.552+0000 7f3c6069c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:15:07.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:07.804+0000 7f3988888640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 msgr2=0x7f39740a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:07.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:07.804+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-12T04:15:08.044 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:07.565+0000 7ff7eb029200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T04:15:08.044 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:07.701+0000 7ff7eb029200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T04:15:08.165 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:07.881+0000 7f61715f0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:15:08.166 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:07.968+0000 7f61715f0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:15:08.166 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:08.163+0000 7f61715f0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:15:08.173 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:07.887+0000 7f3c6069c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:15:08.173 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:07.976+0000 7f3c6069c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:15:08.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:08.042+0000 7ff7eb029200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T04:15:08.316 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:08.136+0000 7ff7eb029200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T04:15:08.445 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:08.347+0000 7f61715f0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:15:08.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:08.171+0000 7f3c6069c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:15:08.457 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:08.356+0000 7f3c6069c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:15:08.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:08.339+0000 7ff7eb029200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:15:08.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:08.533+0000 7ff7eb029200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T04:15:08.890 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:08.629+0000 7f61715f0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:15:08.903 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:08.638+0000 7f3c6069c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:15:09.097 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:08.829+0000 7ff7eb029200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T04:15:09.195 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:08.888+0000 7f61715f0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:15:09.195 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:09.125+0000 7f61715f0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:15:09.207 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:08.901+0000 7f3c6069c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:15:09.207 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:09 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:09.137+0000 7f3c6069c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:15:09.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:09.094+0000 7ff7eb029200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T04:15:09.566 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:09.340+0000 7ff7eb029200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T04:15:10.195 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:09.917+0000 7f61715f0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:15:10.207 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:09 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:09.935+0000 7f3c6069c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:15:10.467 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:10.179+0000 7ff7eb029200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T04:15:10.803 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:10.486+0000 7f61715f0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:15:10.804 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:10.584+0000 7f61715f0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:15:10.804 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y[37495]: 2024-03-12T04:15:10.801+0000 7f61715f0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:15:10.823 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:10.506+0000 7f3c6069c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:15:10.823 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:10.603+0000 7f3c6069c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:15:10.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:10.876+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 20) v1 ==== 136296+0+0 (secure 0 0 0) 0x7f39700964d0 con 0x7f3984100410 2024-03-12T04:15:11.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:11.006+0000 7f3988888640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 msgr2=0x7f39740a3fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2757555113 2024-03-12T04:15:11.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:11.006+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-12T04:15:11.066 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:10.777+0000 7ff7eb029200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T04:15:11.066 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:10.895+0000 7ff7eb029200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T04:15:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: Standby manager daemon y restarted 2024-03-12T04:15:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: Standby manager daemon y started 2024-03-12T04:15:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: Standby manager daemon z restarted 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: Standby manager daemon z started 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T04:15:11.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:10 smithi042 ceph-mon[32260]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:15:11.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:11.155+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 21) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f39700ca990 con 0x7f3984100410 2024-03-12T04:15:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:11.155+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 msgr2=0x7f39740a3fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:15:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:11.155+0000 7f3981ffb640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2757555113,v1:172.21.15.42:6801/2757555113] conn(0x7f39740a1b20 0x7f39740a3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: Standby manager daemon y restarted 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: Standby manager daemon y started 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: Standby manager daemon z restarted 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: Standby manager daemon z started 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T04:15:11.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:10 smithi119 ceph-mon[36119]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:15:11.207 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z[37423]: 2024-03-12T04:15:10.821+0000 7f3c6069c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:15:11.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: Standby manager daemon y restarted 2024-03-12T04:15:11.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: Standby manager daemon y started 2024-03-12T04:15:11.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.119:0/1740334784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: Standby manager daemon z restarted 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: Standby manager daemon z started 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T04:15:11.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:10 smithi136 ceph-mon[35571]: from='mgr.? 172.21.15.136:0/1876998787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T04:15:11.412 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.130+0000 7ff7eb029200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T04:15:11.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.409+0000 7ff74dadb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.409+0000 7ff74dadb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.409+0000 7ff74dadb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.409+0000 7ff74dadb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.817 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.409+0000 7ff74dadb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.817 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.411+0000 7ff74a2d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.817 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.411+0000 7ff74a2d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.817 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.411+0000 7ff74a2d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.817 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.411+0000 7ff74a2d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:11.817 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:15:11.411+0000 7ff74a2d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:15:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.157+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mon.0 v2:172.21.15.42:3300/0 9 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f3970009d90 con 0x7f3984100410 2024-03-12T04:15:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.158+0000 7f3981ffb640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39740341b0 0x7f3974034590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.158+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 --> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f398402ce10 con 0x7f39740341b0 2024-03-12T04:15:12.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.161+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39740341b0 0x7f3974034590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.163+0000 7f3988888640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39740341b0 0x7f3974034590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3978009230 tx=0x7f3978005fd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: Active manager daemon x restarted 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: Activating manager daemon x 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: osdmap e90: 12 total, 12 up, 12 in 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: mgrmap e21: x(active, starting, since 0.012078s), standbys: z, y 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:15:12.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:15:12.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: Manager daemon x is now available 2024-03-12T04:15:12.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:12.199 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:12.199 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T04:15:12.199 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:11 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T04:15:12.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-12T04:15:12.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: Active manager daemon x restarted 2024-03-12T04:15:12.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: Activating manager daemon x 2024-03-12T04:15:12.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: osdmap e90: 12 total, 12 up, 12 in 2024-03-12T04:15:12.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: mgrmap e21: x(active, starting, since 0.012078s), standbys: z, y 2024-03-12T04:15:12.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:15:12.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:15:12.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:15:12.210 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: Manager daemon x is now available 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T04:15:12.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:11 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T04:15:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.234+0000 7f3981ffb640 1 -- 172.21.15.42:0/4150328713 <== mgr.15522 v2:172.21.15.42:6800/2974863619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f398402ce10 con 0x7f39740341b0 2024-03-12T04:15:12.236 INFO:teuthology.orchestra.run.smithi042.stdout:Schedule set for path / 2024-03-12T04:15:12.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39740341b0 msgr2=0x7f3974034590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39740341b0 0x7f3974034590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3978009230 tx=0x7f3978005fd0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3984100410 msgr2=0x7f39841998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 --2- 172.21.15.42:0/4150328713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3984100410 0x7f39841998d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f397000d770 tx=0x7f397000dc30 comp rx=0 tx=0).stop 2024-03-12T04:15:12.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f3989089640 1 -- 172.21.15.42:0/4150328713 reap_dead start 2024-03-12T04:15:12.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 shutdown_connections 2024-03-12T04:15:12.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 >> 172.21.15.42:0/4150328713 conn(0x7f39840fb650 msgr2=0x7f39840fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:12.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 shutdown_connections 2024-03-12T04:15:12.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.236+0000 7f398aaec640 1 -- 172.21.15.42:0/4150328713 wait complete. 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: Active manager daemon x restarted 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: Activating manager daemon x 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: osdmap e90: 12 total, 12 up, 12 in 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: mgrmap e21: x(active, starting, since 0.012078s), standbys: z, y 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T04:15:12.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:15:12.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T04:15:12.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: Manager daemon x is now available 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T04:15:12.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:11 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T04:15:12.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- 172.21.15.42:0/1109476025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101c50 msgr2=0x7fa9881084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/1109476025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101c50 0x7fa9881084e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa978009900 tx=0x7fa97802bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- 172.21.15.42:0/1109476025 shutdown_connections 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/1109476025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101c50 0x7fa9881084e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/1109476025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa988101290 0x7fa988101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/1109476025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9881000a0 0x7fa9881004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- 172.21.15.42:0/1109476025 >> 172.21.15.42:0/1109476025 conn(0x7fa9880fb610 msgr2=0x7fa9880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- 172.21.15.42:0/1109476025 shutdown_connections 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- 172.21.15.42:0/1109476025 wait complete. 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 Processor -- start 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- start start 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9881000a0 0x7fa988197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101290 0x7fa9881975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa988101c50 0x7fa98819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98819eb80 con 0x7fa988101290 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98819ecf0 con 0x7fa988101c50 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.363+0000 7fa98ef6c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98819eeb0 con 0x7fa9881000a0 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101290 0x7fa9881975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101290 0x7fa9881975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55770/0 (socket says 172.21.15.42:55770) 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 -- 172.21.15.42:0/152415030 learned_addr learned my addr 172.21.15.42:0/152415030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:12.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa98d509640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa988101c50 0x7fa98819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa98cd08640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9881000a0 0x7fa988197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 -- 172.21.15.42:0/152415030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9881000a0 msgr2=0x7fa988197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9881000a0 0x7fa988197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 -- 172.21.15.42:0/152415030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa988101c50 msgr2=0x7fa98819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa988101c50 0x7fa98819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 -- 172.21.15.42:0/152415030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa978009560 con 0x7fa988101290 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa987fff640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101290 0x7fa9881975d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa97000e9f0 tx=0x7fa97000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa985ffb640 1 -- 172.21.15.42:0/152415030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970009700 con 0x7fa988101290 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98819f190 con 0x7fa988101290 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa985ffb640 1 -- 172.21.15.42:0/152415030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa97000cd80 con 0x7fa988101290 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa985ffb640 1 -- 172.21.15.42:0/152415030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970018840 con 0x7fa988101290 2024-03-12T04:15:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.364+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98819f6a0 con 0x7fa988101290 2024-03-12T04:15:12.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.365+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa988107410 con 0x7fa988101290 2024-03-12T04:15:12.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.366+0000 7fa985ffb640 1 -- 172.21.15.42:0/152415030 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7fa9700189a0 con 0x7fa988101290 2024-03-12T04:15:12.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.367+0000 7fa985ffb640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa9740a18e0 0x7fa9740a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.367+0000 7fa98cd08640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa9740a18e0 0x7fa9740a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.367+0000 7fa985ffb640 1 -- 172.21.15.42:0/152415030 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa970014070 con 0x7fa988101290 2024-03-12T04:15:12.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.367+0000 7fa98cd08640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa9740a18e0 0x7fa9740a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9881010f0 tx=0x7fa980009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:12.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.371+0000 7fa985ffb640 1 -- 172.21.15.42:0/152415030 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa970052d20 con 0x7fa988101290 2024-03-12T04:15:12.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.581+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 --> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fa98802ceb0 con 0x7fa9740a18e0 2024-03-12T04:15:12.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.644+0000 7fa985ffb640 1 -- 172.21.15.42:0/152415030 <== mgr.15522 v2:172.21.15.42:6800/2974863619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fa98802ceb0 con 0x7fa9740a18e0 2024-03-12T04:15:12.647 INFO:teuthology.orchestra.run.smithi042.stdout:Retention added to path / 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa9740a18e0 msgr2=0x7fa9740a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa9740a18e0 0x7fa9740a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9881010f0 tx=0x7fa980009500 comp rx=0 tx=0).stop 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101290 msgr2=0x7fa9881975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101290 0x7fa9881975d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa97000e9f0 tx=0x7fa97000eeb0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 shutdown_connections 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa9740a18e0 0x7fa9740a3da0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa988101c50 0x7fa98819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa988101290 0x7fa9881975d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 --2- 172.21.15.42:0/152415030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9881000a0 0x7fa988197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 >> 172.21.15.42:0/152415030 conn(0x7fa9880fb610 msgr2=0x7fa9881045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:12.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 shutdown_connections 2024-03-12T04:15:12.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.646+0000 7fa98ef6c640 1 -- 172.21.15.42:0/152415030 wait complete. 2024-03-12T04:15:12.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-12T04:15:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.762+0000 7f0eb5348640 1 -- 172.21.15.42:0/2464530663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb0102e20 msgr2=0x7f0eb01096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.762+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2464530663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb0102e20 0x7f0eb01096b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009ca0 tx=0x7f0ea802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.762+0000 7f0eb5348640 1 -- 172.21.15.42:0/2464530663 shutdown_connections 2024-03-12T04:15:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.762+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2464530663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb0102e20 0x7f0eb01096b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.762+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2464530663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb0102460 0x7f0eb01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.762+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2464530663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0eb00ff490 0x7f0eb00ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.762+0000 7f0eb5348640 1 -- 172.21.15.42:0/2464530663 >> 172.21.15.42:0/2464530663 conn(0x7f0eb00fada0 msgr2=0x7f0eb00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 -- 172.21.15.42:0/2464530663 shutdown_connections 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 -- 172.21.15.42:0/2464530663 wait complete. 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 Processor -- start 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 -- start start 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb00ff490 0x7f0eb0101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0eb0102460 0x7f0eb0100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb0102e20 0x7f0eb0100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb01010b0 con 0x7f0eb00ff490 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0101220 con 0x7f0eb0102e20 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eb5348640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb01a10b0 con 0x7f0eb0102460 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eaeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb00ff490 0x7f0eb0101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eaeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb00ff490 0x7f0eb0101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55792/0 (socket says 172.21.15.42:55792) 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.763+0000 7f0eaeffd640 1 -- 172.21.15.42:0/2881008284 learned_addr learned my addr 172.21.15.42:0/2881008284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaf7fe640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb0102e20 0x7f0eb0100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaeffd640 1 -- 172.21.15.42:0/2881008284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0eb0102460 msgr2=0x7f0eb0100410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eae7fc640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0eb0102460 0x7f0eb0100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaeffd640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0eb0102460 0x7f0eb0100410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaeffd640 1 -- 172.21.15.42:0/2881008284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb0102e20 msgr2=0x7f0eb0100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaeffd640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb0102e20 0x7f0eb0100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaeffd640 1 -- 172.21.15.42:0/2881008284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8009900 con 0x7f0eb00ff490 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaf7fe640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb0102e20 0x7f0eb0100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eae7fc640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0eb0102460 0x7f0eb0100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eaeffd640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb00ff490 0x7f0eb0101dd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000d980 tx=0x7f0ea000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0e97fff640 1 -- 172.21.15.42:0/2881008284 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0009c30 con 0x7f0eb00ff490 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0e97fff640 1 -- 172.21.15.42:0/2881008284 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0ea0009d90 con 0x7f0eb00ff490 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb01a1360 con 0x7f0eb00ff490 2024-03-12T04:15:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0e97fff640 1 -- 172.21.15.42:0/2881008284 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000f620 con 0x7f0eb00ff490 2024-03-12T04:15:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.764+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb01a1820 con 0x7f0eb00ff490 2024-03-12T04:15:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.765+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb0108530 con 0x7f0eb00ff490 2024-03-12T04:15:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.766+0000 7f0e97fff640 1 -- 172.21.15.42:0/2881008284 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f0ea0010460 con 0x7f0eb00ff490 2024-03-12T04:15:12.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.767+0000 7f0e97fff640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e980a18e0 0x7f0e980a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:12.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.767+0000 7f0eae7fc640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e980a18e0 0x7f0e980a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:12.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.767+0000 7f0e97fff640 1 -- 172.21.15.42:0/2881008284 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0ea00ccf00 con 0x7f0eb00ff490 2024-03-12T04:15:12.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.767+0000 7f0eae7fc640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e980a18e0 0x7f0e980a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0101b40 tx=0x7f0ea4005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:12.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.771+0000 7f0e97fff640 1 -- 172.21.15.42:0/2881008284 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ea0096330 con 0x7f0eb00ff490 2024-03-12T04:15:12.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.978+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 --> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb002ceb0 con 0x7f0e980a18e0 2024-03-12T04:15:12.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.987+0000 7f0e97fff640 1 -- 172.21.15.42:0/2881008284 <== mgr.15522 v2:172.21.15.42:6800/2974863619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f0eb002ceb0 con 0x7f0e980a18e0 2024-03-12T04:15:12.990 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-12T00:00:00", "created": "2024-03-12T04:15:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-12T04:15:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e980a18e0 msgr2=0x7f0e980a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e980a18e0 0x7f0e980a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0101b40 tx=0x7f0ea4005f90 comp rx=0 tx=0).stop 2024-03-12T04:15:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb00ff490 msgr2=0x7f0eb0101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb00ff490 0x7f0eb0101dd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000d980 tx=0x7f0ea000de40 comp rx=0 tx=0).stop 2024-03-12T04:15:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 shutdown_connections 2024-03-12T04:15:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e980a18e0 0x7f0e980a3da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb0102e20 0x7f0eb0100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0eb0102460 0x7f0eb0100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 --2- 172.21.15.42:0/2881008284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb00ff490 0x7f0eb0101dd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 >> 172.21.15.42:0/2881008284 conn(0x7f0eb00fada0 msgr2=0x7f0eb0105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.989+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 shutdown_connections 2024-03-12T04:15:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:12.990+0000 7f0eb5348640 1 -- 172.21.15.42:0/2881008284 wait complete. 2024-03-12T04:15:13.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/4076713925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c08106ed0 msgr2=0x7f0c08109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/4076713925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c08106ed0 0x7f0c08109320 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009ca0 tx=0x7f0c0002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/4076713925 shutdown_connections 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/4076713925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c08106ed0 0x7f0c08109320 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/4076713925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c081045a0 0x7f0c08106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/4076713925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08101c70 0x7f0c08104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/4076713925 >> 172.21.15.42:0/4076713925 conn(0x7f0c080fb660 msgr2=0x7f0c080fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/4076713925 shutdown_connections 2024-03-12T04:15:13.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/4076713925 wait complete. 2024-03-12T04:15:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 Processor -- start 2024-03-12T04:15:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.114+0000 7f0c0ca4d640 1 -- start start 2024-03-12T04:15:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0ca4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c08101c70 0x7f0c08101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0ca4d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c081045a0 0x7f0c080ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0ca4d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08106ed0 0x7f0c08100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0ca4d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0819aef0 con 0x7f0c081045a0 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0ca4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c081006c0 con 0x7f0c08101c70 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0ca4d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c08100850 con 0x7f0c08106ed0 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c08101c70 0x7f0c08101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c05d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c081045a0 0x7f0c080ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c08101c70 0x7f0c08101600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:43112/0 (socket says 172.21.15.42:43112) 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 -- 172.21.15.42:0/2712210850 learned_addr learned my addr 172.21.15.42:0/2712210850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c06d9d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08106ed0 0x7f0c08100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 -- 172.21.15.42:0/2712210850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08106ed0 msgr2=0x7f0c08100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08106ed0 0x7f0c08100180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 -- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c081045a0 msgr2=0x7f0c080ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c081045a0 0x7f0c080ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c0659c640 1 -- 172.21.15.42:0/2712210850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00009900 con 0x7f0c08101c70 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.115+0000 7f0c06d9d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08106ed0 0x7f0c08100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.116+0000 7f0c0659c640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c08101c70 0x7f0c08101600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e9f0 tx=0x7f0bf800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.116+0000 7f0bf77fe640 1 -- 172.21.15.42:0/2712210850 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8009720 con 0x7f0c08101c70 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.116+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c081a58b0 con 0x7f0c08101c70 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.116+0000 7f0bf77fe640 1 -- 172.21.15.42:0/2712210850 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0bf800cd80 con 0x7f0c08101c70 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.116+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c081a5c70 con 0x7f0c08101c70 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.116+0000 7f0bf77fe640 1 -- 172.21.15.42:0/2712210850 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8010430 con 0x7f0c08101c70 2024-03-12T04:15:13.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.117+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0804eee0 con 0x7f0c08101c70 2024-03-12T04:15:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.118+0000 7f0bf77fe640 1 -- 172.21.15.42:0/2712210850 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f0bf8005020 con 0x7f0c08101c70 2024-03-12T04:15:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.119+0000 7f0bf77fe640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0bf00a18e0 0x7f0bf00a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.119+0000 7f0bf77fe640 1 -- 172.21.15.42:0/2712210850 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0bf8014070 con 0x7f0c08101c70 2024-03-12T04:15:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.119+0000 7f0c05d9b640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0bf00a18e0 0x7f0bf00a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.120+0000 7f0c05d9b640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0bf00a18e0 0x7f0bf00a3da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c08101370 tx=0x7f0bfc005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:13.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.124+0000 7f0bf77fe640 1 -- 172.21.15.42:0/2712210850 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0bf8096910 con 0x7f0c08101c70 2024-03-12T04:15:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:13 smithi042 ceph-mon[32260]: mgrmap e22: x(active, since 1.02546s), standbys: z, y 2024-03-12T04:15:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:13 smithi042 ceph-mon[32260]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:13 smithi042 ceph-mon[32260]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:15:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:13 smithi042 ceph-mon[32260]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:13 smithi042 ceph-mon[32260]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.425+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 --> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0c0802ce10 con 0x7f0bf00a18e0 2024-03-12T04:15:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.432+0000 7f0bf77fe640 1 -- 172.21.15.42:0/2712210850 <== mgr.15522 v2:172.21.15.42:6800/2974863619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f0c0802ce10 con 0x7f0bf00a18e0 2024-03-12T04:15:13.434 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0bf00a18e0 msgr2=0x7f0bf00a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0bf00a18e0 0x7f0bf00a3da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c08101370 tx=0x7f0bfc005f90 comp rx=0 tx=0).stop 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c08101c70 msgr2=0x7f0c08101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c08101c70 0x7f0c08101600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e9f0 tx=0x7f0bf800edf0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 shutdown_connections 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0bf00a18e0 0x7f0bf00a3da0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08106ed0 0x7f0c08100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c081045a0 0x7f0c080ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 --2- 172.21.15.42:0/2712210850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c08101c70 0x7f0c08101600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 >> 172.21.15.42:0/2712210850 conn(0x7f0c080fb660 msgr2=0x7f0c081025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 shutdown_connections 2024-03-12T04:15:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:13.434+0000 7f0c0ca4d640 1 -- 172.21.15.42:0/2712210850 wait complete. 2024-03-12T04:15:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:13 smithi119 ceph-mon[36119]: mgrmap e22: x(active, since 1.02546s), standbys: z, y 2024-03-12T04:15:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:13 smithi119 ceph-mon[36119]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:13 smithi119 ceph-mon[36119]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:15:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:13 smithi119 ceph-mon[36119]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:13 smithi119 ceph-mon[36119]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.448 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-12T04:15:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:13 smithi136 ceph-mon[35571]: mgrmap e22: x(active, since 1.02546s), standbys: z, y 2024-03-12T04:15:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:13 smithi136 ceph-mon[35571]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:13 smithi136 ceph-mon[35571]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:15:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:13 smithi136 ceph-mon[35571]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:13 smithi136 ceph-mon[35571]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:15:13.475 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-12T04:15:13.488 INFO:teuthology.run_tasks:Running task workunit... 2024-03-12T04:15:13.500 INFO:tasks.workunit:Pulling workunits from ref b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-12T04:15:13.500 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-12T04:15:13.500 INFO:tasks.workunit:timeout=3h 2024-03-12T04:15:13.500 INFO:tasks.workunit:cleanup=True 2024-03-12T04:15:13.501 DEBUG:teuthology.orchestra.run.smithi042:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-12T04:15:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-12T04:15:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-12T04:15:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-03-12T04:15:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T04:15:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-12T04:15:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-03-12 04:12:59.753422030 +0000 2024-03-12T04:15:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-03-12 04:12:59.753422030 +0000 2024-03-12T04:15:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-03-12 04:14:55.846674821 +0000 2024-03-12T04:15:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-03-12T04:15:13.531 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-12T04:15:13.531 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-12T04:15:13.604 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-12T04:15:13.660 INFO:tasks.workunit.client.0.smithi042.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-12T04:15:14.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: [12/Mar/2024:04:15:13] ENGINE Bus STARTING 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: [12/Mar/2024:04:15:13] ENGINE Serving on http://172.21.15.42:8765 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: [12/Mar/2024:04:15:13] ENGINE Serving on https://172.21.15.42:7150 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: [12/Mar/2024:04:15:13] ENGINE Bus STARTED 2024-03-12T04:15:14.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:14 smithi119 ceph-mon[36119]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: [12/Mar/2024:04:15:13] ENGINE Bus STARTING 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: [12/Mar/2024:04:15:13] ENGINE Serving on http://172.21.15.42:8765 2024-03-12T04:15:14.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: [12/Mar/2024:04:15:13] ENGINE Serving on https://172.21.15.42:7150 2024-03-12T04:15:14.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: [12/Mar/2024:04:15:13] ENGINE Bus STARTED 2024-03-12T04:15:14.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:14 smithi136 ceph-mon[35571]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-12T04:15:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: [12/Mar/2024:04:15:13] ENGINE Bus STARTING 2024-03-12T04:15:14.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: [12/Mar/2024:04:15:13] ENGINE Serving on http://172.21.15.42:8765 2024-03-12T04:15:14.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: [12/Mar/2024:04:15:13] ENGINE Serving on https://172.21.15.42:7150 2024-03-12T04:15:14.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: [12/Mar/2024:04:15:13] ENGINE Bus STARTED 2024-03-12T04:15:14.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:14 smithi042 ceph-mon[32260]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-12T04:15:15.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Unable to set osd_memory_target on smithi136 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi042 to 177.2M 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Unable to set osd_memory_target on smithi042 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Updating smithi042:/etc/ceph/ceph.conf 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Updating smithi136:/etc/ceph/ceph.conf 2024-03-12T04:15:15.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.569 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.569 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:15 smithi042 ceph-mon[32260]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-03-12T04:15:15.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Unable to set osd_memory_target on smithi136 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi042 to 177.2M 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Unable to set osd_memory_target on smithi042 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:15:15.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:15:15.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Updating smithi042:/etc/ceph/ceph.conf 2024-03-12T04:15:15.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-12T04:15:15.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Updating smithi136:/etc/ceph/ceph.conf 2024-03-12T04:15:15.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:15 smithi119 ceph-mon[36119]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Unable to set osd_memory_target on smithi136 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi042 to 177.2M 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Unable to set osd_memory_target on smithi042 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Updating smithi042:/etc/ceph/ceph.conf 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Updating smithi136:/etc/ceph/ceph.conf 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:15.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:15 smithi136 ceph-mon[35571]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:15:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:15:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:15:16.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:16.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:16.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:16.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:15:16.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T04:15:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-12T04:15:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: Updating smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: Updating smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: Updating smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:15:17.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:15:18.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:18 smithi119 ceph-mon[36119]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-12T04:15:18.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:18 smithi136 ceph-mon[35571]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-12T04:15:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:18 smithi042 ceph-mon[32260]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-12T04:15:20.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:20 smithi119 ceph-mon[36119]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-12T04:15:20.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:20 smithi136 ceph-mon[35571]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-12T04:15:21.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:20 smithi042 ceph-mon[32260]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-12T04:15:22.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:22 smithi119 ceph-mon[36119]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-03-12T04:15:22.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:22 smithi136 ceph-mon[35571]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-03-12T04:15:23.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:22 smithi042 ceph-mon[32260]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-03-12T04:15:24.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:24 smithi119 ceph-mon[36119]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-12T04:15:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:24 smithi136 ceph-mon[35571]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-12T04:15:25.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:24 smithi042 ceph-mon[32260]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-12T04:15:26.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:26 smithi119 ceph-mon[36119]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-12T04:15:26.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:26 smithi136 ceph-mon[35571]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-12T04:15:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:26 smithi042 ceph-mon[32260]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-12T04:15:28.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:28 smithi119 ceph-mon[36119]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-03-12T04:15:28.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:28 smithi136 ceph-mon[35571]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-03-12T04:15:29.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:28 smithi042 ceph-mon[32260]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-03-12T04:15:30.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:30 smithi119 ceph-mon[36119]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-12T04:15:30.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:30 smithi136 ceph-mon[35571]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-12T04:15:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:30 smithi042 ceph-mon[32260]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-12T04:15:32.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:32 smithi119 ceph-mon[36119]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-12T04:15:32.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:32 smithi136 ceph-mon[35571]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-12T04:15:33.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:32 smithi042 ceph-mon[32260]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-12T04:15:33.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:15:33.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:15:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 -- 172.21.15.42:0/1803515506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090100270 msgr2=0x7fb090100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 --2- 172.21.15.42:0/1803515506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090100270 0x7fb090100650 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb080009930 tx=0x7fb08002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 -- 172.21.15.42:0/1803515506 shutdown_connections 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 --2- 172.21.15.42:0/1803515506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb090105340 0x7fb09010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 --2- 172.21.15.42:0/1803515506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090100c20 0x7fb090104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 --2- 172.21.15.42:0/1803515506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090100270 0x7fb090100650 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 -- 172.21.15.42:0/1803515506 >> 172.21.15.42:0/1803515506 conn(0x7fb090077230 msgr2=0x7fb090077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 -- 172.21.15.42:0/1803515506 shutdown_connections 2024-03-12T04:15:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 -- 172.21.15.42:0/1803515506 wait complete. 2024-03-12T04:15:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.255+0000 7fb094ec3640 1 Processor -- start 2024-03-12T04:15:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb094ec3640 1 -- start start 2024-03-12T04:15:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb094ec3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090100270 0x7fb090197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb094ec3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090105340 0x7fb090197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb094ec3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09019c000 0x7fb09019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb094ec3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09019c9f0 con 0x7fb090105340 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb094ec3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09019cb60 con 0x7fb09019c000 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb094ec3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09019ccd0 con 0x7fb090100270 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090105340 0x7fb090197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090105340 0x7fb090197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48998/0 (socket says 172.21.15.42:48998) 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 -- 172.21.15.42:0/3614665615 learned_addr learned my addr 172.21.15.42:0/3614665615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb08e59c640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090100270 0x7fb090197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09019c000 0x7fb09019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 -- 172.21.15.42:0/3614665615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090100270 msgr2=0x7fb090197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090100270 0x7fb090197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 -- 172.21.15.42:0/3614665615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09019c000 msgr2=0x7fb09019c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09019c000 0x7fb09019c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 -- 172.21.15.42:0/3614665615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb080009560 con 0x7fb090105340 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09019c000 0x7fb09019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb08e59c640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090100270 0x7fb090197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.256+0000 7fb087fff640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090105340 0x7fb090197c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e9f0 tx=0x7fb07800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.257+0000 7fb0877fe640 1 -- 172.21.15.42:0/3614665615 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780096c0 con 0x7fb090105340 2024-03-12T04:15:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.257+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0901a42b0 con 0x7fb090105340 2024-03-12T04:15:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.257+0000 7fb0877fe640 1 -- 172.21.15.42:0/3614665615 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0780054d0 con 0x7fb090105340 2024-03-12T04:15:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.257+0000 7fb0877fe640 1 -- 172.21.15.42:0/3614665615 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb078010430 con 0x7fb090105340 2024-03-12T04:15:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.257+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0901a4760 con 0x7fb090105340 2024-03-12T04:15:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.259+0000 7fb0877fe640 1 -- 172.21.15.42:0/3614665615 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7fb078005000 con 0x7fb090105340 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.259+0000 7fb0877fe640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb07c0a1c30 0x7fb07c0a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.259+0000 7fb097b73740 1 Processor -- start 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.259+0000 7fb08e59c640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb07c0a1c30 0x7fb07c0a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.259+0000 7fb0877fe640 1 -- 172.21.15.42:0/3614665615 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb078014070 con 0x7fb090105340 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb097b73740 1 -- start start 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb097b73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b539c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb097b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb097b73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb097b73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b530b280 con 0x55a7b53a57b0 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb097b73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b53087b0 con 0x55a7b539c2b0 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb097b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b5303f90 con 0x55a7b539cbd0 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08e59c640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb07c0a1c30 0x7fb07c0a40f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb080036040 tx=0x7fb080000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb087fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49006/0 (socket says 172.21.15.42:49006) 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2875884604 learned_addr learned my addr 172.21.15.42:0/2875884604 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08e59c640 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b539c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2875884604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 msgr2=0x55a7b53a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2875884604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 msgr2=0x55a7b539c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b539c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2875884604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7b52c2570 con 0x55a7b53a57b0 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.260+0000 7fb08e59c640 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b539c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.261+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a7ba0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb088009ea0 tx=0x7fb088009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b62be2071ea789 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.261+0000 7fb084ff9640 1 -- 172.21.15.42:0/2875884604 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088004d20 con 0x55a7b53a57b0 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.261+0000 7fb084ff9640 1 -- 172.21.15.42:0/2875884604 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb088004e80 con 0x55a7b53a57b0 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.261+0000 7fb084ff9640 1 -- 172.21.15.42:0/2875884604 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088011790 con 0x55a7b53a57b0 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.261+0000 7fb097b73740 1 -- 172.21.15.42:0/2875884604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 msgr2=0x55a7b53a7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.261+0000 7fb097b73740 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a7ba0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb088009ea0 tx=0x7fb088009f90 comp rx=0 tx=0).stop 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- 172.21.15.42:0/2875884604 shutdown_connections 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a7ba0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 --2- 172.21.15.42:0/2875884604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b539c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- 172.21.15.42:0/2875884604 >> 172.21.15.42:0/2875884604 conn(0x55a7b5396990 msgr2=0x55a7b5396d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- 172.21.15.42:0/2875884604 shutdown_connections 2024-03-12T04:15:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- 172.21.15.42:0/2875884604 wait complete. 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- start start 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 Processor -- start 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b4fd7430 con 0x55a7b53a57b0 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b4f93e00 con 0x55a7b539cbd0 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b5288fd0 con 0x55a7b539c2b0 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb097b73740 10 client.0 _ll_register_callbacks cb 0x55a7b52defd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49016/0 (socket says 172.21.15.42:49016) 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb087fff640 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.262+0000 7fb08ed9d640 1 -- 172.21.15.42:0/1576020390 learned_addr learned my addr 172.21.15.42:0/1576020390 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb08e59c640 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb08ed9d640 1 -- 172.21.15.42:0/1576020390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 msgr2=0x55a7b53a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb08ed9d640 1 -- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 msgr2=0x55a7b53a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb08ed9d640 1 -- 172.21.15.42:0/1576020390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb088009b00 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a2a90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb08800eb50 tx=0x7fb088005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2e3e3fed39829b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb075ffb640 1 -- 172.21.15.42:0/1576020390 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088021070 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb075ffb640 1 -- 172.21.15.42:0/1576020390 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb08800ee80 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7b5040720 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb075ffb640 1 -- 172.21.15.42:0/1576020390 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0880058f0 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb097b73740 10 client.15612 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7b5302610 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.263+0000 7fb075ffb640 1 -- 172.21.15.42:0/1576020390 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb088033020 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb075ffb640 1 -- 172.21.15.42:0/1576020390 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb08800c070 con 0x55a7b53a57b0 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb097b73740 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb097b73740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb097b73740 20 client.15612 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb097b73740 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b53b57e0 0x55a7b53b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb097b73740 4 client.15612 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7b508f890 con 0x55a7b53b57e0 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb08e59c640 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b53b57e0 0x55a7b53b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb075ffb640 10 client.15612 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.264+0000 7fb08e59c640 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b53b57e0 0x55a7b53b7c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7c9bb61db42aad server_cookie=a74f95fb96c3c602 in_seq=0 out_seq=0 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb075ffb640 1 -- 172.21.15.42:0/1576020390 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7b508f890 con 0x55a7b53b57e0 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb075ffb640 10 client.15612 handle_command_reply: tid=0 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 client.15612 shutdown 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 msgr2=0x55a7b53a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a2a90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb08800eb50 tx=0x7fb088005230 comp rx=0 tx=0).stop 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 shutdown_connections 2024-03-12T04:15:33.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b53b57e0 0x55a7b53b7c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a2a90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb075ffb640 0 client.15612 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 --2- 172.21.15.42:0/1576020390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 >> 172.21.15.42:0/1576020390 conn(0x55a7b5396990 msgr2=0x55a7b5398730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 shutdown_connections 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 1 -- 172.21.15.42:0/1576020390 wait complete. 2024-03-12T04:15:33.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.265+0000 7fb097b73740 20 client.15612 trim_cache size 0 max 16384 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 Processor -- start 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 -- start start 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b53b5590 con 0x55a7b53a57b0 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b53922b0 con 0x55a7b539cbd0 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb097b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b539bc80 con 0x55a7b539c2b0 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49032/0 (socket says 172.21.15.42:49032) 2024-03-12T04:15:33.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 -- 172.21.15.42:0/510697575 learned_addr learned my addr 172.21.15.42:0/510697575 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08e59c640 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 -- 172.21.15.42:0/510697575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 msgr2=0x55a7b53a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb087fff640 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 -- 172.21.15.42:0/510697575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 msgr2=0x55a7b53a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08ed9d640 1 -- 172.21.15.42:0/510697575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7b529eac0 con 0x55a7b53a57b0 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb08e59c640 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.281+0000 7fb087fff640 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53ac650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb08802f570 tx=0x7fb088009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d93b0c8802a1039 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb077fff640 1 -- 172.21.15.42:0/510697575 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088021070 con 0x55a7b53a57b0 2024-03-12T04:15:33.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb077fff640 1 -- 172.21.15.42:0/510697575 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb088033070 con 0x55a7b53a57b0 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb077fff640 1 -- 172.21.15.42:0/510697575 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08802f760 con 0x55a7b53a57b0 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 -- 172.21.15.42:0/510697575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 msgr2=0x55a7b53ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53ac650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb08802f570 tx=0x7fb088009580 comp rx=0 tx=0).stop 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 -- 172.21.15.42:0/510697575 shutdown_connections 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53ac650 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539cbd0 0x55a7b53a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 --2- 172.21.15.42:0/510697575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539c2b0 0x55a7b53a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 -- 172.21.15.42:0/510697575 >> 172.21.15.42:0/510697575 conn(0x55a7b5396990 msgr2=0x55a7b539d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 -- 172.21.15.42:0/510697575 shutdown_connections 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.282+0000 7fb097b73740 1 -- 172.21.15.42:0/510697575 wait complete. 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 -- start start 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 Processor -- start 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b53ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 0x55a7b53a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b53a96b0 con 0x55a7b53a57b0 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b53a9820 con 0x55a7b539c2b0 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7b53a99e0 con 0x55a7b539cbd0 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb097b73740 10 client.0 _ll_register_callbacks cb 0x7fb088009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49042/0 (socket says 172.21.15.42:49042) 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2032575724 learned_addr learned my addr 172.21.15.42:0/2032575724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2032575724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 msgr2=0x55a7b53a8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 0x55a7b53a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2032575724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 msgr2=0x55a7b53ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b53ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 -- 172.21.15.42:0/2032575724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7b5302610 con 0x55a7b53a57b0 2024-03-12T04:15:33.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb08ed9d640 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a9170 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb088005270 tx=0x7fb088009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b7baf477c7e086 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.283+0000 7fb075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb075ffb640 1 -- 172.21.15.42:0/2032575724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088021070 con 0x55a7b53a57b0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb075ffb640 1 -- 172.21.15.42:0/2032575724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb088034080 con 0x55a7b53a57b0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7b50037e0 con 0x55a7b53a57b0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb075ffb640 1 -- 172.21.15.42:0/2032575724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb08800ee80 con 0x55a7b53a57b0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 10 client.15618 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7b52c46a0 con 0x55a7b53a57b0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb075ffb640 1 -- 172.21.15.42:0/2032575724 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a7b50037e0 con 0x55a7b53a57b0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb075ffb640 1 -- 172.21.15.42:0/2032575724 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb08800c040 con 0x55a7b53a57b0 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 20 client.15618 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b540cc50 0x55a7b540f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 4 client.15618 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:15:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7b4ff24e0 con 0x55a7b540cc50 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.284+0000 7fb08e59c640 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b540cc50 0x55a7b540f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb075ffb640 10 client.15618 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb08e59c640 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b540cc50 0x55a7b540f080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96457302fa27500 server_cookie=c7ae943e0e55f13c in_seq=0 out_seq=0 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb075ffb640 1 -- 172.21.15.42:0/2032575724 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7b4ff24e0 con 0x55a7b540cc50 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb075ffb640 10 client.15618 handle_command_reply: tid=0 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 client.15618 shutdown 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 msgr2=0x55a7b53a9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a9170 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb088005270 tx=0x7fb088009a60 comp rx=0 tx=0).stop 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 shutdown_connections 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a7b540cc50 0x55a7b540f080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7b53a57b0 0x55a7b53a9170 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b539cbd0 0x55a7b53a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 --2- 172.21.15.42:0/2032575724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7b539c2b0 0x55a7b53ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb075ffb640 0 client.15618 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 >> 172.21.15.42:0/2032575724 conn(0x55a7b5396990 msgr2=0x55a7b53a0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 shutdown_connections 2024-03-12T04:15:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 1 -- 172.21.15.42:0/2032575724 wait complete. 2024-03-12T04:15:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.285+0000 7fb097b73740 20 client.15618 trim_cache size 0 max 16384 2024-03-12T04:15:33.294 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:15:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:15:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:15:33.294 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:15:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb07c0a1c30 msgr2=0x7fb07c0a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb07c0a1c30 0x7fb07c0a40f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb080036040 tx=0x7fb080000bc0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090105340 msgr2=0x7fb090197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090105340 0x7fb090197c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e9f0 tx=0x7fb07800eeb0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 shutdown_connections 2024-03-12T04:15:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb07c0a1c30 0x7fb07c0a40f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09019c000 0x7fb09019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb090105340 0x7fb090197c80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 --2- 172.21.15.42:0/3614665615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090100270 0x7fb090197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 >> 172.21.15.42:0/3614665615 conn(0x7fb090077230 msgr2=0x7fb090104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 shutdown_connections 2024-03-12T04:15:33.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.286+0000 7fb094ec3640 1 -- 172.21.15.42:0/3614665615 wait complete. 2024-03-12T04:15:33.298 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:15:33.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 -- 172.21.15.42:0/1650590851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 msgr2=0x7f9ac810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/1650590851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac810e020 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009cd0 tx=0x7f9ac002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 -- 172.21.15.42:0/1650590851 shutdown_connections 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/1650590851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac810e020 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/1650590851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac8101d30 0x7f9ac8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/1650590851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ac8101380 0x7f9ac8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 -- 172.21.15.42:0/1650590851 >> 172.21.15.42:0/1650590851 conn(0x7f9ac80fcc20 msgr2=0x7f9ac80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 -- 172.21.15.42:0/1650590851 shutdown_connections 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 -- 172.21.15.42:0/1650590851 wait complete. 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 Processor -- start 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.400+0000 7f9ace3d7640 1 -- start start 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ace3d7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac8101380 0x7f9ac8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ace3d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ace3d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ac819c010 0x7f9ac819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ace3d7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac819ca00 con 0x7f9ac8106450 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ace3d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac819cb70 con 0x7f9ac819c010 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ace3d7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac819cce0 con 0x7f9ac8101380 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac8197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49070/0 (socket says 172.21.15.42:49070) 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 -- 172.21.15.42:0/700788584 learned_addr learned my addr 172.21.15.42:0/700788584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9acc974640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ac819c010 0x7f9ac819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac7fff640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac8101380 0x7f9ac8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 -- 172.21.15.42:0/700788584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac8101380 msgr2=0x7f9ac8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac8101380 0x7f9ac8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 -- 172.21.15.42:0/700788584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ac819c010 msgr2=0x7f9ac819c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ac819c010 0x7f9ac819c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 -- 172.21.15.42:0/700788584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0009900 con 0x7f9ac8106450 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac7fff640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac8101380 0x7f9ac8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.401+0000 7f9ac77fe640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac8197c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800d740 tx=0x7f9ab800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.402+0000 7f9ac57fa640 1 -- 172.21.15.42:0/700788584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8009c30 con 0x7f9ac8106450 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.402+0000 7f9ac57fa640 1 -- 172.21.15.42:0/700788584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ab800de50 con 0x7f9ac8106450 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.402+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac81a42c0 con 0x7f9ac8106450 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.402+0000 7f9ac57fa640 1 -- 172.21.15.42:0/700788584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8004e40 con 0x7f9ac8106450 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.402+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac81a4770 con 0x7f9ac8106450 2024-03-12T04:15:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.403+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac80683c0 con 0x7f9ac8106450 2024-03-12T04:15:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.404+0000 7f9ac57fa640 1 -- 172.21.15.42:0/700788584 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f9ab8004fa0 con 0x7f9ac8106450 2024-03-12T04:15:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.404+0000 7f9ac57fa640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9ab40a1cb0 0x7f9ab40a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.404+0000 7f9ac7fff640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9ab40a1cb0 0x7f9ab40a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.404+0000 7f9ac57fa640 1 -- 172.21.15.42:0/700788584 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9ab80cd650 con 0x7f9ac8106450 2024-03-12T04:15:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.405+0000 7f9ac7fff640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9ab40a1cb0 0x7f9ab40a4170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00058c0 tx=0x7f9ab0005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.408+0000 7f9ac57fa640 1 -- 172.21.15.42:0/700788584 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ab8096480 con 0x7f9ac8106450 2024-03-12T04:15:33.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.629+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ac8101d30 con 0x7f9ac8106450 2024-03-12T04:15:33.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.630+0000 7f9ac57fa640 1 -- 172.21.15.42:0/700788584 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f9ab8017070 con 0x7f9ac8106450 2024-03-12T04:15:33.631 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:15:33.632 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:15:33.632 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:15:33.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9ab40a1cb0 msgr2=0x7f9ab40a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9ab40a1cb0 0x7f9ab40a4170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00058c0 tx=0x7f9ab0005850 comp rx=0 tx=0).stop 2024-03-12T04:15:33.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 msgr2=0x7f9ac8197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac8197c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800d740 tx=0x7f9ab800dc00 comp rx=0 tx=0).stop 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 shutdown_connections 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9ab40a1cb0 0x7f9ab40a4170 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ac819c010 0x7f9ac819c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ac8106450 0x7f9ac8197c90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 --2- 172.21.15.42:0/700788584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac8101380 0x7f9ac8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 >> 172.21.15.42:0/700788584 conn(0x7f9ac80fcc20 msgr2=0x7f9ac80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 shutdown_connections 2024-03-12T04:15:33.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.632+0000 7f9ace3d7640 1 -- 172.21.15.42:0/700788584 wait complete. 2024-03-12T04:15:33.644 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:15:33.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:15:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 -- 172.21.15.42:0/133895723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb224100c50 msgr2=0x7fb224104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 --2- 172.21.15.42:0/133895723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb224100c50 0x7fb224104c90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb210009930 tx=0x7fb21002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 -- 172.21.15.42:0/133895723 shutdown_connections 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 --2- 172.21.15.42:0/133895723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb224105370 0x7fb22410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 --2- 172.21.15.42:0/133895723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb224100c50 0x7fb224104c90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 --2- 172.21.15.42:0/133895723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2241002a0 0x7fb224100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 -- 172.21.15.42:0/133895723 >> 172.21.15.42:0/133895723 conn(0x7fb224076400 msgr2=0x7fb224076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 -- 172.21.15.42:0/133895723 shutdown_connections 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.747+0000 7fb22ab79640 1 -- 172.21.15.42:0/133895723 wait complete. 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 Processor -- start 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 -- start start 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2241002a0 0x7fb224196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb224105370 0x7fb224196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb22419aee0 0x7fb22419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22419b8d0 con 0x7fb22419aee0 2024-03-12T04:15:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22419ba40 con 0x7fb224105370 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb22ab79640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22419bbb0 con 0x7fb2241002a0 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb229116640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb22419aee0 0x7fb22419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb229116640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb22419aee0 0x7fb22419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49092/0 (socket says 172.21.15.42:49092) 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb229116640 1 -- 172.21.15.42:0/2972804542 learned_addr learned my addr 172.21.15.42:0/2972804542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb228915640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2241002a0 0x7fb224196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.748+0000 7fb223fff640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb224105370 0x7fb224196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb229116640 1 -- 172.21.15.42:0/2972804542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2241002a0 msgr2=0x7fb224196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb229116640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2241002a0 0x7fb224196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb229116640 1 -- 172.21.15.42:0/2972804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb224105370 msgr2=0x7fb224196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb229116640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb224105370 0x7fb224196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb229116640 1 -- 172.21.15.42:0/2972804542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210009560 con 0x7fb22419aee0 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb223fff640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb224105370 0x7fb224196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb228915640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2241002a0 0x7fb224196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb229116640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb22419aee0 0x7fb22419b390 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00eaa0 tx=0x7fb21c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb221ffb640 1 -- 172.21.15.42:0/2972804542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00cdb0 con 0x7fb22419aee0 2024-03-12T04:15:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb221ffb640 1 -- 172.21.15.42:0/2972804542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb21c005020 con 0x7fb22419aee0 2024-03-12T04:15:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb221ffb640 1 -- 172.21.15.42:0/2972804542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0116b0 con 0x7fb22419aee0 2024-03-12T04:15:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb224069770 con 0x7fb22419aee0 2024-03-12T04:15:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.749+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb224069ba0 con 0x7fb22419aee0 2024-03-12T04:15:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.751+0000 7fb221ffb640 1 -- 172.21.15.42:0/2972804542 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7fb21c005190 con 0x7fb22419aee0 2024-03-12T04:15:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 Processor -- start 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 -- start start 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb221ffb640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2140a1c30 0x7fb2140a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0c9d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0c58960 con 0x55bbe0c9db70 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0c0c310 con 0x55bbe0ca6750 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb22d820740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0c09840 con 0x55bbe0c9d250 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb223fff640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2140a1c30 0x7fb2140a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb221ffb640 1 -- 172.21.15.42:0/2972804542 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb21c0cdbc0 con 0x7fb22419aee0 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb229116640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb228915640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0c9d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb223fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb229116640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:59456/0 (socket says 172.21.15.42:59456) 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb228915640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0c9d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59408/0 (socket says 172.21.15.42:59408) 2024-03-12T04:15:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb229116640 1 -- 172.21.15.42:0/1705665811 learned_addr learned my addr 172.21.15.42:0/1705665811 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.752+0000 7fb229116640 1 -- 172.21.15.42:0/1705665811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 msgr2=0x55bbe0c9d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb229116640 1 --2- 172.21.15.42:0/1705665811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0c9d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb229116640 1 -- 172.21.15.42:0/1705665811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 msgr2=0x55bbe0ca6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb229116640 1 --2- 172.21.15.42:0/1705665811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb229116640 1 -- 172.21.15.42:0/1705665811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbe0bc7060 con 0x55bbe0ca6750 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb223fff640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2140a1c30 0x7fb2140a40f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb210009900 tx=0x7fb2100047d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb229116640 1 --2- 172.21.15.42:0/1705665811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca8b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00efd0 tx=0x7fb21c09a1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3572b91ea7e4d870 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb20effd640 1 -- 172.21.15.42:0/1705665811 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c09ad70 con 0x55bbe0ca6750 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb20effd640 1 -- 172.21.15.42:0/1705665811 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb21c09a670 con 0x55bbe0ca6750 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb20effd640 1 -- 172.21.15.42:0/1705665811 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c005520 con 0x55bbe0ca6750 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb22d820740 1 -- 172.21.15.42:0/1705665811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 msgr2=0x55bbe0ca8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.753+0000 7fb22d820740 1 --2- 172.21.15.42:0/1705665811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca8b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00efd0 tx=0x7fb21c09a1b0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- 172.21.15.42:0/1705665811 shutdown_connections 2024-03-12T04:15:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 --2- 172.21.15.42:0/1705665811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca8b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 --2- 172.21.15.42:0/1705665811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 --2- 172.21.15.42:0/1705665811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0c9d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- 172.21.15.42:0/1705665811 >> 172.21.15.42:0/1705665811 conn(0x55bbe0c97980 msgr2=0x55bbe0c97d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- 172.21.15.42:0/1705665811 shutdown_connections 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- 172.21.15.42:0/1705665811 wait complete. 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- start start 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 Processor -- start 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0c05020 con 0x55bbe0c9db70 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe08d8430 con 0x55bbe0ca6750 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0894e00 con 0x55bbe0c9d250 2024-03-12T04:15:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb22d820740 10 client.0 _ll_register_callbacks cb 0x55bbe0baccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.754+0000 7fb223fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49106/0 (socket says 172.21.15.42:49106) 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 -- 172.21.15.42:0/460458177 learned_addr learned my addr 172.21.15.42:0/460458177 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb229116640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb228915640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 -- 172.21.15.42:0/460458177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 msgr2=0x55bbe0ca3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 -- 172.21.15.42:0/460458177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 msgr2=0x55bbe0ca41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 -- 172.21.15.42:0/460458177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb224069ba0 con 0x55bbe0c9db70 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb229116640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb223fff640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca3c80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb210009930 tx=0x7fb210000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec34e2805734ff12 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb203fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb203fff640 1 -- 172.21.15.42:0/460458177 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210005d50 con 0x55bbe0c9db70 2024-03-12T04:15:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb203fff640 1 -- 172.21.15.42:0/460458177 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb210004dc0 con 0x55bbe0c9db70 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbe09047e0 con 0x55bbe0c9db70 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb203fff640 1 -- 172.21.15.42:0/460458177 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb210004f80 con 0x55bbe0c9db70 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb22d820740 10 client.15636 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.755+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bbe0c036a0 con 0x55bbe0c9db70 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb203fff640 1 -- 172.21.15.42:0/460458177 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb21005e020 con 0x55bbe0c9db70 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb203fff640 1 -- 172.21.15.42:0/460458177 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb21004e070 con 0x55bbe0c9db70 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb22d820740 10 client.15636 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb22d820740 10 client.15636 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb22d820740 20 client.15636 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb22d820740 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0cb6ba0 0x55bbe0cb8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb22d820740 4 client.15636 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbe0941720 con 0x55bbe0cb6ba0 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb228915640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0cb6ba0 0x55bbe0cb8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb228915640 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0cb6ba0 0x55bbe0cb8fc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4488e593dd4bf15 server_cookie=d7e1113192debabc in_seq=0 out_seq=0 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.756+0000 7fb203fff640 10 client.15636 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb203fff640 1 -- 172.21.15.42:0/460458177 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bbe0941720 con 0x55bbe0cb6ba0 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb203fff640 10 client.15636 handle_command_reply: tid=0 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 client.15636 shutdown 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 msgr2=0x55bbe0ca3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca3c80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb210009930 tx=0x7fb210000bc0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 shutdown_connections 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0ca6750 0x55bbe0ca41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0cb6ba0 0x55bbe0cb8fc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0c9db70 0x55bbe0ca3c80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb203fff640 0 client.15636 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 --2- 172.21.15.42:0/460458177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 >> 172.21.15.42:0/460458177 conn(0x55bbe0c97980 msgr2=0x55bbe0c996d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 shutdown_connections 2024-03-12T04:15:33.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 1 -- 172.21.15.42:0/460458177 wait complete. 2024-03-12T04:15:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.757+0000 7fb22d820740 20 client.15636 trim_cache size 0 max 16384 2024-03-12T04:15:33.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 Processor -- start 2024-03-12T04:15:33.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 -- start start 2024-03-12T04:15:33.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0c96a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0cad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0b8a130 con 0x55bbe0ca6750 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0ca1c90 con 0x55bbe0c9d250 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb22d820740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0c98300 con 0x55bbe0c9db70 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb229116640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0cad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb229116640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0cad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49110/0 (socket says 172.21.15.42:49110) 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb229116640 1 -- 172.21.15.42:0/2382050068 learned_addr learned my addr 172.21.15.42:0/2382050068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb228915640 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb223fff640 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0c96a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.769+0000 7fb229116640 1 -- 172.21.15.42:0/2382050068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 msgr2=0x55bbe0c96a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb229116640 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0c96a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb229116640 1 -- 172.21.15.42:0/2382050068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 msgr2=0x55bbe0ca3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb229116640 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb229116640 1 -- 172.21.15.42:0/2382050068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbe0b9fc20 con 0x55bbe0ca6750 2024-03-12T04:15:33.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb228915640 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb229116640 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0cad600 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb21c096ff0 tx=0x7fb21c09a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf385ba19d31342a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb20e7fc640 1 -- 172.21.15.42:0/2382050068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c09ad70 con 0x55bbe0ca6750 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb20e7fc640 1 -- 172.21.15.42:0/2382050068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb21c0a2e10 con 0x55bbe0ca6750 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb20e7fc640 1 -- 172.21.15.42:0/2382050068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0a2980 con 0x55bbe0ca6750 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 -- 172.21.15.42:0/2382050068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 msgr2=0x55bbe0cad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0cad600 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb21c096ff0 tx=0x7fb21c09a2e0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 -- 172.21.15.42:0/2382050068 shutdown_connections 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0cad600 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0c96a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 --2- 172.21.15.42:0/2382050068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0ca3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 -- 172.21.15.42:0/2382050068 >> 172.21.15.42:0/2382050068 conn(0x55bbe0c97980 msgr2=0x55bbe0c962f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 -- 172.21.15.42:0/2382050068 shutdown_connections 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.770+0000 7fb22d820740 1 -- 172.21.15.42:0/2382050068 wait complete. 2024-03-12T04:15:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 -- start start 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 Processor -- start 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0cacc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0cad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0d0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0d101c0 con 0x55bbe0ca6750 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0d10330 con 0x55bbe0c9d250 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe0d104f0 con 0x55bbe0c9db70 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb22d820740 10 client.0 _ll_register_callbacks cb 0x7fb21c0cb0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb229116640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0d0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb229116640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0d0fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49112/0 (socket says 172.21.15.42:49112) 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb229116640 1 -- 172.21.15.42:0/1823266969 learned_addr learned my addr 172.21.15.42:0/1823266969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb228915640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0cacc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.771+0000 7fb223fff640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0cad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb229116640 1 -- 172.21.15.42:0/1823266969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 msgr2=0x55bbe0cad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb229116640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0cad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb229116640 1 -- 172.21.15.42:0/1823266969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 msgr2=0x55bbe0cacc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb229116640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0cacc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb229116640 1 -- 172.21.15.42:0/1823266969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbe0bc7060 con 0x55bbe0ca6750 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb228915640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0cacc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb229116640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0d0fc80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0ba010 tx=0x7fb21c0ba0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d58730bbea9719 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb203fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb203fff640 1 -- 172.21.15.42:0/1823266969 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0b3170 con 0x55bbe0ca6750 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb203fff640 1 -- 172.21.15.42:0/1823266969 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb21c09ad70 con 0x55bbe0ca6750 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb21c09a670 con 0x55bbe0ca6750 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb203fff640 1 -- 172.21.15.42:0/1823266969 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb21c098970 con 0x55bbe0ca6750 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 10 client.15642 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bbe0bc58c0 con 0x55bbe0ca6750 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb203fff640 1 -- 172.21.15.42:0/1823266969 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb21c09a670 con 0x55bbe0ca6750 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb203fff640 1 -- 172.21.15.42:0/1823266969 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb21c018030 con 0x55bbe0ca6750 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 10 client.15642 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 10 client.15642 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 20 client.15642 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0d178b0 0x55bbe0d19ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 4 client.15642 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.772+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bbe0990890 con 0x55bbe0d178b0 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb228915640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0d178b0 0x55bbe0d19ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb228915640 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0d178b0 0x55bbe0d19ca0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7cf75c50b3f47cd server_cookie=966c4962e4dd4575 in_seq=0 out_seq=0 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb203fff640 10 client.15642 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb203fff640 1 -- 172.21.15.42:0/1823266969 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bbe0990890 con 0x55bbe0d178b0 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb203fff640 10 client.15642 handle_command_reply: tid=0 2024-03-12T04:15:33.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 client.15642 shutdown 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 msgr2=0x55bbe0d0fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0d0fc80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0ba010 tx=0x7fb21c0ba0b0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 shutdown_connections 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55bbe0d178b0 0x55bbe0d19ca0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bbe0ca6750 0x55bbe0d0fc80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbe0c9db70 0x55bbe0cad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 --2- 172.21.15.42:0/1823266969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bbe0c9d250 0x55bbe0cacc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb203fff640 0 client.15642 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 >> 172.21.15.42:0/1823266969 conn(0x55bbe0c97980 msgr2=0x55bbe0c96b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 shutdown_connections 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 1 -- 172.21.15.42:0/1823266969 wait complete. 2024-03-12T04:15:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.773+0000 7fb22d820740 20 client.15642 trim_cache size 0 max 16384 2024-03-12T04:15:33.783 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2140a1c30 msgr2=0x7fb2140a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2140a1c30 0x7fb2140a40f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb210009900 tx=0x7fb2100047d0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb22419aee0 msgr2=0x7fb22419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb22419aee0 0x7fb22419b390 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00eaa0 tx=0x7fb21c00ef60 comp rx=0 tx=0).stop 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 shutdown_connections 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2140a1c30 0x7fb2140a40f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb22419aee0 0x7fb22419b390 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb224105370 0x7fb224196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 --2- 172.21.15.42:0/2972804542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2241002a0 0x7fb224196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 >> 172.21.15.42:0/2972804542 conn(0x7fb224076400 msgr2=0x7fb2240fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 shutdown_connections 2024-03-12T04:15:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.774+0000 7fb22ab79640 1 -- 172.21.15.42:0/2972804542 wait complete. 2024-03-12T04:15:33.787 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:15:33.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:15:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.890+0000 7f869ff02640 1 -- 172.21.15.42:0/677129176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068c80 msgr2=0x7f869810d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.890+0000 7f869ff02640 1 --2- 172.21.15.42:0/677129176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068c80 0x7f869810d5e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8688009900 tx=0x7f868802bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.890+0000 7f869ff02640 1 -- 172.21.15.42:0/677129176 shutdown_connections 2024-03-12T04:15:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.890+0000 7f869ff02640 1 --2- 172.21.15.42:0/677129176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f869810db20 0x7f869810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.890+0000 7f869ff02640 1 --2- 172.21.15.42:0/677129176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068c80 0x7f869810d5e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.890+0000 7f869ff02640 1 --2- 172.21.15.42:0/677129176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8698068360 0x7f8698068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.890+0000 7f869ff02640 1 -- 172.21.15.42:0/677129176 >> 172.21.15.42:0/677129176 conn(0x7f8698077180 msgr2=0x7f8698077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 -- 172.21.15.42:0/677129176 shutdown_connections 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 -- 172.21.15.42:0/677129176 wait complete. 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 Processor -- start 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 -- start start 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068360 0x7f869819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698068c80 0x7f869819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869810db20 0x7f8698199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8698199e30 con 0x7f8698068360 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8698199fa0 con 0x7f8698068c80 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869ff02640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819a160 con 0x7f869810db20 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869dc9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068360 0x7f869819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869dc9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068360 0x7f869819f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49128/0 (socket says 172.21.15.42:49128) 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869dc9e640 1 -- 172.21.15.42:0/1857696904 learned_addr learned my addr 172.21.15.42:0/1857696904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.891+0000 7f869d49d640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698068c80 0x7f869819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869e49f640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869810db20 0x7f8698199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869dc9e640 1 -- 172.21.15.42:0/1857696904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869810db20 msgr2=0x7f8698199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869dc9e640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869810db20 0x7f8698199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869dc9e640 1 -- 172.21.15.42:0/1857696904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698068c80 msgr2=0x7f869819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869dc9e640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698068c80 0x7f869819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869dc9e640 1 -- 172.21.15.42:0/1857696904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688009560 con 0x7f8698068360 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869dc9e640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068360 0x7f869819f620 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f869000c970 tx=0x7f869000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f868effd640 1 -- 172.21.15.42:0/1857696904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690007bb0 con 0x7f8698068360 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869819a440 con 0x7f8698068360 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f868effd640 1 -- 172.21.15.42:0/1857696904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8690007d10 con 0x7f8698068360 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f868effd640 1 -- 172.21.15.42:0/1857696904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690004e80 con 0x7f8698068360 2024-03-12T04:15:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.892+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86981a6650 con 0x7f8698068360 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.894+0000 7f868effd640 1 -- 172.21.15.42:0/1857696904 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f8690005020 con 0x7f8698068360 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 Processor -- start 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f868effd640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f86840a1c30 0x7f86840a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f868effd640 1 -- 172.21.15.42:0/1857696904 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f86900cd730 con 0x7f8698068360 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f86840a1c30 0x7f86840a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 -- start start 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb278960 con 0x564aeb2c6750 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb22c310 con 0x564aeb2bdb70 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb229840 con 0x564aeb2bd250 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869d49d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49132/0 (socket says 172.21.15.42:49132) 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 -- 172.21.15.42:0/3075948081 learned_addr learned my addr 172.21.15.42:0/3075948081 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869dc9e640 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f86840a1c30 0x7f86840a40f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8694009370 tx=0x7f86940092c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 -- 172.21.15.42:0/3075948081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 msgr2=0x564aeb2bd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2bd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 -- 172.21.15.42:0/3075948081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 msgr2=0x564aeb2c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869e49f640 1 -- 172.21.15.42:0/3075948081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aeb1e7060 con 0x564aeb2c6750 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.895+0000 7f869d49d640 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:15:33.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.896+0000 7f869e49f640 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2c8b40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8694011600 tx=0x7f8694012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d74c5c744fcc9a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.896+0000 7f8683fff640 1 -- 172.21.15.42:0/3075948081 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694010690 con 0x564aeb2c6750 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.896+0000 7f8683fff640 1 -- 172.21.15.42:0/3075948081 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8694010ca0 con 0x564aeb2c6750 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.896+0000 7f8683fff640 1 -- 172.21.15.42:0/3075948081 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869401ec90 con 0x564aeb2c6750 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.896+0000 7f86a2bb2740 1 -- 172.21.15.42:0/3075948081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 msgr2=0x564aeb2c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.896+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2c8b40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8694011600 tx=0x7f8694012000 comp rx=0 tx=0).stop 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- 172.21.15.42:0/3075948081 shutdown_connections 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2c8b40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/3075948081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- 172.21.15.42:0/3075948081 >> 172.21.15.42:0/3075948081 conn(0x564aeb2b7980 msgr2=0x564aeb2b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- 172.21.15.42:0/3075948081 shutdown_connections 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- 172.21.15.42:0/3075948081 wait complete. 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- start start 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 Processor -- start 2024-03-12T04:15:33.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2bd250 0x564aeb2c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2c6750 0x564aeb2c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb225020 con 0x564aeb2bd250 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeaef8430 con 0x564aeb2c6750 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeaeb4e00 con 0x564aeb2bdb70 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.897+0000 7f86a2bb2740 10 client.0 _ll_register_callbacks cb 0x564aeb1cccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2bd250 0x564aeb2c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2bd250 0x564aeb2c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49140/0 (socket says 172.21.15.42:49140) 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 -- 172.21.15.42:0/1095134863 learned_addr learned my addr 172.21.15.42:0/1095134863 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869d49d640 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869e49f640 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2c6750 0x564aeb2c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 -- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bdb70 msgr2=0x564aeb2c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 -- 172.21.15.42:0/1095134863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2c6750 msgr2=0x564aeb2c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2c6750 0x564aeb2c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 -- 172.21.15.42:0/1095134863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869400fe40 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f869dc9e640 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2bd250 0x564aeb2c36e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f869009a170 tx=0x7f8690005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d58cd300cc5fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86817fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86817fa640 1 -- 172.21.15.42:0/1095134863 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86900a2980 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86817fa640 1 -- 172.21.15.42:0/1095134863 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f86900052d0 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564aeaf247e0 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86817fa640 1 -- 172.21.15.42:0/1095134863 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86900cb060 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86a2bb2740 10 client.15654 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564aeb2236a0 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.898+0000 7f86817fa640 1 -- 172.21.15.42:0/1095134863 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564aeaf247e0 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86817fa640 1 -- 172.21.15.42:0/1095134863 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8690016030 con 0x564aeb2bd250 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86a2bb2740 10 client.15654 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86a2bb2740 10 client.15654 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:15:33.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86a2bb2740 20 client.15654 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb2d6b10 0x564aeb2d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86a2bb2740 4 client.15654 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564aeaf61720 con 0x564aeb2d6b10 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f869d49d640 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb2d6b10 0x564aeb2d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f86817fa640 10 client.15654 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.899+0000 7f869d49d640 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb2d6b10 0x564aeb2d8f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=893f7fbb2ababa6 server_cookie=e7a02ea96c31f939 in_seq=0 out_seq=0 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86817fa640 1 -- 172.21.15.42:0/1095134863 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564aeaf61720 con 0x564aeb2d6b10 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86817fa640 10 client.15654 handle_command_reply: tid=0 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 client.15654 shutdown 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2bd250 msgr2=0x564aeb2c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2bd250 0x564aeb2c36e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f869009a170 tx=0x7f8690005700 comp rx=0 tx=0).stop 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 shutdown_connections 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb2d6b10 0x564aeb2d8f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2c6750 0x564aeb2c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86817fa640 0 client.15654 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:15:33.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1095134863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2bd250 0x564aeb2c36e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 >> 172.21.15.42:0/1095134863 conn(0x564aeb2b7980 msgr2=0x564aeb2b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 shutdown_connections 2024-03-12T04:15:33.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.900+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1095134863 wait complete. 2024-03-12T04:15:33.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.901+0000 7f86a2bb2740 20 client.15654 trim_cache size 0 max 16384 2024-03-12T04:15:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 Processor -- start 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 -- start start 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb1aa130 con 0x564aeb2c6750 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb2b8980 con 0x564aeb2bdb70 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.912+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb2b35c0 con 0x564aeb2bd250 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49156/0 (socket says 172.21.15.42:49156) 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 -- 172.21.15.42:0/1010383849 learned_addr learned my addr 172.21.15.42:0/1010383849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869dc9e640 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869d49d640 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 -- 172.21.15.42:0/1010383849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 msgr2=0x564aeb2b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 -- 172.21.15.42:0/1010383849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 msgr2=0x564aeb2c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 -- 172.21.15.42:0/1010383849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aeb1bfc20 con 0x564aeb2c6750 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f869e49f640 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2cd600 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f86940115a0 tx=0x7f8694005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9773e6141670a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f86837fe640 1 -- 172.21.15.42:0/1010383849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940049a0 con 0x564aeb2c6750 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f86837fe640 1 -- 172.21.15.42:0/1010383849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f869402e040 con 0x564aeb2c6750 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.913+0000 7f86837fe640 1 -- 172.21.15.42:0/1010383849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940053b0 con 0x564aeb2c6750 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1010383849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 msgr2=0x564aeb2cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2cd600 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f86940115a0 tx=0x7f8694005a10 comp rx=0 tx=0).stop 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1010383849 shutdown_connections 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb2cd600 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/1010383849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1010383849 >> 172.21.15.42:0/1010383849 conn(0x564aeb2b7980 msgr2=0x564aeb2b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1010383849 shutdown_connections 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 -- 172.21.15.42:0/1010383849 wait complete. 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 -- start start 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.914+0000 7f86a2bb2740 1 Processor -- start 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb32cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb2cd1c0 con 0x564aeb2c6750 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb32d4c0 con 0x564aeb2bdb70 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeb32d600 con 0x564aeb2bd250 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 10 client.0 _ll_register_callbacks cb 0x7f8694015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb32cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb32cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49160/0 (socket says 172.21.15.42:49160) 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 -- 172.21.15.42:0/447212571 learned_addr learned my addr 172.21.15.42:0/447212571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869d49d640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869dc9e640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 -- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 msgr2=0x564aeb2cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 -- 172.21.15.42:0/447212571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 msgr2=0x564aeb2cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 -- 172.21.15.42:0/447212571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564aeb1e7060 con 0x564aeb2c6750 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869dc9e640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f869e49f640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb32cf80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8694009bc0 tx=0x7f8694004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acc7b45a23133d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86817fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86817fa640 1 -- 172.21.15.42:0/447212571 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694010690 con 0x564aeb2c6750 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86817fa640 1 -- 172.21.15.42:0/447212571 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f86940107f0 con 0x564aeb2c6750 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.915+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8694010ca0 con 0x564aeb2c6750 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86817fa640 1 -- 172.21.15.42:0/447212571 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86940109b0 con 0x564aeb2c6750 2024-03-12T04:15:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 10 client.15660 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564aeb1e58c0 con 0x564aeb2c6750 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86817fa640 1 -- 172.21.15.42:0/447212571 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8694010ca0 con 0x564aeb2c6750 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86817fa640 1 -- 172.21.15.42:0/447212571 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f869403b5b0 con 0x564aeb2c6750 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 10 client.15660 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 20 client.15660 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb32e9b0 0x564aeb330dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 4 client.15660 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564aeafb0890 con 0x564aeb32e9b0 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.916+0000 7f869dc9e640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb32e9b0 0x564aeb330dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86817fa640 10 client.15660 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f869dc9e640 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb32e9b0 0x564aeb330dd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e490e60c37a3c2b server_cookie=e442a91f17d37c3c in_seq=0 out_seq=0 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86817fa640 1 -- 172.21.15.42:0/447212571 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564aeafb0890 con 0x564aeb32e9b0 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86817fa640 10 client.15660 handle_command_reply: tid=0 2024-03-12T04:15:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 client.15660 shutdown 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 msgr2=0x564aeb32cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb32cf80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8694009bc0 tx=0x7f8694004c50 comp rx=0 tx=0).stop 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 shutdown_connections 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x564aeb32e9b0 0x564aeb330dd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564aeb2c6750 0x564aeb32cf80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564aeb2bdb70 0x564aeb2cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86817fa640 0 client.15660 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 --2- 172.21.15.42:0/447212571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aeb2bd250 0x564aeb2cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 >> 172.21.15.42:0/447212571 conn(0x564aeb2b7980 msgr2=0x564aeb2b6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 shutdown_connections 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 1 -- 172.21.15.42:0/447212571 wait complete. 2024-03-12T04:15:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.917+0000 7f86a2bb2740 20 client.15660 trim_cache size 0 max 16384 2024-03-12T04:15:33.926 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:15:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f86840a1c30 msgr2=0x7f86840a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f86840a1c30 0x7f86840a40f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8694009370 tx=0x7f86940092c0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068360 msgr2=0x7f869819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068360 0x7f869819f620 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f869000c970 tx=0x7f869000ce30 comp rx=0 tx=0).stop 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 shutdown_connections 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f86840a1c30 0x7f86840a40f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869810db20 0x7f8698199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698068c80 0x7f869819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 --2- 172.21.15.42:0/1857696904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8698068360 0x7f869819f620 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 >> 172.21.15.42:0/1857696904 conn(0x7f8698077180 msgr2=0x7f869810f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 shutdown_connections 2024-03-12T04:15:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:33.918+0000 7f869ff02640 1 -- 172.21.15.42:0/1857696904 wait complete. 2024-03-12T04:15:33.928 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:33 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/700788584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:33.930 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:15:33.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:15:33.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:33 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/700788584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:33.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:33 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/700788584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 -- 172.21.15.42:0/2932797960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354107380 msgr2=0x7f3354107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 --2- 172.21.15.42:0/2932797960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354107380 0x7f3354107760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3350009900 tx=0x7f335002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 -- 172.21.15.42:0/2932797960 shutdown_connections 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 --2- 172.21.15.42:0/2932797960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354100400 0x7f335410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 --2- 172.21.15.42:0/2932797960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33540ffa40 0x7f33540ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 --2- 172.21.15.42:0/2932797960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354107380 0x7f3354107760 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 -- 172.21.15.42:0/2932797960 >> 172.21.15.42:0/2932797960 conn(0x7f33540753c0 msgr2=0x7f33540757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 -- 172.21.15.42:0/2932797960 shutdown_connections 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.033+0000 7f335c68a640 1 -- 172.21.15.42:0/2932797960 wait complete. 2024-03-12T04:15:34.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 Processor -- start 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 -- start start 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33540ffa40 0x7f3354197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354100400 0x7f3354197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107380 0x7f335419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335419c5c0 con 0x7f3354100400 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335419c730 con 0x7f3354107380 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335c68a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335419c8f0 con 0x7f33540ffa40 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354100400 0x7f3354197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354100400 0x7f3354197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49170/0 (socket says 172.21.15.42:49170) 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 -- 172.21.15.42:0/974729075 learned_addr learned my addr 172.21.15.42:0/974729075 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335a426640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33540ffa40 0x7f3354197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f335ac27640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107380 0x7f335419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 -- 172.21.15.42:0/974729075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33540ffa40 msgr2=0x7f3354197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33540ffa40 0x7f3354197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 -- 172.21.15.42:0/974729075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107380 msgr2=0x7f335419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107380 0x7f335419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.034+0000 7f3359c25640 1 -- 172.21.15.42:0/974729075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3350009560 con 0x7f3354100400 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.035+0000 7f3359c25640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354100400 0x7f3354197c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f334400c920 tx=0x7f334400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.035+0000 7f334b7fe640 1 -- 172.21.15.42:0/974729075 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344007c70 con 0x7f3354100400 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.035+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335419cbd0 con 0x7f3354100400 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.035+0000 7f334b7fe640 1 -- 172.21.15.42:0/974729075 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3344007dd0 con 0x7f3354100400 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.035+0000 7f334b7fe640 1 -- 172.21.15.42:0/974729075 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440187c0 con 0x7f3354100400 2024-03-12T04:15:34.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.035+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33541a4450 con 0x7f3354100400 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.037+0000 7f334b7fe640 1 -- 172.21.15.42:0/974729075 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f334400f450 con 0x7f3354100400 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.037+0000 7f335f256740 1 Processor -- start 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335f256740 1 -- start start 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335f256740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2250 0x55e24d6f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335f256740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2b70 0x55e24d6fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335f256740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6fb750 0x55e24d6fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335f256740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d6ad960 con 0x55e24d6fb750 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335f256740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d661310 con 0x55e24d6f2250 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335f256740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d65e840 con 0x55e24d6f2b70 2024-03-12T04:15:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f334b7fe640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f33400a1c30 0x7f33400a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6fb750 0x55e24d6fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6fb750 0x55e24d6fdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49178/0 (socket says 172.21.15.42:49178) 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 -- 172.21.15.42:0/2807137240 learned_addr learned my addr 172.21.15.42:0/2807137240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335ac27640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f33400a1c30 0x7f33400a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335a426640 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2b70 0x55e24d6fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f335ac27640 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2250 0x55e24d6f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f334b7fe640 1 -- 172.21.15.42:0/974729075 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f33440cdb80 con 0x7f3354100400 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 -- 172.21.15.42:0/2807137240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2b70 msgr2=0x55e24d6fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2b70 0x55e24d6fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 -- 172.21.15.42:0/2807137240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2250 msgr2=0x55e24d6f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2250 0x55e24d6f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.038+0000 7f3359c25640 1 -- 172.21.15.42:0/2807137240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24d61c060 con 0x55e24d6fb750 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.039+0000 7f335ac27640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f33400a1c30 0x7f33400a40f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f334c009370 tx=0x7f334c009c50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.039+0000 7f3359c25640 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6fb750 0x55e24d6fdb40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f33440cb920 tx=0x7f334400a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8983ac70a14fc29e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.039+0000 7f3348ff9640 1 -- 172.21.15.42:0/2807137240 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400fa90 con 0x55e24d6fb750 2024-03-12T04:15:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.039+0000 7f3348ff9640 1 -- 172.21.15.42:0/2807137240 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f334409ad90 con 0x55e24d6fb750 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.039+0000 7f3348ff9640 1 -- 172.21.15.42:0/2807137240 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344096d00 con 0x55e24d6fb750 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.039+0000 7f335f256740 1 -- 172.21.15.42:0/2807137240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6fb750 msgr2=0x55e24d6fdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.039+0000 7f335f256740 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6fb750 0x55e24d6fdb40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f33440cb920 tx=0x7f334400a630 comp rx=0 tx=0).stop 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- 172.21.15.42:0/2807137240 shutdown_connections 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6fb750 0x55e24d6fdb40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2b70 0x55e24d6fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 --2- 172.21.15.42:0/2807137240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2250 0x55e24d6f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- 172.21.15.42:0/2807137240 >> 172.21.15.42:0/2807137240 conn(0x55e24d6ec980 msgr2=0x55e24d6ecd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- 172.21.15.42:0/2807137240 shutdown_connections 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- 172.21.15.42:0/2807137240 wait complete. 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- start start 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 Processor -- start 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6f8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 0x55e24d6f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d65a020 con 0x55e24d6f2250 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d32d430 con 0x55e24d6f2b70 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d2e9e00 con 0x55e24d6fb750 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335f256740 10 client.0 _ll_register_callbacks cb 0x55e24d601cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335a426640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335a426640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6f8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49184/0 (socket says 172.21.15.42:49184) 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.040+0000 7f335a426640 1 -- 172.21.15.42:0/3502618293 learned_addr learned my addr 172.21.15.42:0/3502618293 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f3359c25640 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335ac27640 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 0x55e24d6f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335a426640 1 -- 172.21.15.42:0/3502618293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 msgr2=0x55e24d6f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335a426640 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 0x55e24d6f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335a426640 1 -- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 msgr2=0x55e24d6f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335a426640 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335a426640 1 -- 172.21.15.42:0/3502618293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33541a4450 con 0x55e24d6f2250 2024-03-12T04:15:34.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335a426640 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6f8780 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3350009900 tx=0x7f335002c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00467b1e2597551 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f333dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f333dffb640 1 -- 172.21.15.42:0/3502618293 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350004460 con 0x55e24d6f2250 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f333dffb640 1 -- 172.21.15.42:0/3502618293 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f33500035f0 con 0x55e24d6f2250 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e24d3597e0 con 0x55e24d6f2250 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f333dffb640 1 -- 172.21.15.42:0/3502618293 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335002cbd0 con 0x55e24d6f2250 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335f256740 10 client.15672 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e24d6586a0 con 0x55e24d6f2250 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.041+0000 7f333dffb640 1 -- 172.21.15.42:0/3502618293 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e24d3597e0 con 0x55e24d6f2250 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f333dffb640 1 -- 172.21.15.42:0/3502618293 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f335004f050 con 0x55e24d6f2250 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f335f256740 10 client.15672 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f335f256740 10 client.15672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f335f256740 20 client.15672 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f335f256740 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d70bb40 0x55e24d70df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f335f256740 4 client.15672 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e24d396720 con 0x55e24d70bb40 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f3359c25640 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d70bb40 0x55e24d70df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f333dffb640 10 client.15672 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.042+0000 7f3359c25640 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d70bb40 0x55e24d70df80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=688dad8db8c546e4 server_cookie=56fda76971fec0d3 in_seq=0 out_seq=0 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f333dffb640 1 -- 172.21.15.42:0/3502618293 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e24d396720 con 0x55e24d70bb40 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f333dffb640 10 client.15672 handle_command_reply: tid=0 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 client.15672 shutdown 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 msgr2=0x55e24d6f8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6f8780 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3350009900 tx=0x7f335002c9a0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 shutdown_connections 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d70bb40 0x55e24d70df80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 0x55e24d6f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f333dffb640 0 client.15672 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 --2- 172.21.15.42:0/3502618293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6f8780 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 >> 172.21.15.42:0/3502618293 conn(0x55e24d6ec980 msgr2=0x55e24d6ee6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 shutdown_connections 2024-03-12T04:15:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.043+0000 7f335f256740 1 -- 172.21.15.42:0/3502618293 wait complete. 2024-03-12T04:15:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.044+0000 7f335f256740 20 client.15672 trim_cache size 0 max 16384 2024-03-12T04:15:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 Processor -- start 2024-03-12T04:15:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 -- start start 2024-03-12T04:15:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 0x55e24d6f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d5df130 con 0x55e24d6f2250 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d6f1eb0 con 0x55e24d6f2b70 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.055+0000 7f335f256740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d6ed980 con 0x55e24d6fb750 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6faeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49196/0 (socket says 172.21.15.42:49196) 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 -- 172.21.15.42:0/2641421615 learned_addr learned my addr 172.21.15.42:0/2641421615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f3359c25640 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 -- 172.21.15.42:0/2641421615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 msgr2=0x55e24d6f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 0x55e24d6f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 -- 172.21.15.42:0/2641421615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 msgr2=0x55e24d6f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 -- 172.21.15.42:0/2641421615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24d5f4c20 con 0x55e24d6f2250 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f3359c25640 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f335a426640 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6faeb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f335002c170 tx=0x7f335002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e398f76290d2a591 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f333ffff640 1 -- 172.21.15.42:0/2641421615 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350004af0 con 0x55e24d6f2250 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f333ffff640 1 -- 172.21.15.42:0/2641421615 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f335002de90 con 0x55e24d6f2250 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.056+0000 7f333ffff640 1 -- 172.21.15.42:0/2641421615 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350003850 con 0x55e24d6f2250 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- 172.21.15.42:0/2641421615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 msgr2=0x55e24d6faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6faeb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f335002c170 tx=0x7f335002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- 172.21.15.42:0/2641421615 shutdown_connections 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6fb750 0x55e24d6f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6f2b70 0x55e24d6f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 --2- 172.21.15.42:0/2641421615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2250 0x55e24d6faeb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- 172.21.15.42:0/2641421615 >> 172.21.15.42:0/2641421615 conn(0x55e24d6ec980 msgr2=0x55e24d6e7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- 172.21.15.42:0/2641421615 shutdown_connections 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- 172.21.15.42:0/2641421615 wait complete. 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- start start 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 Processor -- start 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2250 0x55e24d702a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2b70 0x55e24d700de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6fb750 0x55e24d701460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d701b60 con 0x55e24d6f2b70 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d7620b0 con 0x55e24d6fb750 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24d7621f0 con 0x55e24d6f2250 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f335f256740 10 client.0 _ll_register_callbacks cb 0x7f33440cb0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f3359c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2b70 0x55e24d700de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f3359c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2b70 0x55e24d700de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49198/0 (socket says 172.21.15.42:49198) 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.057+0000 7f3359c25640 1 -- 172.21.15.42:0/832657034 learned_addr learned my addr 172.21.15.42:0/832657034 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335a426640 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2250 0x55e24d702a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335ac27640 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6fb750 0x55e24d701460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f3359c25640 1 -- 172.21.15.42:0/832657034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2250 msgr2=0x55e24d702a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f3359c25640 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2250 0x55e24d702a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f3359c25640 1 -- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6fb750 msgr2=0x55e24d701460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f3359c25640 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6fb750 0x55e24d701460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f3359c25640 1 -- 172.21.15.42:0/832657034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e24d6586a0 con 0x55e24d6f2b70 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f3359c25640 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2b70 0x55e24d700de0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f334409a390 tx=0x7f334409a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc0cfccba13c078 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f333dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f333dffb640 1 -- 172.21.15.42:0/832657034 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400fa90 con 0x55e24d6f2b70 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f333dffb640 1 -- 172.21.15.42:0/832657034 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f33440a2e30 con 0x55e24d6f2b70 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f334409ad90 con 0x55e24d6f2b70 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f333dffb640 1 -- 172.21.15.42:0/832657034 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f334409a690 con 0x55e24d6f2b70 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 10 client.15678 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e24d61a8c0 con 0x55e24d6f2b70 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f333dffb640 1 -- 172.21.15.42:0/832657034 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f334409ad90 con 0x55e24d6f2b70 2024-03-12T04:15:34.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f333dffb640 1 -- 172.21.15.42:0/832657034 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3344063070 con 0x55e24d6f2b70 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 10 client.15678 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 10 client.15678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 20 client.15678 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d763540 0x55e24d765960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 4 client.15678 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.058+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e24d3e5890 con 0x55e24d763540 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.059+0000 7f335a426640 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d763540 0x55e24d765960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.059+0000 7f333dffb640 10 client.15678 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.059+0000 7f335a426640 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d763540 0x55e24d765960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0757f89a23cfc70 server_cookie=6912b5caf27121b4 in_seq=0 out_seq=0 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.059+0000 7f333dffb640 1 -- 172.21.15.42:0/832657034 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e24d3e5890 con 0x55e24d763540 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.059+0000 7f333dffb640 10 client.15678 handle_command_reply: tid=0 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.059+0000 7f335f256740 1 client.15678 shutdown 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2b70 msgr2=0x55e24d700de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2b70 0x55e24d700de0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f334409a390 tx=0x7f334409a0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 shutdown_connections 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e24d763540 0x55e24d765960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24d6fb750 0x55e24d701460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e24d6f2b70 0x55e24d700de0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 --2- 172.21.15.42:0/832657034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e24d6f2250 0x55e24d702a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f333dffb640 0 client.15678 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 >> 172.21.15.42:0/832657034 conn(0x55e24d6ec980 msgr2=0x55e24d6f3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 shutdown_connections 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 1 -- 172.21.15.42:0/832657034 wait complete. 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335f256740 20 client.15678 trim_cache size 0 max 16384 2024-03-12T04:15:34.068 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:15:34.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f33400a1c30 msgr2=0x7f33400a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f33400a1c30 0x7f33400a40f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f334c009370 tx=0x7f334c009c50 comp rx=0 tx=0).stop 2024-03-12T04:15:34.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354100400 msgr2=0x7f3354197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354100400 0x7f3354197c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f334400c920 tx=0x7f334400cde0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 shutdown_connections 2024-03-12T04:15:34.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f33400a1c30 0x7f33400a40f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107380 0x7f335419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3354100400 0x7f3354197c40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 --2- 172.21.15.42:0/974729075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33540ffa40 0x7f3354197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 >> 172.21.15.42:0/974729075 conn(0x7f33540753c0 msgr2=0x7f33540fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 shutdown_connections 2024-03-12T04:15:34.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.060+0000 7f335c68a640 1 -- 172.21.15.42:0/974729075 wait complete. 2024-03-12T04:15:34.072 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:15:34.072 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 -- 172.21.15.42:0/802048871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a80ffcf0 msgr2=0x7fb2a81000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 --2- 172.21.15.42:0/802048871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a80ffcf0 0x7fb2a81000d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb298009930 tx=0x7fb29802bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 -- 172.21.15.42:0/802048871 shutdown_connections 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 --2- 172.21.15.42:0/802048871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a81064b0 0x7fb2a810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 --2- 172.21.15.42:0/802048871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a81006a0 0x7fb2a8105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 --2- 172.21.15.42:0/802048871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a80ffcf0 0x7fb2a81000d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 -- 172.21.15.42:0/802048871 >> 172.21.15.42:0/802048871 conn(0x7fb2a80752e0 msgr2=0x7fb2a80756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 -- 172.21.15.42:0/802048871 shutdown_connections 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 -- 172.21.15.42:0/802048871 wait complete. 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.176+0000 7fb2ad040640 1 Processor -- start 2024-03-12T04:15:34.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2ad040640 1 -- start start 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2ad040640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80ffcf0 0x7fb2a819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2ad040640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a81064b0 0x7fb2a819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2ad040640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a8197640 0x7fb2a8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2ad040640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8198030 con 0x7fb2a81064b0 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2ad040640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a81981a0 con 0x7fb2a80ffcf0 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2ad040640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8198360 con 0x7fb2a8197640 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a81064b0 0x7fb2a819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a81064b0 0x7fb2a819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49218/0 (socket says 172.21.15.42:49218) 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 -- 172.21.15.42:0/165156520 learned_addr learned my addr 172.21.15.42:0/165156520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a759e640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a8197640 0x7fb2a8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80ffcf0 0x7fb2a819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 -- 172.21.15.42:0/165156520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a8197640 msgr2=0x7fb2a8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a8197640 0x7fb2a8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 -- 172.21.15.42:0/165156520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80ffcf0 msgr2=0x7fb2a819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80ffcf0 0x7fb2a819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 -- 172.21.15.42:0/165156520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb298009560 con 0x7fb2a81064b0 2024-03-12T04:15:34.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.177+0000 7fb2a659c640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a81064b0 0x7fb2a819d980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00d740 tx=0x7fb29c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.178+0000 7fb28ffff640 1 -- 172.21.15.42:0/165156520 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c009c30 con 0x7fb2a81064b0 2024-03-12T04:15:34.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.178+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a81a4300 con 0x7fb2a81064b0 2024-03-12T04:15:34.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.178+0000 7fb28ffff640 1 -- 172.21.15.42:0/165156520 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb29c00de50 con 0x7fb2a81064b0 2024-03-12T04:15:34.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.178+0000 7fb28ffff640 1 -- 172.21.15.42:0/165156520 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c004e40 con 0x7fb2a81064b0 2024-03-12T04:15:34.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.178+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a81a4810 con 0x7fb2a81064b0 2024-03-12T04:15:34.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb28ffff640 1 -- 172.21.15.42:0/165156520 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7fb29c004fa0 con 0x7fb2a81064b0 2024-03-12T04:15:34.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 Processor -- start 2024-03-12T04:15:34.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 -- start start 2024-03-12T04:15:34.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6250 0x5625b97e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ef210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97ef750 0x5625b97f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b97a1960 con 0x5625b97e6250 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b9755310 con 0x5625b97ef750 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2afc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b9752840 con 0x5625b97e6b70 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6250 0x5625b97e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6250 0x5625b97e6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49226/0 (socket says 172.21.15.42:49226) 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2a759e640 1 -- 172.21.15.42:0/1592861492 learned_addr learned my addr 172.21.15.42:0/1592861492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb28ffff640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2900a1c30 0x7fb2900a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.180+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a659c640 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97ef750 0x5625b97f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2900a1c30 0x7fb2900a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 -- 172.21.15.42:0/1592861492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 msgr2=0x5625b97ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb28ffff640 1 -- 172.21.15.42:0/165156520 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb29c0cd320 con 0x7fb2a81064b0 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 -- 172.21.15.42:0/1592861492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97ef750 msgr2=0x5625b97f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97ef750 0x5625b97f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 -- 172.21.15.42:0/1592861492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b9710060 con 0x5625b97e6250 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a659c640 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97ef750 0x5625b97f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6250 0x5625b97e6630 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0013940 tx=0x7fb2a0013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91336775470b6ab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb2a759e640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2900a1c30 0x7fb2900a40f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00117b0 tx=0x7fb2a001e000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb28d7fa640 1 -- 172.21.15.42:0/1592861492 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0013250 con 0x5625b97e6250 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb28d7fa640 1 -- 172.21.15.42:0/1592861492 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb2a0019070 con 0x5625b97e6250 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.181+0000 7fb28d7fa640 1 -- 172.21.15.42:0/1592861492 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0004280 con 0x5625b97e6250 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 -- 172.21.15.42:0/1592861492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6250 msgr2=0x5625b97e6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6250 0x5625b97e6630 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0013940 tx=0x7fb2a0013e00 comp rx=0 tx=0).stop 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 -- 172.21.15.42:0/1592861492 shutdown_connections 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97ef750 0x5625b97f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 --2- 172.21.15.42:0/1592861492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6250 0x5625b97e6630 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 -- 172.21.15.42:0/1592861492 >> 172.21.15.42:0/1592861492 conn(0x5625b97e0980 msgr2=0x5625b97e0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 -- 172.21.15.42:0/1592861492 shutdown_connections 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 -- 172.21.15.42:0/1592861492 wait complete. 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 -- start start 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.182+0000 7fb2afc14740 1 Processor -- start 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97ec750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b974e020 con 0x5625b97ef750 2024-03-12T04:15:34.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b9421430 con 0x5625b97e6250 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b93dde00 con 0x5625b97e6b70 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 10 client.0 _ll_register_callbacks cb 0x5625b96f5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97ed1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49232/0 (socket says 172.21.15.42:49232) 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 -- 172.21.15.42:0/159501002 learned_addr learned my addr 172.21.15.42:0/159501002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a659c640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 -- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 msgr2=0x5625b97ecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 -- 172.21.15.42:0/159501002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 msgr2=0x5625b97ec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97ec750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 -- 172.21.15.42:0/159501002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a0013580 con 0x5625b97ef750 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a659c640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2a759e640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97ed1d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005e50 tx=0x7fb2a0005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce08e2f41add4b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb28aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb28aff5640 1 -- 172.21.15.42:0/159501002 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0003b70 con 0x5625b97ef750 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb28aff5640 1 -- 172.21.15.42:0/159501002 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb2a0003cd0 con 0x5625b97ef750 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625b944d7e0 con 0x5625b97ef750 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb28aff5640 1 -- 172.21.15.42:0/159501002 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2a001dc20 con 0x5625b97ef750 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 10 client.15690 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.183+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625b974c6a0 con 0x5625b97ef750 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb28aff5640 1 -- 172.21.15.42:0/159501002 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5625b944d7e0 con 0x5625b97ef750 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb28aff5640 1 -- 172.21.15.42:0/159501002 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb2a00254b0 con 0x5625b97ef750 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2afc14740 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2afc14740 10 client.15690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2afc14740 20 client.15690 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2afc14740 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b97ffaa0 0x5625b9801e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2afc14740 4 client.15690 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625b948a720 con 0x5625b97ffaa0 2024-03-12T04:15:34.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b97ffaa0 0x5625b9801e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb28aff5640 10 client.15690 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.184+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b97ffaa0 0x5625b9801e90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=135a44a34937d744 server_cookie=673d35fa2a54060 in_seq=0 out_seq=0 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb28aff5640 1 -- 172.21.15.42:0/159501002 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625b948a720 con 0x5625b97ffaa0 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb28aff5640 10 client.15690 handle_command_reply: tid=0 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 client.15690 shutdown 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 msgr2=0x5625b97ed1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97ed1d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005e50 tx=0x7fb2a0005f60 comp rx=0 tx=0).stop 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 shutdown_connections 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b97ffaa0 0x5625b9801e90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97ed1d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6b70 0x5625b97ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb28aff5640 0 client.15690 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 --2- 172.21.15.42:0/159501002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97ec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 >> 172.21.15.42:0/159501002 conn(0x5625b97e0980 msgr2=0x5625b97e26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 shutdown_connections 2024-03-12T04:15:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.185+0000 7fb2afc14740 1 -- 172.21.15.42:0/159501002 wait complete. 2024-03-12T04:15:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.186+0000 7fb2afc14740 20 client.15690 trim_cache size 0 max 16384 2024-03-12T04:15:34.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.197+0000 7fb2afc14740 1 Processor -- start 2024-03-12T04:15:34.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.197+0000 7fb2afc14740 1 -- start start 2024-03-12T04:15:34.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.197+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.197+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6b70 0x5625b97ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.197+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97ef750 0x5625b97e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.197+0000 7fb2afc14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b96d3130 con 0x5625b97e6b70 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.197+0000 7fb2afc14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b97e1980 con 0x5625b97e6250 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2afc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b98070a0 con 0x5625b97ef750 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6b70 0x5625b97ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6b70 0x5625b97ecff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49234/0 (socket says 172.21.15.42:49234) 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 -- 172.21.15.42:0/3580382755 learned_addr learned my addr 172.21.15.42:0/3580382755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a759e640 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97ef750 0x5625b97e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 -- 172.21.15.42:0/3580382755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97ef750 msgr2=0x5625b97e99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97ef750 0x5625b97e99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 -- 172.21.15.42:0/3580382755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 msgr2=0x5625b97eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 -- 172.21.15.42:0/3580382755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b96e8c20 con 0x5625b97e6b70 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a759e640 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97ef750 0x5625b97e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb2a659c640 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6b70 0x5625b97ecff0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb29c096800 tx=0x7fb29c06c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8339525443b55c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb28cff9640 1 -- 172.21.15.42:0/3580382755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c09a530 con 0x5625b97e6b70 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb28cff9640 1 -- 172.21.15.42:0/3580382755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb29c099e30 con 0x5625b97e6b70 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.198+0000 7fb28cff9640 1 -- 172.21.15.42:0/3580382755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0964a0 con 0x5625b97e6b70 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- 172.21.15.42:0/3580382755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6b70 msgr2=0x5625b97ecff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6b70 0x5625b97ecff0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb29c096800 tx=0x7fb29c06c030 comp rx=0 tx=0).stop 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- 172.21.15.42:0/3580382755 shutdown_connections 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97ef750 0x5625b97e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97e6b70 0x5625b97ecff0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 --2- 172.21.15.42:0/3580382755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6250 0x5625b97eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- 172.21.15.42:0/3580382755 >> 172.21.15.42:0/3580382755 conn(0x5625b97e0980 msgr2=0x5625b97e7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- 172.21.15.42:0/3580382755 shutdown_connections 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- 172.21.15.42:0/3580382755 wait complete. 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- start start 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 Processor -- start 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6250 0x5625b9850370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6b70 0x5625b98508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97f5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b97f5e70 con 0x5625b97ef750 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b97f5fe0 con 0x5625b97e6b70 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b97f61a0 con 0x5625b97e6250 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.199+0000 7fb2afc14740 10 client.0 _ll_register_callbacks cb 0x7fb2a000a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49248/0 (socket says 172.21.15.42:49248) 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 -- 172.21.15.42:0/4046027681 learned_addr learned my addr 172.21.15.42:0/4046027681 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a659c640 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6b70 0x5625b98508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6250 0x5625b9850370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 -- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6250 msgr2=0x5625b9850370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6250 0x5625b9850370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 -- 172.21.15.42:0/4046027681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6b70 msgr2=0x5625b98508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6b70 0x5625b98508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 -- 172.21.15.42:0/4046027681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a81a4810 con 0x5625b97ef750 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2a759e640 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97f5900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000aa30 tx=0x7fb2a000ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14120b9d0929089f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb28aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb28aff5640 1 -- 172.21.15.42:0/4046027681 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0004c00 con 0x5625b97ef750 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb28aff5640 1 -- 172.21.15.42:0/4046027681 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb2a0025630 con 0x5625b97ef750 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2a0019070 con 0x5625b97ef750 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb28aff5640 1 -- 172.21.15.42:0/4046027681 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2a001de80 con 0x5625b97ef750 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2afc14740 10 client.15696 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625b970e8c0 con 0x5625b97ef750 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.200+0000 7fb28aff5640 1 -- 172.21.15.42:0/4046027681 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb2a0019070 con 0x5625b97ef750 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb28aff5640 1 -- 172.21.15.42:0/4046027681 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb2a003a460 con 0x5625b97ef750 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2afc14740 10 client.15696 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2afc14740 10 client.15696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2afc14740 20 client.15696 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2afc14740 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b9857fb0 0x5625b985a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2afc14740 4 client.15696 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5625b94d9890 con 0x5625b9857fb0 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b9857fb0 0x5625b985a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb28aff5640 10 client.15696 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb2a6d9d640 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b9857fb0 0x5625b985a3a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d7dd7ac0bbf1446 server_cookie=28b2cd35c2b25f02 in_seq=0 out_seq=0 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb28aff5640 1 -- 172.21.15.42:0/4046027681 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5625b94d9890 con 0x5625b9857fb0 2024-03-12T04:15:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.201+0000 7fb28aff5640 10 client.15696 handle_command_reply: tid=0 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 client.15696 shutdown 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 msgr2=0x5625b97f5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97f5900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000aa30 tx=0x7fb2a000ad10 comp rx=0 tx=0).stop 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 shutdown_connections 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5625b9857fb0 0x5625b985a3a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625b97ef750 0x5625b97f5900 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb28aff5640 0 client.15696 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625b97e6b70 0x5625b98508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 --2- 172.21.15.42:0/4046027681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5625b97e6250 0x5625b9850370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 >> 172.21.15.42:0/4046027681 conn(0x5625b97e0980 msgr2=0x5625b97df2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 shutdown_connections 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 1 -- 172.21.15.42:0/4046027681 wait complete. 2024-03-12T04:15:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2afc14740 20 client.15696 trim_cache size 0 max 16384 2024-03-12T04:15:34.212 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:15:34.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2900a1c30 msgr2=0x7fb2900a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2900a1c30 0x7fb2900a40f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00117b0 tx=0x7fb2a001e000 comp rx=0 tx=0).stop 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a81064b0 msgr2=0x7fb2a819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a81064b0 0x7fb2a819d980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00d740 tx=0x7fb29c00dc00 comp rx=0 tx=0).stop 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 shutdown_connections 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb2900a1c30 0x7fb2900a40f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a8197640 0x7fb2a8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2a81064b0 0x7fb2a819d980 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 --2- 172.21.15.42:0/165156520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80ffcf0 0x7fb2a819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 >> 172.21.15.42:0/165156520 conn(0x7fb2a80752e0 msgr2=0x7fb2a80fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 shutdown_connections 2024-03-12T04:15:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.202+0000 7fb2ad040640 1 -- 172.21.15.42:0/165156520 wait complete. 2024-03-12T04:15:34.215 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:15:34.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 -- 172.21.15.42:0/1193289689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31180ff8f0 msgr2=0x7f31180ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 --2- 172.21.15.42:0/1193289689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31180ff8f0 0x7f31180ffcd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3110009930 tx=0x7f311002bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 -- 172.21.15.42:0/1193289689 shutdown_connections 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 --2- 172.21.15.42:0/1193289689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3118106500 0x7f311810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 --2- 172.21.15.42:0/1193289689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31181002a0 0x7f3118105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 --2- 172.21.15.42:0/1193289689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31180ff8f0 0x7f31180ffcd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 -- 172.21.15.42:0/1193289689 >> 172.21.15.42:0/1193289689 conn(0x7f3118076640 msgr2=0x7f31180fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 -- 172.21.15.42:0/1193289689 shutdown_connections 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.318+0000 7f311f393640 1 -- 172.21.15.42:0/1193289689 wait complete. 2024-03-12T04:15:34.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 Processor -- start 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 -- start start 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31180ff8f0 0x7f3118100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3118106500 0x7f3118100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31181012e0 0x7f31181a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31181a5eb0 con 0x7f3118106500 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31181a6020 con 0x7f31181012e0 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311f393640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31181a61e0 con 0x7f31180ff8f0 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3118106500 0x7f3118100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3118106500 0x7f3118100da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49260/0 (socket says 172.21.15.42:49260) 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 -- 172.21.15.42:0/896886576 learned_addr learned my addr 172.21.15.42:0/896886576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311d930640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31181012e0 0x7f31181a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311d12f640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31180ff8f0 0x7f3118100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 -- 172.21.15.42:0/896886576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31180ff8f0 msgr2=0x7f3118100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31180ff8f0 0x7f3118100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 -- 172.21.15.42:0/896886576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31181012e0 msgr2=0x7f31181a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31181012e0 0x7f31181a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 -- 172.21.15.42:0/896886576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110009560 con 0x7f3118106500 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.319+0000 7f311c92e640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3118106500 0x7f3118100da0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f310800c8c0 tx=0x7f310800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.320+0000 7f310e7fc640 1 -- 172.21.15.42:0/896886576 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108007bb0 con 0x7f3118106500 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.320+0000 7f310e7fc640 1 -- 172.21.15.42:0/896886576 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3108007d10 con 0x7f3118106500 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.320+0000 7f310e7fc640 1 -- 172.21.15.42:0/896886576 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31080186f0 con 0x7f3118106500 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.320+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31181a64c0 con 0x7f3118106500 2024-03-12T04:15:34.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.320+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31181a6a20 con 0x7f3118106500 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f310e7fc640 1 -- 172.21.15.42:0/896886576 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f310800f450 con 0x7f3118106500 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 Processor -- start 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 -- start start 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8250 0x558a5a3c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8b70 0x558a5a3d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a383960 con 0x558a5a3c8250 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a337310 con 0x558a5a3d1750 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f3121f6c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a334840 con 0x558a5a3c8b70 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f311d930640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8250 0x558a5a3c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f311d930640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8250 0x558a5a3c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49268/0 (socket says 172.21.15.42:49268) 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.322+0000 7f311d930640 1 -- 172.21.15.42:0/2222326086 learned_addr learned my addr 172.21.15.42:0/2222326086 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d12f640 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8b70 0x558a5a3d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f310e7fc640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f31040a1ce0 0x7f31040a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311c92e640 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 -- 172.21.15.42:0/2222326086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8b70 msgr2=0x558a5a3d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8b70 0x558a5a3d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 -- 172.21.15.42:0/2222326086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 msgr2=0x558a5a3d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3d3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 -- 172.21.15.42:0/2222326086 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a5a2f2060 con 0x558a5a3c8250 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311c92e640 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f31040a1ce0 0x7f31040a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f310e7fc640 1 -- 172.21.15.42:0/896886576 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f31080cdaf0 con 0x7f3118106500 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8250 0x558a5a3c8630 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3114008ec0 tx=0x7f311400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c6efbf60340dca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f31037fe640 1 -- 172.21.15.42:0/2222326086 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114011ab0 con 0x558a5a3c8250 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f31037fe640 1 -- 172.21.15.42:0/2222326086 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f31140087f0 con 0x558a5a3c8250 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f31037fe640 1 -- 172.21.15.42:0/2222326086 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311401c950 con 0x558a5a3c8250 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.323+0000 7f311d930640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f31040a1ce0 0x7f31040a41a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f311401f000 tx=0x7f3114020000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 -- 172.21.15.42:0/2222326086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8250 msgr2=0x558a5a3c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8250 0x558a5a3c8630 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3114008ec0 tx=0x7f311400e720 comp rx=0 tx=0).stop 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 -- 172.21.15.42:0/2222326086 shutdown_connections 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8b70 0x558a5a3d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2222326086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8250 0x558a5a3c8630 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 -- 172.21.15.42:0/2222326086 >> 172.21.15.42:0/2222326086 conn(0x558a5a3c2980 msgr2=0x558a5a3c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 -- 172.21.15.42:0/2222326086 shutdown_connections 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 -- 172.21.15.42:0/2222326086 wait complete. 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 -- start start 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.324+0000 7f3121f6c740 1 Processor -- start 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3121f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a330020 con 0x558a5a3c8b70 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3121f6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a003430 con 0x558a5a3d1750 2024-03-12T04:15:34.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3121f6c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a59fbfe00 con 0x558a5a3c8250 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3121f6c740 10 client.0 _ll_register_callbacks cb 0x558a5a2d7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49280/0 (socket says 172.21.15.42:49280) 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 -- 172.21.15.42:0/2165688875 learned_addr learned my addr 172.21.15.42:0/2165688875 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311d12f640 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 -- 172.21.15.42:0/2165688875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 msgr2=0x558a5a3ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 -- 172.21.15.42:0/2165688875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 msgr2=0x558a5a3cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 -- 172.21.15.42:0/2165688875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31140089c0 con 0x558a5a3c8b70 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f311c92e640 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3ced10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f31080cb260 tx=0x7f310809adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ac0b35cb795193 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.325+0000 7f3100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3100ff9640 1 -- 172.21.15.42:0/2165688875 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310809e750 con 0x558a5a3c8b70 2024-03-12T04:15:34.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3100ff9640 1 -- 172.21.15.42:0/2165688875 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f31080a2da0 con 0x558a5a3c8b70 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a5a06c720 con 0x558a5a3c8b70 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3100ff9640 1 -- 172.21.15.42:0/2165688875 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31080850f0 con 0x558a5a3c8b70 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 10 client.15708 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558a5a32e6a0 con 0x558a5a3c8b70 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3100ff9640 1 -- 172.21.15.42:0/2165688875 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558a5a06c720 con 0x558a5a3c8b70 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3100ff9640 1 -- 172.21.15.42:0/2165688875 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3108016020 con 0x558a5a3c8b70 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 20 client.15708 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a3e1bf0 0x558a5a3e4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 4 client.15708 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a5a0bb890 con 0x558a5a3e1bf0 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.326+0000 7f311d12f640 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a3e1bf0 0x558a5a3e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.327+0000 7f311d12f640 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a3e1bf0 0x558a5a3e4010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365f493ffa07a792 server_cookie=15f293b179b5f1d in_seq=0 out_seq=0 2024-03-12T04:15:34.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.327+0000 7f3100ff9640 10 client.15708 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.327+0000 7f3100ff9640 1 -- 172.21.15.42:0/2165688875 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a5a0bb890 con 0x558a5a3e1bf0 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.327+0000 7f3100ff9640 10 client.15708 handle_command_reply: tid=0 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.327+0000 7f3121f6c740 1 client.15708 shutdown 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 msgr2=0x558a5a3ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3ced10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f31080cb260 tx=0x7f310809adc0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 shutdown_connections 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a3e1bf0 0x558a5a3e4010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3d1750 0x558a5a3cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3ced10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3100ff9640 0 client.15708 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 --2- 172.21.15.42:0/2165688875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 >> 172.21.15.42:0/2165688875 conn(0x558a5a3c2980 msgr2=0x558a5a3c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 shutdown_connections 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 1 -- 172.21.15.42:0/2165688875 wait complete. 2024-03-12T04:15:34.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.328+0000 7f3121f6c740 20 client.15708 trim_cache size 0 max 16384 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.339+0000 7f3121f6c740 1 Processor -- start 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3121f6c740 1 -- start start 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8250 0x558a5a3d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3d1750 0x558a5a3cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3121f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a2b5130 con 0x558a5a3c8b70 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3121f6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a3e9220 con 0x558a5a3c8250 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3121f6c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a3be080 con 0x558a5a3d1750 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49288/0 (socket says 172.21.15.42:49288) 2024-03-12T04:15:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 -- 172.21.15.42:0/1411677144 learned_addr learned my addr 172.21.15.42:0/1411677144 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 -- 172.21.15.42:0/1411677144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3d1750 msgr2=0x558a5a3cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 --2- 172.21.15.42:0/1411677144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3d1750 0x558a5a3cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 -- 172.21.15.42:0/1411677144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8250 msgr2=0x558a5a3d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 --2- 172.21.15.42:0/1411677144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8250 0x558a5a3d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 -- 172.21.15.42:0/1411677144 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a5a2cac20 con 0x558a5a3c8b70 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f311c92e640 1 --2- 172.21.15.42:0/1411677144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3cf070 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108098b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8231b129100781f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3102ffd640 1 -- 172.21.15.42:0/1411677144 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310809e750 con 0x558a5a3c8b70 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3102ffd640 1 -- 172.21.15.42:0/1411677144 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f31080a2da0 con 0x558a5a3c8b70 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.340+0000 7f3102ffd640 1 -- 172.21.15.42:0/1411677144 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31080ba0b0 con 0x558a5a3c8b70 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 -- 172.21.15.42:0/1411677144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 msgr2=0x558a5a3cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1411677144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3cf070 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108098b10 comp rx=0 tx=0).stop 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 -- 172.21.15.42:0/1411677144 shutdown_connections 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1411677144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3d1750 0x558a5a3cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1411677144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3c8b70 0x558a5a3cf070 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1411677144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8250 0x558a5a3d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 -- 172.21.15.42:0/1411677144 >> 172.21.15.42:0/1411677144 conn(0x558a5a3c2980 msgr2=0x558a5a3c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 -- 172.21.15.42:0/1411677144 shutdown_connections 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 -- 172.21.15.42:0/1411677144 wait complete. 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 -- start start 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.341+0000 7f3121f6c740 1 Processor -- start 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8b70 0x558a5a3d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3d1750 0x558a5a3d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a3d7af0 con 0x558a5a3d1750 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a3d7c60 con 0x558a5a3c8b70 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5a438100 con 0x558a5a3c8250 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 10 client.0 _ll_register_callbacks cb 0x7f31140084d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3d1750 0x558a5a3d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3d1750 0x558a5a3d7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49298/0 (socket says 172.21.15.42:49298) 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 -- 172.21.15.42:0/1013352431 learned_addr learned my addr 172.21.15.42:0/1013352431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:34.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d12f640 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 -- 172.21.15.42:0/1013352431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 msgr2=0x558a5a3d8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3d8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 -- 172.21.15.42:0/1013352431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8b70 msgr2=0x558a5a3d6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8b70 0x558a5a3d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 -- 172.21.15.42:0/1013352431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a5a32e6a0 con 0x558a5a3d1750 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d12f640 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3d8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f311d930640 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3d1750 0x558a5a3d7450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f311401c6a0 tx=0x7f311401c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9671d3dbbc7a73b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3100ff9640 1 -- 172.21.15.42:0/1013352431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311402c070 con 0x558a5a3d1750 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3100ff9640 1 -- 172.21.15.42:0/1013352431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3114011830 con 0x558a5a3d1750 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.342+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a5a02f7e0 con 0x558a5a3d1750 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3100ff9640 1 -- 172.21.15.42:0/1013352431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31140125b0 con 0x558a5a3d1750 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 10 client.15714 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:34.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558a5a2f08c0 con 0x558a5a3d1750 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3100ff9640 1 -- 172.21.15.42:0/1013352431 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f311403e020 con 0x558a5a3d1750 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3100ff9640 1 -- 172.21.15.42:0/1013352431 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f311403a760 con 0x558a5a3d1750 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 10 client.15714 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 20 client.15714 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a439920 0x558a5a43bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 4 client.15714 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a59fdb700 con 0x558a5a439920 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.343+0000 7f311d12f640 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a439920 0x558a5a43bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3100ff9640 10 client.15714 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f311d12f640 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a439920 0x558a5a43bd40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828940ac77c748ce server_cookie=23efc44214bd293d in_seq=0 out_seq=0 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3100ff9640 1 -- 172.21.15.42:0/1013352431 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a59fdb700 con 0x558a5a439920 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3100ff9640 10 client.15714 handle_command_reply: tid=0 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 client.15714 shutdown 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3d1750 msgr2=0x558a5a3d7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3d1750 0x558a5a3d7450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f311401c6a0 tx=0x7f311401c6d0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 shutdown_connections 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558a5a439920 0x558a5a43bd40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558a5a3d1750 0x558a5a3d7450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3100ff9640 0 client.15714 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a5a3c8b70 0x558a5a3d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 --2- 172.21.15.42:0/1013352431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a5a3c8250 0x558a5a3d8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 >> 172.21.15.42:0/1013352431 conn(0x558a5a3c2980 msgr2=0x558a5a3c12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 shutdown_connections 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 1 -- 172.21.15.42:0/1013352431 wait complete. 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.344+0000 7f3121f6c740 20 client.15714 trim_cache size 0 max 16384 2024-03-12T04:15:34.352 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:15:34.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f31040a1ce0 msgr2=0x7f31040a41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f31040a1ce0 0x7f31040a41a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f311401f000 tx=0x7f3114020000 comp rx=0 tx=0).stop 2024-03-12T04:15:34.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3118106500 msgr2=0x7f3118100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:34.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3118106500 0x7f3118100da0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f310800c8c0 tx=0x7f310800cd80 comp rx=0 tx=0).stop 2024-03-12T04:15:34.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 shutdown_connections 2024-03-12T04:15:34.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f31040a1ce0 0x7f31040a41a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31181012e0 0x7f31181a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3118106500 0x7f3118100da0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 --2- 172.21.15.42:0/896886576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31180ff8f0 0x7f3118100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:34.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 >> 172.21.15.42:0/896886576 conn(0x7f3118076640 msgr2=0x7f31181048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:34.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 shutdown_connections 2024-03-12T04:15:34.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:34.345+0000 7f311f393640 1 -- 172.21.15.42:0/896886576 wait complete. 2024-03-12T04:15:34.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:34 smithi119 ceph-mon[36119]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:15:34.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:34 smithi136 ceph-mon[35571]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:15:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:34 smithi042 ceph-mon[32260]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-12T04:15:35.357 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:15:35.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:15:35.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 -- 172.21.15.42:0/2814458057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4106450 msgr2=0x7f5fc410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/2814458057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4106450 0x7f5fc410e020 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009cd0 tx=0x7f5fb402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 -- 172.21.15.42:0/2814458057 shutdown_connections 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/2814458057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4106450 0x7f5fc410e020 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/2814458057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc4101d30 0x7f5fc4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/2814458057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc4101380 0x7f5fc4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 -- 172.21.15.42:0/2814458057 >> 172.21.15.42:0/2814458057 conn(0x7f5fc40fcc20 msgr2=0x7f5fc40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 -- 172.21.15.42:0/2814458057 shutdown_connections 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 -- 172.21.15.42:0/2814458057 wait complete. 2024-03-12T04:15:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.476+0000 7f5fc92f4640 1 Processor -- start 2024-03-12T04:15:35.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc92f4640 1 -- start start 2024-03-12T04:15:35.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc92f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4101380 0x7f5fc4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc92f4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc4106450 0x7f5fc4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc92f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc419bfc0 0x7f5fc419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc92f4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc419c9b0 con 0x7f5fc4101380 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc92f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc419cb20 con 0x7f5fc419bfc0 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc92f4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc419cc90 con 0x7f5fc4106450 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4101380 0x7f5fc4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4101380 0x7f5fc4197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49320/0 (socket says 172.21.15.42:49320) 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1352511751 learned_addr learned my addr 172.21.15.42:0/1352511751 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc419bfc0 0x7f5fc419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc4106450 0x7f5fc4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1352511751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc4106450 msgr2=0x7f5fc4197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc4106450 0x7f5fc4197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1352511751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc419bfc0 msgr2=0x7f5fc419c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc419bfc0 0x7f5fc419c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1352511751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4009900 con 0x7f5fc4101380 2024-03-12T04:15:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4101380 0x7f5fc4197750 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00d950 tx=0x7f5fbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.477+0000 7f5fbbfff640 1 -- 172.21.15.42:0/1352511751 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc009c30 con 0x7f5fc4101380 2024-03-12T04:15:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.478+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc41a4270 con 0x7f5fc4101380 2024-03-12T04:15:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.478+0000 7f5fbbfff640 1 -- 172.21.15.42:0/1352511751 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fbc009d90 con 0x7f5fc4101380 2024-03-12T04:15:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.478+0000 7f5fbbfff640 1 -- 172.21.15.42:0/1352511751 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00f6b0 con 0x7f5fc4101380 2024-03-12T04:15:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.478+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc41a4770 con 0x7f5fc4101380 2024-03-12T04:15:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.480+0000 7f5fbbfff640 1 -- 172.21.15.42:0/1352511751 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f5fbc010aa0 con 0x7f5fc4101380 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.480+0000 7f5fcbfa6740 1 Processor -- start 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.480+0000 7f5fcbfa6740 1 -- start start 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594030ef90 con 0x5559403a98b0 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594030c4c0 con 0x5559403a0380 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555940307ca0 con 0x5559403a0d30 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fbbfff640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fb00a1c30 0x7f5fb00a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49332/0 (socket says 172.21.15.42:49332) 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 -- 172.21.15.42:0/3967315781 learned_addr learned my addr 172.21.15.42:0/3967315781 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fb00a1c30 0x7f5fb00a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fbbfff640 1 -- 172.21.15.42:0/1352511751 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5fbc0cd6e0 con 0x7f5fc4101380 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 -- 172.21.15.42:0/3967315781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 msgr2=0x5559403a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 -- 172.21.15.42:0/3967315781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 msgr2=0x5559403a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 -- 172.21.15.42:0/3967315781 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559402ca0b0 con 0x5559403a98b0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403abd00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00dbd0 tx=0x7f5fac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4946cb27686d41ae server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.481+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fb00a1c30 0x7f5fb00a40f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4005cc0 tx=0x7f5fb4032f20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fb97fa640 1 -- 172.21.15.42:0/3967315781 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00dec0 con 0x5559403a98b0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fb97fa640 1 -- 172.21.15.42:0/3967315781 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fac0044e0 con 0x5559403a98b0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fb97fa640 1 -- 172.21.15.42:0/3967315781 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac011440 con 0x5559403a98b0 2024-03-12T04:15:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3967315781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 msgr2=0x5559403abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403abd00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00dbd0 tx=0x7f5fac00f730 comp rx=0 tx=0).stop 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3967315781 shutdown_connections 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403abd00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3967315781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.482+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3967315781 >> 172.21.15.42:0/3967315781 conn(0x55594039a840 msgr2=0x55594039ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3967315781 shutdown_connections 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3967315781 wait complete. 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 -- start start 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 Processor -- start 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403a6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403a7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559402acf40 con 0x5559403a0380 2024-03-12T04:15:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593ffdb4b0 con 0x5559403a98b0 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594028d670 con 0x5559403a0d30 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fcbfa6740 10 client.0 _ll_register_callbacks cb 0x5559402faf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403a6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49348/0 (socket says 172.21.15.42:49348) 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1837605826 learned_addr learned my addr 172.21.15.42:0/1837605826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.483+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1837605826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 msgr2=0x5559403a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 msgr2=0x5559403a7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403a7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/1837605826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac00d800 con 0x5559403a0380 2024-03-12T04:15:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403a6810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc096b60 tx=0x7f5fbc096540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2045e3e518a0e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5faa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5faa7fc640 1 -- 172.21.15.42:0/1837605826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc09a190 con 0x5559403a0380 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5faa7fc640 1 -- 172.21.15.42:0/1837605826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5fbc09e2e0 con 0x5559403a0380 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559402e1f40 con 0x5559403a0380 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5faa7fc640 1 -- 172.21.15.42:0/1837605826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fbc0a2930 con 0x5559403a0380 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 10 client.15726 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555940306320 con 0x5559403a0380 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5faa7fc640 1 -- 172.21.15.42:0/1837605826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5559402e1f40 con 0x5559403a0380 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5faa7fc640 1 -- 172.21.15.42:0/1837605826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fbc020cc0 con 0x5559403a0380 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 20 client.15726 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559403b9ee0 0x5559403bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 4 client.15726 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.484+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559402fe310 con 0x5559403b9ee0 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.485+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559403b9ee0 0x5559403bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.485+0000 7f5faa7fc640 10 client.15726 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.485+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559403b9ee0 0x5559403bc2d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212967a7c603523b server_cookie=d526c548742c8b96 in_seq=0 out_seq=0 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5faa7fc640 1 -- 172.21.15.42:0/1837605826 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559402fe310 con 0x5559403b9ee0 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5faa7fc640 10 client.15726 handle_command_reply: tid=0 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 client.15726 shutdown 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 msgr2=0x5559403a6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403a6810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc096b60 tx=0x7f5fbc096540 comp rx=0 tx=0).stop 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 shutdown_connections 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559403b9ee0 0x5559403bc2d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/1837605826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403a6810 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5faa7fc640 0 client.15726 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 >> 172.21.15.42:0/1837605826 conn(0x55594039a840 msgr2=0x55594039c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 shutdown_connections 2024-03-12T04:15:35.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/1837605826 wait complete. 2024-03-12T04:15:35.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.486+0000 7f5fcbfa6740 20 client.15726 trim_cache size 0 max 16384 2024-03-12T04:15:35.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 Processor -- start 2024-03-12T04:15:35.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 -- start start 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559403962c0 con 0x5559403a98b0 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559403b57f0 con 0x5559403a0380 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594039fd40 con 0x5559403a0d30 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403a72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49364/0 (socket says 172.21.15.42:49364) 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 -- 172.21.15.42:0/3529873981 learned_addr learned my addr 172.21.15.42:0/3529873981 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 -- 172.21.15.42:0/3529873981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 msgr2=0x5559403a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 -- 172.21.15.42:0/3529873981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 msgr2=0x5559403a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.502+0000 7f5fc37fe640 1 -- 172.21.15.42:0/3529873981 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559402a2c00 con 0x5559403a98b0 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403a72a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009cd0 tx=0x7f5fb4046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ec6a8b7cdffb2a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fb8ff9640 1 -- 172.21.15.42:0/3529873981 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4034a60 con 0x5559403a98b0 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fb8ff9640 1 -- 172.21.15.42:0/3529873981 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fb4032a80 con 0x5559403a98b0 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fb8ff9640 1 -- 172.21.15.42:0/3529873981 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4004040 con 0x5559403a98b0 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3529873981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 msgr2=0x5559403a72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403a72a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009cd0 tx=0x7f5fb4046000 comp rx=0 tx=0).stop 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3529873981 shutdown_connections 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a98b0 0x5559403a72a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/3529873981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a0380 0x5559403a8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3529873981 >> 172.21.15.42:0/3529873981 conn(0x55594039a840 msgr2=0x5559403a50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3529873981 shutdown_connections 2024-03-12T04:15:35.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.503+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/3529873981 wait complete. 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 -- start start 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 Processor -- start 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559404104d0 con 0x5559403a0380 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555940410640 con 0x5559403a98b0 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555940410800 con 0x5559403a0d30 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fcbfa6740 10 client.0 _ll_register_callbacks cb 0x7f5fac00d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49368/0 (socket says 172.21.15.42:49368) 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/831698443 learned_addr learned my addr 172.21.15.42:0/831698443 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc37fe640 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/831698443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 msgr2=0x5559403afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 msgr2=0x5559403aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.504+0000 7f5fc2ffd640 1 -- 172.21.15.42:0/831698443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb4048040 con 0x5559403a0380 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fc2ffd640 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403af4e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0a2400 tx=0x7f5fbc0b2f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f143e472fb8aabdb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5faa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5faa7fc640 1 -- 172.21.15.42:0/831698443 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc099f60 con 0x5559403a0380 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5faa7fc640 1 -- 172.21.15.42:0/831698443 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5fbc09a190 con 0x5559403a0380 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55594032d900 con 0x5559403a0380 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5faa7fc640 1 -- 172.21.15.42:0/831698443 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fbc09e2e0 con 0x5559403a0380 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 10 client.15732 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5559402c8910 con 0x5559403a0380 2024-03-12T04:15:35.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5faa7fc640 1 -- 172.21.15.42:0/831698443 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55594032d900 con 0x5559403a0380 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5faa7fc640 1 -- 172.21.15.42:0/831698443 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fbc099050 con 0x5559403a0380 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 20 client.15732 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555940411c90 0x5559404140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 4 client.15732 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.505+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555940007860 con 0x555940411c90 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.506+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555940411c90 0x5559404140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.506+0000 7f5faa7fc640 10 client.15732 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.506+0000 7f5fc27fc640 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555940411c90 0x5559404140b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd44deba3406599 server_cookie=75e2c1ed1cca16ac in_seq=0 out_seq=0 2024-03-12T04:15:35.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5faa7fc640 1 -- 172.21.15.42:0/831698443 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555940007860 con 0x555940411c90 2024-03-12T04:15:35.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5faa7fc640 10 client.15732 handle_command_reply: tid=0 2024-03-12T04:15:35.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 client.15732 shutdown 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 msgr2=0x5559403af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403af4e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0a2400 tx=0x7f5fbc0b2f50 comp rx=0 tx=0).stop 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 shutdown_connections 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555940411c90 0x5559404140b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559403a98b0 0x5559403aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559403a0d30 0x5559403afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 --2- 172.21.15.42:0/831698443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559403a0380 0x5559403af4e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 >> 172.21.15.42:0/831698443 conn(0x55594039a840 msgr2=0x55594039be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5faa7fc640 0 client.15732 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 shutdown_connections 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 1 -- 172.21.15.42:0/831698443 wait complete. 2024-03-12T04:15:35.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.507+0000 7f5fcbfa6740 20 client.15732 trim_cache size 0 max 16384 2024-03-12T04:15:35.515 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:15:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:15:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "b0e74c05-dab1-4581-85e1-9098863d9d53", 2024-03-12T04:15:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:15:35.515 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fb00a1c30 msgr2=0x7f5fb00a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fb00a1c30 0x7f5fb00a40f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4005cc0 tx=0x7f5fb4032f20 comp rx=0 tx=0).stop 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4101380 msgr2=0x7f5fc4197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4101380 0x7f5fc4197750 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00d950 tx=0x7f5fbc00de10 comp rx=0 tx=0).stop 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 shutdown_connections 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fb00a1c30 0x7f5fb00a40f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc419bfc0 0x7f5fc419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc4106450 0x7f5fc4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 --2- 172.21.15.42:0/1352511751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fc4101380 0x7f5fc4197750 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 >> 172.21.15.42:0/1352511751 conn(0x7f5fc40fcc20 msgr2=0x7f5fc40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 shutdown_connections 2024-03-12T04:15:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.508+0000 7f5fc92f4640 1 -- 172.21.15.42:0/1352511751 wait complete. 2024-03-12T04:15:35.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:15:35.521 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:15:35.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.624+0000 7fb5ae58a640 1 -- 172.21.15.42:0/126617171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8100270 msgr2=0x7fb5a8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.624+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/126617171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8100270 0x7fb5a8100650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb598009930 tx=0x7fb59802bc30 comp rx=0 tx=0).stop 2024-03-12T04:15:35.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- 172.21.15.42:0/126617171 shutdown_connections 2024-03-12T04:15:35.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/126617171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a8105340 0x7fb5a810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/126617171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a8100c20 0x7fb5a8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/126617171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8100270 0x7fb5a8100650 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- 172.21.15.42:0/126617171 >> 172.21.15.42:0/126617171 conn(0x7fb5a8077230 msgr2=0x7fb5a8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- 172.21.15.42:0/126617171 shutdown_connections 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- 172.21.15.42:0/126617171 wait complete. 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 Processor -- start 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- start start 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a8100270 0x7fb5a8101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8105340 0x7fb5a8102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a8102780 0x7fb5a81a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a81a5f00 con 0x7fb5a8105340 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a81a6070 con 0x7fb5a8102780 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.625+0000 7fb5ae58a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a81a6230 con 0x7fb5a8100270 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8105340 0x7fb5a8102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8105340 0x7fb5a8102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49382/0 (socket says 172.21.15.42:49382) 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 -- 172.21.15.42:0/1000747629 learned_addr learned my addr 172.21.15.42:0/1000747629 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5acb27640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a8102780 0x7fb5a81a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a8100270 0x7fb5a8101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 -- 172.21.15.42:0/1000747629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a8100270 msgr2=0x7fb5a8101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a8100270 0x7fb5a8101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 -- 172.21.15.42:0/1000747629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a8102780 msgr2=0x7fb5a81a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a8102780 0x7fb5a81a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 -- 172.21.15.42:0/1000747629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb598009560 con 0x7fb5a8105340 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5acb27640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a8102780 0x7fb5a81a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8105340 0x7fb5a8102240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c970 tx=0x7fb59c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a57fa640 1 -- 172.21.15.42:0/1000747629 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c007bb0 con 0x7fb5a8105340 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a81a6510 con 0x7fb5a8105340 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a57fa640 1 -- 172.21.15.42:0/1000747629 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59c007d10 con 0x7fb5a8105340 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5a57fa640 1 -- 172.21.15.42:0/1000747629 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c018660 con 0x7fb5a8105340 2024-03-12T04:15:35.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.626+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a81a69d0 con 0x7fb5a8105340 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.628+0000 7fb5a57fa640 1 -- 172.21.15.42:0/1000747629 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136455+0+0 (secure 0 0 0) 0x7fb59c00f450 con 0x7fb5a8105340 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a57fa640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb5900a1c60 0x7fb5900a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 Processor -- start 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 -- start start 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb5900a1c60 0x7fb5900a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a57fa640 1 -- 172.21.15.42:0/1000747629 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb59c0cda50 con 0x7fb5a8105340 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be6138d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61396270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be61398ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be612fc280 con 0x55be6138d2b0 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be612f97b0 con 0x55be6138dbd0 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5b1233740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be612f4f90 con 0x55be613967b0 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be6138d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be6138d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49390/0 (socket says 172.21.15.42:49390) 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3785452140 learned_addr learned my addr 172.21.15.42:0/3785452140 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61396270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5acb27640 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be61398ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.629+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb5900a1c60 0x7fb5900a4120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb598005ab0 tx=0x7fb5980048c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3785452140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 msgr2=0x55be61398ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be61398ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3785452140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 msgr2=0x55be61396270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61396270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3785452140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be612b3570 con 0x55be6138d2b0 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be6138d690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c710 tx=0x7fb598047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce02997ee14fbf84 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb58e7fc640 1 -- 172.21.15.42:0/3785452140 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598004200 con 0x55be6138d2b0 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb58e7fc640 1 -- 172.21.15.42:0/3785452140 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59802de90 con 0x55be6138d2b0 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.630+0000 7fb58e7fc640 1 -- 172.21.15.42:0/3785452140 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598003970 con 0x55be6138d2b0 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- 172.21.15.42:0/3785452140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 msgr2=0x55be6138d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be6138d690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c710 tx=0x7fb598047000 comp rx=0 tx=0).stop 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- 172.21.15.42:0/3785452140 shutdown_connections 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be61398ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61396270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3785452140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be6138d690 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- 172.21.15.42:0/3785452140 >> 172.21.15.42:0/3785452140 conn(0x55be61387990 msgr2=0x55be61387d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- 172.21.15.42:0/3785452140 shutdown_connections 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- 172.21.15.42:0/3785452140 wait complete. 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- start start 2024-03-12T04:15:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 Processor -- start 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be61393620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61393b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be613940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be60fc8430 con 0x55be6138d2b0 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be60f84e00 con 0x55be6138dbd0 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.631+0000 7fb5b1233740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be61279fd0 con 0x55be613967b0 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5b1233740 10 client.0 _ll_register_callbacks cb 0x55be612cffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be61393620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be61393620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49398/0 (socket says 172.21.15.42:49398) 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3884567865 learned_addr learned my addr 172.21.15.42:0/3884567865 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61393b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5acb27640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be613940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3884567865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 msgr2=0x55be613940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be613940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 msgr2=0x55be61393b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61393b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 -- 172.21.15.42:0/3884567865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a810d720 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61393b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be61393620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c170 tx=0x7fb598003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364ca1ad68e33fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5877fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5877fe640 1 -- 172.21.15.42:0/3884567865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598004dc0 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5877fe640 1 -- 172.21.15.42:0/3884567865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb598004f20 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be60ff47e0 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5877fe640 1 -- 172.21.15.42:0/3884567865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5980050e0 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5b1233740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.632+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55be612f3610 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5877fe640 1 -- 172.21.15.42:0/3884567865 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55be60ff47e0 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5877fe640 1 -- 172.21.15.42:0/3884567865 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb598041a80 con 0x55be6138d2b0 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5b1233740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:35.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5b1233740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5b1233740 20 client.15744 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613a66f0 0x55be613a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5b1233740 4 client.15744 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be61031720 con 0x55be613a66f0 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613a66f0 0x55be613a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5877fe640 10 client.15744 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.633+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613a66f0 0x55be613a8ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ec7e8e4520f0dd server_cookie=12cb08d8d437eea8 in_seq=0 out_seq=0 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5877fe640 1 -- 172.21.15.42:0/3884567865 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be61031720 con 0x55be613a66f0 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5877fe640 10 client.15744 handle_command_reply: tid=0 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 client.15744 shutdown 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 msgr2=0x55be61393620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be61393620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c170 tx=0x7fb598003920 comp rx=0 tx=0).stop 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 shutdown_connections 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613a66f0 0x55be613a8ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be613967b0 0x55be613940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be6138dbd0 0x55be61393b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5877fe640 0 client.15744 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 --2- 172.21.15.42:0/3884567865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be61393620 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 >> 172.21.15.42:0/3884567865 conn(0x55be61387990 msgr2=0x55be61389730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 shutdown_connections 2024-03-12T04:15:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 1 -- 172.21.15.42:0/3884567865 wait complete. 2024-03-12T04:15:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.634+0000 7fb5b1233740 20 client.15744 trim_cache size 0 max 16384 2024-03-12T04:15:35.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.649+0000 7fb5b1233740 1 Processor -- start 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.649+0000 7fb5b1233740 1 -- start start 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.649+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138d2b0 0x55be613894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.649+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138dbd0 0x55be6139d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be6139db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5b1233740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be613add00 con 0x55be6138dbd0 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5b1233740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be61398f20 con 0x55be613967b0 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5b1233740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be613965d0 con 0x55be6138d2b0 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138dbd0 0x55be6139d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138dbd0 0x55be6139d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49410/0 (socket says 172.21.15.42:49410) 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 -- 172.21.15.42:0/2050806891 learned_addr learned my addr 172.21.15.42:0/2050806891 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138d2b0 0x55be613894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 -- 172.21.15.42:0/2050806891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138d2b0 msgr2=0x55be613894d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138d2b0 0x55be613894d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 -- 172.21.15.42:0/2050806891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 msgr2=0x55be6139db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be6139db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 -- 172.21.15.42:0/2050806891 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6128fac0 con 0x55be6138dbd0 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138d2b0 0x55be613894d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138dbd0 0x55be6139d600 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0cb7f0 tx=0x7fb59c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8762c902af94b9d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb58dffb640 1 -- 172.21.15.42:0/2050806891 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c09e6b0 con 0x55be6138dbd0 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb58dffb640 1 -- 172.21.15.42:0/2050806891 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59c0a2d00 con 0x55be6138dbd0 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.650+0000 7fb58dffb640 1 -- 172.21.15.42:0/2050806891 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0ba1d0 con 0x55be6138dbd0 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 -- 172.21.15.42:0/2050806891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138dbd0 msgr2=0x55be6139d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138dbd0 0x55be6139d600 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0cb7f0 tx=0x7fb59c00af90 comp rx=0 tx=0).stop 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 -- 172.21.15.42:0/2050806891 shutdown_connections 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be6139db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138dbd0 0x55be6139d600 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 --2- 172.21.15.42:0/2050806891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138d2b0 0x55be613894d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 -- 172.21.15.42:0/2050806891 >> 172.21.15.42:0/2050806891 conn(0x55be61387990 msgr2=0x55be61388dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 -- 172.21.15.42:0/2050806891 shutdown_connections 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 -- 172.21.15.42:0/2050806891 wait complete. 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 -- start start 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 Processor -- start 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be613f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.651+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138dbd0 0x55be613f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be613f0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be613f10f0 con 0x55be6138d2b0 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be613f1260 con 0x55be613967b0 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be613f1420 con 0x55be6138dbd0 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 10 client.0 _ll_register_callbacks cb 0x7fb5980046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be613f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be613f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49418/0 (socket says 172.21.15.42:49418) 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 -- 172.21.15.42:0/643750220 learned_addr learned my addr 172.21.15.42:0/643750220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:15:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138dbd0 0x55be613f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5acb27640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be613f0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 -- 172.21.15.42:0/643750220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138dbd0 msgr2=0x55be613f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138dbd0 0x55be613f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 -- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 msgr2=0x55be613f0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be613f0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 -- 172.21.15.42:0/643750220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a81a69d0 con 0x55be6138d2b0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5acb27640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be613f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5a7fff640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be613f6630 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb598000c00 tx=0x7fb59802c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19879518427cfe6c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5877fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5877fe640 1 -- 172.21.15.42:0/643750220 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980349c0 con 0x55be6138d2b0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5877fe640 1 -- 172.21.15.42:0/643750220 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb598034b20 con 0x55be6138d2b0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb59802de90 con 0x55be6138d2b0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5877fe640 1 -- 172.21.15.42:0/643750220 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb598003990 con 0x55be6138d2b0 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 10 client.15750 fetch_fsmap learned FSMap version 27 2024-03-12T04:15:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.652+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55be612b56a0 con 0x55be6138d2b0 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5877fe640 1 -- 172.21.15.42:0/643750220 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59806f080 con 0x55be6138d2b0 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5877fe640 1 -- 172.21.15.42:0/643750220 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb598004dc0 con 0x55be6138d2b0 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5b1233740 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5b1233740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5b1233740 20 client.15750 populate_metadata read hostname 'smithi042' 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5b1233740 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613fe040 0x55be61400460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5b1233740 4 client.15750 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be61080890 con 0x55be613fe040 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613fe040 0x55be61400460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5a77fe640 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613fe040 0x55be61400460 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5792455ba9c78105 server_cookie=c9ba60a1af55f298 in_seq=0 out_seq=0 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5877fe640 10 client.15750 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5877fe640 1 -- 172.21.15.42:0/643750220 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55be61080890 con 0x55be613fe040 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5877fe640 10 client.15750 handle_command_reply: tid=0 2024-03-12T04:15:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.653+0000 7fb5b1233740 1 client.15750 shutdown 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 msgr2=0x55be613f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be613f6630 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb598000c00 tx=0x7fb59802c330 comp rx=0 tx=0).stop 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 shutdown_connections 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55be613fe040 0x55be61400460 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be613967b0 0x55be613f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be6138dbd0 0x55be613f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5877fe640 0 client.15750 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 --2- 172.21.15.42:0/643750220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55be6138d2b0 0x55be613f6630 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 >> 172.21.15.42:0/643750220 conn(0x55be61387990 msgr2=0x55be61386a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 shutdown_connections 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 1 -- 172.21.15.42:0/643750220 wait complete. 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5b1233740 20 client.15750 trim_cache size 0 max 16384 2024-03-12T04:15:35.662 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: "b0e74c05-dab1-4581-85e1-9098863d9d53": { 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "b0e74c05-dab1-4581-85e1-9098863d9d53", 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:15:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb5900a1c60 msgr2=0x7fb5900a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb5900a1c60 0x7fb5900a4120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb598005ab0 tx=0x7fb5980048c0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8105340 msgr2=0x7fb5a8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8105340 0x7fb5a8102240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c970 tx=0x7fb59c00ce30 comp rx=0 tx=0).stop 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 shutdown_connections 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb5900a1c60 0x7fb5900a4120 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a8102780 0x7fb5a81a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5a8105340 0x7fb5a8102240 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 --2- 172.21.15.42:0/1000747629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a8100270 0x7fb5a8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 >> 172.21.15.42:0/1000747629 conn(0x7fb5a8077230 msgr2=0x7fb5a8068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:15:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 shutdown_connections 2024-03-12T04:15:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:15:35.654+0000 7fb5ae58a640 1 -- 172.21.15.42:0/1000747629 wait complete. 2024-03-12T04:15:35.666 INFO:tasks.cephfs.filesystem:scrub status for tag:b0e74c05-dab1-4581-85e1-9098863d9d53 - {'path': '/', 'tag': 'b0e74c05-dab1-4581-85e1-9098863d9d53', 'options': 'recursive,force'} 2024-03-12T04:15:36.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:36 smithi119 ceph-mon[36119]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-12T04:15:36.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:36 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:15:36.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:36 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:36.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:36 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:15:36.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:36 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:36 smithi136 ceph-mon[35571]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-12T04:15:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:36 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:15:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:36 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:36 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:15:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:36 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:36 smithi042 ceph-mon[32260]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-12T04:15:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:36 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:15:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:36 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:36 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:15:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:36 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:15:38.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:38 smithi119 ceph-mon[36119]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-12T04:15:38.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:38 smithi136 ceph-mon[35571]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-12T04:15:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:38 smithi042 ceph-mon[32260]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-12T04:15:39.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:39 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:15:39.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:39 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:15:40.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:39 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:15:40.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:40 smithi136 ceph-mon[35571]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-12T04:15:40.967 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:40 smithi042 ceph-mon[32260]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-12T04:15:41.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:40 smithi119 ceph-mon[36119]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-12T04:15:42.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:42 smithi136 ceph-mon[35571]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:15:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:42 smithi042 ceph-mon[32260]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:15:43.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:42 smithi119 ceph-mon[36119]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:15:44.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:44 smithi136 ceph-mon[35571]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:15:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:44 smithi042 ceph-mon[32260]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:15:45.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:44 smithi119 ceph-mon[36119]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:15:46.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:46 smithi136 ceph-mon[35571]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:15:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:46 smithi042 ceph-mon[32260]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:15:47.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:46 smithi119 ceph-mon[36119]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:15:48.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:48 smithi136 ceph-mon[35571]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-12T04:15:49.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:48 smithi042 ceph-mon[32260]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-12T04:15:49.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:48 smithi119 ceph-mon[36119]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-12T04:15:51.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:50 smithi042 ceph-mon[32260]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-12T04:15:51.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:50 smithi119 ceph-mon[36119]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-12T04:15:51.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:50 smithi136 ceph-mon[35571]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-12T04:15:53.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:52 smithi042 ceph-mon[32260]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:15:53.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:52 smithi119 ceph-mon[36119]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:15:53.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:52 smithi136 ceph-mon[35571]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:15:55.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:54 smithi042 ceph-mon[32260]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:15:55.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:54 smithi119 ceph-mon[36119]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:15:55.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:54 smithi136 ceph-mon[35571]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:15:57.009 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:56 smithi136 ceph-mon[35571]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:15:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:56 smithi042 ceph-mon[32260]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:15:57.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:56 smithi119 ceph-mon[36119]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:15:58.998 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:15:58 smithi119 ceph-mon[36119]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:15:59.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:15:58 smithi042 ceph-mon[32260]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:15:59.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:15:58 smithi136 ceph-mon[35571]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:16:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:00 smithi042 ceph-mon[32260]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-12T04:16:01.066 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:16:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:16:00.655+0000 7ff7366ed700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:16:01.066 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:16:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:16:00.655+0000 7ff7366ed700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:16:01.066 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:16:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:16:00.655+0000 7ff7366ed700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:16:01.067 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:16:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:16:00.655+0000 7ff7366ed700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:16:01.067 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:16:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:16:00.655+0000 7ff7366ed700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T04:16:01.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:00 smithi119 ceph-mon[36119]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-12T04:16:01.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:00 smithi136 ceph-mon[35571]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-12T04:16:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:02 smithi042 ceph-mon[32260]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:16:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:02 smithi042 ceph-mon[32260]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-03-12T04:16:03.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:02 smithi119 ceph-mon[36119]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:16:03.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:02 smithi119 ceph-mon[36119]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-03-12T04:16:03.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:02 smithi136 ceph-mon[35571]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:16:03.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:02 smithi136 ceph-mon[35571]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-03-12T04:16:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:04 smithi042 ceph-mon[32260]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:05.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:04 smithi119 ceph-mon[36119]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:05.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:04 smithi136 ceph-mon[35571]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:05.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:16:05.668 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:16:05.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.777+0000 7f1b14b65640 1 -- 172.21.15.42:0/147780426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10101370 msgr2=0x7f1b10101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.777+0000 7f1b14b65640 1 --2- 172.21.15.42:0/147780426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10101370 0x7f1b10101750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009930 tx=0x7f1b0002bc20 comp rx=0 tx=0).stop 2024-03-12T04:16:05.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.777+0000 7f1b14b65640 1 -- 172.21.15.42:0/147780426 shutdown_connections 2024-03-12T04:16:05.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.777+0000 7f1b14b65640 1 --2- 172.21.15.42:0/147780426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b10106440 0x7f1b1010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.777+0000 7f1b14b65640 1 --2- 172.21.15.42:0/147780426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b10101d20 0x7f1b10105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.777+0000 7f1b14b65640 1 --2- 172.21.15.42:0/147780426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10101370 0x7f1b10101750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.777+0000 7f1b14b65640 1 -- 172.21.15.42:0/147780426 >> 172.21.15.42:0/147780426 conn(0x7f1b100fcc30 msgr2=0x7f1b100ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:05.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.778+0000 7f1b14b65640 1 -- 172.21.15.42:0/147780426 shutdown_connections 2024-03-12T04:16:05.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.778+0000 7f1b14b65640 1 -- 172.21.15.42:0/147780426 wait complete. 2024-03-12T04:16:05.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 Processor -- start 2024-03-12T04:16:05.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 -- start start 2024-03-12T04:16:05.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b10101370 0x7f1b10197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10106440 0x7f1b10197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1019bee0 0x7f1b1019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b0e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b10101370 0x7f1b10197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1019c8d0 con 0x7f1b10101370 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1019ca40 con 0x7f1b10106440 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b0dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10106440 0x7f1b10197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b0dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10106440 0x7f1b10197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:37890/0 (socket says 172.21.15.42:37890) 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/1243350257 learned_addr learned my addr 172.21.15.42:0/1243350257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1019bee0 0x7f1b1019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.779+0000 7f1b14b65640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1019cbb0 con 0x7f1b1019bee0 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.780+0000 7f1b0ed9d640 1 -- 172.21.15.42:0/1243350257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10106440 msgr2=0x7f1b10197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.780+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10106440 0x7f1b10197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.780+0000 7f1b0ed9d640 1 -- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b10101370 msgr2=0x7f1b10197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.780+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b10101370 0x7f1b10197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.780+0000 7f1b0ed9d640 1 -- 172.21.15.42:0/1243350257 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00009560 con 0x7f1b1019bee0 2024-03-12T04:16:05.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.780+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1019bee0 0x7f1b1019c390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800cab0 tx=0x7f1b0800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.780+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b10101370 0x7f1b10197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:05.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.781+0000 7f1aff7fe640 1 -- 172.21.15.42:0/1243350257 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800ae20 con 0x7f1b1019bee0 2024-03-12T04:16:05.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.781+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b101a4190 con 0x7f1b1019bee0 2024-03-12T04:16:05.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.781+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b101a4640 con 0x7f1b1019bee0 2024-03-12T04:16:05.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.781+0000 7f1aff7fe640 1 -- 172.21.15.42:0/1243350257 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b0800f900 con 0x7f1b1019bee0 2024-03-12T04:16:05.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.781+0000 7f1aff7fe640 1 -- 172.21.15.42:0/1243350257 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b080189b0 con 0x7f1b1019bee0 2024-03-12T04:16:05.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.784+0000 7f1b17814740 1 Processor -- start 2024-03-12T04:16:05.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.784+0000 7f1aff7fe640 1 -- 172.21.15.42:0/1243350257 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f1b080213f0 con 0x7f1b1019bee0 2024-03-12T04:16:05.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.784+0000 7f1b17814740 1 -- start start 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b17814740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980163c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b17814740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163cbd0 0x559801645270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b17814740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801647ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b17814740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598015ab280 con 0x55980163c2b0 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b17814740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598015a87b0 con 0x55980163cbd0 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b17814740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598015a3f90 con 0x5598016457b0 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980163c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980163c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45970/0 (socket says 172.21.15.42:45970) 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/759860025 learned_addr learned my addr 172.21.15.42:0/759860025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163cbd0 0x559801645270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801647ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/759860025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 msgr2=0x559801647ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801647ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/759860025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163cbd0 msgr2=0x559801645270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163cbd0 0x559801645270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/759860025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559801562570 con 0x55980163c2b0 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.785+0000 7f1aff7fe640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1af80a1d40 0x7f1af80a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.786+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980163c690 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1b04013640 tx=0x7f1b0400a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da4f54d7b35f0f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.786+0000 7f1afcff9640 1 -- 172.21.15.42:0/759860025 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04019070 con 0x55980163c2b0 2024-03-12T04:16:05.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.786+0000 7f1aff7fe640 1 -- 172.21.15.42:0/1243350257 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1b080ce080 con 0x7f1b1019bee0 2024-03-12T04:16:05.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.786+0000 7f1afcff9640 1 -- 172.21.15.42:0/759860025 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b040044e0 con 0x55980163c2b0 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.786+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1af80a1d40 0x7f1af80a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.786+0000 7f1afcff9640 1 -- 172.21.15.42:0/759860025 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04011ea0 con 0x55980163c2b0 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 -- 172.21.15.42:0/759860025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 msgr2=0x55980163c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1af80a1d40 0x7f1af80a4200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009900 tx=0x7f1b00005a20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980163c690 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1b04013640 tx=0x7f1b0400a870 comp rx=0 tx=0).stop 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 -- 172.21.15.42:0/759860025 shutdown_connections 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801647ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163cbd0 0x559801645270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 --2- 172.21.15.42:0/759860025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980163c690 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 -- 172.21.15.42:0/759860025 >> 172.21.15.42:0/759860025 conn(0x559801636990 msgr2=0x559801636d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 -- 172.21.15.42:0/759860025 shutdown_connections 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 -- 172.21.15.42:0/759860025 wait complete. 2024-03-12T04:16:05.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 -- start start 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.787+0000 7f1b17814740 1 Processor -- start 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b17814740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x5598016450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b17814740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980163cbd0 0x559801642470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b17814740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5598016457b0 0x5598016429b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b17814740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801277430 con 0x5598016457b0 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b17814740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801233e00 con 0x55980163c2b0 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b17814740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801528fd0 con 0x55980163cbd0 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5598016457b0 0x5598016429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5598016457b0 0x5598016429b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45986/0 (socket says 172.21.15.42:45986) 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0ed9d640 1 -- 172.21.15.42:0/3654141995 learned_addr learned my addr 172.21.15.42:0/3654141995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x5598016450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980163cbd0 0x559801642470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b17814740 10 client.0 _ll_register_callbacks cb 0x55980157efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0ed9d640 1 -- 172.21.15.42:0/3654141995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980163cbd0 msgr2=0x559801642470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980163cbd0 0x559801642470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.788+0000 7f1b0ed9d640 1 -- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 msgr2=0x5598016450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x5598016450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1b0ed9d640 1 -- 172.21.15.42:0/3654141995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0400ae30 con 0x5598016457b0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598012a37e0 con 0x5598016457b0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5598016457b0 0x5598016429b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1b08097520 tx=0x7f1b0809aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5071b7fef20805d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1af5ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1af5ffb640 1 -- 172.21.15.42:0/3654141995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800fc80 con 0x5598016457b0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1af5ffb640 1 -- 172.21.15.42:0/3654141995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1b0809eca0 con 0x5598016457b0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1af5ffb640 1 -- 172.21.15.42:0/3654141995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b080a32f0 con 0x5598016457b0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1b17814740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.789+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5598015a2610 con 0x5598016457b0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1af5ffb640 1 -- 172.21.15.42:0/3654141995 <== mon.0 v2:172.21.15.42:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1b08016020 con 0x5598016457b0 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1b17814740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:05.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1b17814740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1b17814740 20 client.15765 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1b17814740 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559801655640 0x559801657ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1b17814740 4 client.15765 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598012e0720 con 0x559801655640 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.790+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559801655640 0x559801657ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.791+0000 7f1af5ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.791+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559801655640 0x559801657ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8d1aeff3934d73 server_cookie=22a5272206c45caa in_seq=0 out_seq=0 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.791+0000 7f1af5ffb640 1 -- 172.21.15.42:0/3654141995 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598012e0720 con 0x559801655640 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.791+0000 7f1af5ffb640 10 client.15765 handle_command_reply: tid=0 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.791+0000 7f1b17814740 1 client.15765 shutdown 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5598016457b0 msgr2=0x5598016429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5598016457b0 0x5598016429b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1b08097520 tx=0x7f1b0809aa30 comp rx=0 tx=0).stop 2024-03-12T04:16:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 shutdown_connections 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559801655640 0x559801657ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5598016457b0 0x5598016429b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980163cbd0 0x559801642470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 --2- 172.21.15.42:0/3654141995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x5598016450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 >> 172.21.15.42:0/3654141995 conn(0x559801636990 msgr2=0x559801638730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1af5ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 shutdown_connections 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 1 -- 172.21.15.42:0/3654141995 wait complete. 2024-03-12T04:16:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.792+0000 7f1b17814740 20 client.15765 trim_cache size 0 max 16384 2024-03-12T04:16:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.822+0000 7f1b17814740 1 Processor -- start 2024-03-12T04:16:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.822+0000 7f1b17814740 1 -- start start 2024-03-12T04:16:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b17814740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x559801644ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b17814740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163cbd0 0x5598016450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b17814740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801640d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b17814740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801651d70 con 0x55980163cbd0 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b17814740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801648480 con 0x55980163c2b0 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b17814740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016379e0 con 0x5598016457b0 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163cbd0 0x5598016450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163cbd0 0x5598016450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46002/0 (socket says 172.21.15.42:46002) 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/834708300 learned_addr learned my addr 172.21.15.42:0/834708300 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x559801644ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801640d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/834708300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 msgr2=0x559801640d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801640d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/834708300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 msgr2=0x559801644ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x559801644ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 -- 172.21.15.42:0/834708300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980153eac0 con 0x55980163cbd0 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x559801644ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:05.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.823+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163cbd0 0x5598016450e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1b04013640 tx=0x7f1b04005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aa57e89639beaf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.824+0000 7f1af7fff640 1 -- 172.21.15.42:0/834708300 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04019040 con 0x55980163cbd0 2024-03-12T04:16:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.824+0000 7f1af7fff640 1 -- 172.21.15.42:0/834708300 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b04011830 con 0x55980163cbd0 2024-03-12T04:16:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.824+0000 7f1af7fff640 1 -- 172.21.15.42:0/834708300 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b040048b0 con 0x55980163cbd0 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 -- 172.21.15.42:0/834708300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163cbd0 msgr2=0x5598016450e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163cbd0 0x5598016450e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1b04013640 tx=0x7f1b04005850 comp rx=0 tx=0).stop 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 -- 172.21.15.42:0/834708300 shutdown_connections 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598016457b0 0x559801640d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163cbd0 0x5598016450e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 --2- 172.21.15.42:0/834708300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980163c2b0 0x559801644ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 -- 172.21.15.42:0/834708300 >> 172.21.15.42:0/834708300 conn(0x559801636990 msgr2=0x55980163d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 -- 172.21.15.42:0/834708300 shutdown_connections 2024-03-12T04:16:05.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 -- 172.21.15.42:0/834708300 wait complete. 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 -- start start 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.825+0000 7f1b17814740 1 Processor -- start 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b17814740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980164c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b17814740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598016457b0 0x559801649bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b17814740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980164a110 0x55980164a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980164c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b17814740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980164cfb0 con 0x55980163c2b0 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b17814740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016acc10 con 0x5598016457b0 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980164c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46010/0 (socket says 172.21.15.42:46010) 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 -- 172.21.15.42:0/964559057 learned_addr learned my addr 172.21.15.42:0/964559057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b17814740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016acd80 con 0x55980164a110 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980164a110 0x55980164a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598016457b0 0x559801649bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b17814740 10 client.0 _ll_register_callbacks cb 0x7f1b0400a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 -- 172.21.15.42:0/964559057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980164a110 msgr2=0x55980164a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980164a110 0x55980164a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 -- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598016457b0 msgr2=0x559801649bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598016457b0 0x559801649bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0e59c640 1 -- 172.21.15.42:0/964559057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559801562570 con 0x55980163c2b0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.826+0000 7f1b0ed9d640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980164a110 0x55980164a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598016457b0 0x559801649bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b040044e0 con 0x55980163c2b0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1b0e59c640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980164c8e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000c00 tx=0x7f1b00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173db00fc8ed6e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1af5ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1af5ffb640 1 -- 172.21.15.42:0/964559057 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00053040 con 0x55980163c2b0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1af5ffb640 1 -- 172.21.15.42:0/964559057 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1b000594e0 con 0x55980163c2b0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1af5ffb640 1 -- 172.21.15.42:0/964559057 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b00045070 con 0x55980163c2b0 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1b17814740 10 client.15771 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.827+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5598015646a0 con 0x55980163c2b0 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1af5ffb640 1 -- 172.21.15.42:0/964559057 <== mon.0 v2:172.21.15.42:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1b00061070 con 0x55980163c2b0 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1b17814740 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1b17814740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1b17814740 20 client.15771 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1b17814740 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5598016ae370 0x5598016b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1b17814740 4 client.15771 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55980132f890 con 0x5598016ae370 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.828+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5598016ae370 0x5598016b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b0dd9b640 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5598016ae370 0x5598016b0790 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f68a69e61851040 server_cookie=43001672e9845feb in_seq=0 out_seq=0 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1af5ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1af5ffb640 1 -- 172.21.15.42:0/964559057 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55980132f890 con 0x5598016ae370 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1af5ffb640 10 client.15771 handle_command_reply: tid=0 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 client.15771 shutdown 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 msgr2=0x55980164c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980164c8e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000c00 tx=0x7f1b00009260 comp rx=0 tx=0).stop 2024-03-12T04:16:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 shutdown_connections 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5598016ae370 0x5598016b0790 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55980164a110 0x55980164a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598016457b0 0x559801649bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 --2- 172.21.15.42:0/964559057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55980163c2b0 0x55980164c8e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 >> 172.21.15.42:0/964559057 conn(0x559801636990 msgr2=0x559801640d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1af5ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 shutdown_connections 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 1 -- 172.21.15.42:0/964559057 wait complete. 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.829+0000 7f1b17814740 20 client.15771 trim_cache size 0 max 16384 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:16:05.837 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:16:05.838 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1af80a1d40 msgr2=0x7f1af80a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1af80a1d40 0x7f1af80a4200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009900 tx=0x7f1b00005a20 comp rx=0 tx=0).stop 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1019bee0 msgr2=0x7f1b1019c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1019bee0 0x7f1b1019c390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800cab0 tx=0x7f1b0800cf70 comp rx=0 tx=0).stop 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 shutdown_connections 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1af80a1d40 0x7f1af80a4200 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1019bee0 0x7f1b1019c390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b10106440 0x7f1b10197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 --2- 172.21.15.42:0/1243350257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b10101370 0x7f1b10197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 >> 172.21.15.42:0/1243350257 conn(0x7f1b100fcc30 msgr2=0x7f1b100fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 shutdown_connections 2024-03-12T04:16:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.830+0000 7f1b14b65640 1 -- 172.21.15.42:0/1243350257 wait complete. 2024-03-12T04:16:05.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:16:05.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:16:05.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 -- 172.21.15.42:0/2224713647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 msgr2=0x7fb340106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 --2- 172.21.15.42:0/2224713647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb340106520 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb330009900 tx=0x7fb33002bcd0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 -- 172.21.15.42:0/2224713647 shutdown_connections 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 --2- 172.21.15.42:0/2224713647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb340100a20 0x7fb34010cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 --2- 172.21.15.42:0/2224713647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb340100080 0x7fb3401004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 --2- 172.21.15.42:0/2224713647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb340106520 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 -- 172.21.15.42:0/2224713647 >> 172.21.15.42:0/2224713647 conn(0x7fb3400771b0 msgr2=0x7fb3400775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 -- 172.21.15.42:0/2224713647 shutdown_connections 2024-03-12T04:16:05.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.952+0000 7fb34546f640 1 -- 172.21.15.42:0/2224713647 wait complete. 2024-03-12T04:16:05.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 Processor -- start 2024-03-12T04:16:05.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 -- start start 2024-03-12T04:16:05.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb340100080 0x7fb340197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb340100a20 0x7fb340197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb34019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019c580 con 0x7fb340106140 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019c6f0 con 0x7fb340100a20 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb34546f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019c8b0 con 0x7fb340100080 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb34019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb34019bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46028/0 (socket says 172.21.15.42:46028) 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33f7fe640 1 -- 172.21.15.42:0/1645630575 learned_addr learned my addr 172.21.15.42:0/1645630575 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33effd640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb340100080 0x7fb340197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb337fff640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb340100a20 0x7fb340197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33f7fe640 1 -- 172.21.15.42:0/1645630575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb340100080 msgr2=0x7fb340197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33f7fe640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb340100080 0x7fb340197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33f7fe640 1 -- 172.21.15.42:0/1645630575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb340100a20 msgr2=0x7fb340197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.953+0000 7fb33f7fe640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb340100a20 0x7fb340197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.954+0000 7fb33f7fe640 1 -- 172.21.15.42:0/1645630575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330009560 con 0x7fb340106140 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.954+0000 7fb33f7fe640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb34019bf80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb33800e960 tx=0x7fb33800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.954+0000 7fb33cff9640 1 -- 172.21.15.42:0/1645630575 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338004d20 con 0x7fb340106140 2024-03-12T04:16:05.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.954+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34019cb90 con 0x7fb340106140 2024-03-12T04:16:05.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.954+0000 7fb33cff9640 1 -- 172.21.15.42:0/1645630575 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb338004e80 con 0x7fb340106140 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.954+0000 7fb33cff9640 1 -- 172.21.15.42:0/1645630575 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338011470 con 0x7fb340106140 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.954+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3401a4410 con 0x7fb340106140 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.955+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3400676e0 con 0x7fb340106140 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.956+0000 7fb33cff9640 1 -- 172.21.15.42:0/1645630575 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fb338005020 con 0x7fb340106140 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.957+0000 7fb33cff9640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb32c0a1cc0 0x7fb32c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.957+0000 7fb33cff9640 1 -- 172.21.15.42:0/1645630575 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb3380cdb00 con 0x7fb340106140 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.957+0000 7fb33effd640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb32c0a1cc0 0x7fb32c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.958+0000 7fb33effd640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb32c0a1cc0 0x7fb32c0a4180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb330005ab0 tx=0x7fb330005a20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:05.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:05.961+0000 7fb33cff9640 1 -- 172.21.15.42:0/1645630575 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb338096880 con 0x7fb340106140 2024-03-12T04:16:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.406+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3401100d0 con 0x7fb340106140 2024-03-12T04:16:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.407+0000 7fb33cff9640 1 -- 172.21.15.42:0/1645630575 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fb338099f40 con 0x7fb340106140 2024-03-12T04:16:06.410 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:16:06.410 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:16:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:16:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb32c0a1cc0 msgr2=0x7fb32c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb32c0a1cc0 0x7fb32c0a4180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb330005ab0 tx=0x7fb330005a20 comp rx=0 tx=0).stop 2024-03-12T04:16:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 msgr2=0x7fb34019bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb34019bf80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb33800e960 tx=0x7fb33800ee20 comp rx=0 tx=0).stop 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 shutdown_connections 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb32c0a1cc0 0x7fb32c0a4180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340106140 0x7fb34019bf80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb340100a20 0x7fb340197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 --2- 172.21.15.42:0/1645630575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb340100080 0x7fb340197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 >> 172.21.15.42:0/1645630575 conn(0x7fb3400771b0 msgr2=0x7fb3400fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 shutdown_connections 2024-03-12T04:16:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.410+0000 7fb34546f640 1 -- 172.21.15.42:0/1645630575 wait complete. 2024-03-12T04:16:06.428 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:06.428 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 -- 172.21.15.42:0/1577706087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 msgr2=0x7f9e7010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1577706087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e7010cf00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009cd0 tx=0x7f9e6802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 -- 172.21.15.42:0/1577706087 shutdown_connections 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1577706087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e7010cf00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1577706087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e70100c10 0x7f9e70104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1577706087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e70100260 0x7f9e70100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 -- 172.21.15.42:0/1577706087 >> 172.21.15.42:0/1577706087 conn(0x7f9e70077180 msgr2=0x7f9e70077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 -- 172.21.15.42:0/1577706087 shutdown_connections 2024-03-12T04:16:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.582+0000 7f9e7636b640 1 -- 172.21.15.42:0/1577706087 wait complete. 2024-03-12T04:16:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 Processor -- start 2024-03-12T04:16:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 -- start start 2024-03-12T04:16:06.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e70100260 0x7f9e70195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e701959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e70199d40 0x7f9e7019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019a730 con 0x7f9e70105330 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019a8a0 con 0x7f9e70100260 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e7636b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019aa10 con 0x7f9e70199d40 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e701959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e701959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46044/0 (socket says 172.21.15.42:46044) 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1996911032 learned_addr learned my addr 172.21.15.42:0/1996911032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e70100260 0x7f9e70195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.583+0000 7f9e74908640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e70199d40 0x7f9e7019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1996911032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e70199d40 msgr2=0x7f9e7019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e70199d40 0x7f9e7019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1996911032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e70100260 msgr2=0x7f9e70195480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e70100260 0x7f9e70195480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1996911032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68009900 con 0x7f9e70105330 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e701959c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e9c0 tx=0x7f9e6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6d7fa640 1 -- 172.21.15.42:0/1996911032 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64009700 con 0x7f9e70105330 2024-03-12T04:16:06.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e701a6400 con 0x7f9e70105330 2024-03-12T04:16:06.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e701a6900 con 0x7f9e70105330 2024-03-12T04:16:06.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6d7fa640 1 -- 172.21.15.42:0/1996911032 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e6400cd80 con 0x7f9e70105330 2024-03-12T04:16:06.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.584+0000 7f9e6d7fa640 1 -- 172.21.15.42:0/1996911032 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64010430 con 0x7f9e70105330 2024-03-12T04:16:06.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.587+0000 7f9e6d7fa640 1 -- 172.21.15.42:0/1996911032 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f9e64004f30 con 0x7f9e70105330 2024-03-12T04:16:06.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.587+0000 7f9e7900e740 1 Processor -- start 2024-03-12T04:16:06.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e7900e740 1 -- start start 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eef250 0x563eb4eef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4efab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e7900e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4eaa960 con 0x563eb4eef250 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e74908640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eef250 0x563eb4eef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e6d7fa640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e580a1d20 0x7f9e580a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e74908640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eef250 0x563eb4eef630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46054/0 (socket says 172.21.15.42:46054) 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e74908640 1 -- 172.21.15.42:0/3836230817 learned_addr learned my addr 172.21.15.42:0/3836230817 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e7900e740 1 -- 172.21.15.42:0/3836230817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4e5e310 con 0x563eb4ef8750 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e7900e740 1 -- 172.21.15.42:0/3836230817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4e5b840 con 0x563eb4eefb70 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e6d7fa640 1 -- 172.21.15.42:0/1996911032 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9e64014070 con 0x7f9e70105330 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.588+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4efab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e6ffff640 1 -- 172.21.15.42:0/3836230817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 msgr2=0x563eb4efab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4efab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e6ffff640 1 -- 172.21.15.42:0/3836230817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eef250 msgr2=0x563eb4eef630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eef250 0x563eb4eef630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e6ffff640 1 -- 172.21.15.42:0/3836230817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eb4e19060 con 0x563eb4eefb70 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e74908640 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eef250 0x563eb4eef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e74908640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e580a1d20 0x7f9e580a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef8210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000bbf0 tx=0x7f9e600107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d6acb13e70f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e567fc640 1 -- 172.21.15.42:0/3836230817 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60004030 con 0x563eb4eefb70 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e567fc640 1 -- 172.21.15.42:0/3836230817 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e60015070 con 0x563eb4eefb70 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.589+0000 7f9e567fc640 1 -- 172.21.15.42:0/3836230817 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60011440 con 0x563eb4eefb70 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e74908640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e580a1d20 0x7f9e580a41e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e68004af0 tx=0x7f9e6802e040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e7900e740 1 -- 172.21.15.42:0/3836230817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 msgr2=0x563eb4ef8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e7900e740 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef8210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000bbf0 tx=0x7f9e600107a0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e7900e740 1 -- 172.21.15.42:0/3836230817 shutdown_connections 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e7900e740 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4efab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e7900e740 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef8210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e7900e740 1 --2- 172.21.15.42:0/3836230817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eef250 0x563eb4eef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.590+0000 7f9e7900e740 1 -- 172.21.15.42:0/3836230817 >> 172.21.15.42:0/3836230817 conn(0x563eb4ee9980 msgr2=0x563eb4ee9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.591+0000 7f9e7900e740 1 -- 172.21.15.42:0/3836230817 shutdown_connections 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.591+0000 7f9e7900e740 1 -- 172.21.15.42:0/3836230817 wait complete. 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.591+0000 7f9e7900e740 1 -- start start 2024-03-12T04:16:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e7900e740 1 Processor -- start 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4ef57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4ef6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e74908640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4ef6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4ef57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e74908640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4ef6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46064/0 (socket says 172.21.15.42:46064) 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4ef57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:37984/0 (socket says 172.21.15.42:37984) 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e6f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.592+0000 7f9e74908640 1 -- 172.21.15.42:0/915980560 learned_addr learned my addr 172.21.15.42:0/915980560 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4e57020 con 0x563eb4ef8750 2024-03-12T04:16:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4b2a430 con 0x563eb4eef250 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4ae6e00 con 0x563eb4eefb70 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e74908640 1 -- 172.21.15.42:0/915980560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 msgr2=0x563eb4ef5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e74908640 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e74908640 1 -- 172.21.15.42:0/915980560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 msgr2=0x563eb4ef57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e74908640 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4ef57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e74908640 1 -- 172.21.15.42:0/915980560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6000b820 con 0x563eb4ef8750 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4ef57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e74908640 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4ef6220 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9e68037040 tx=0x7f9e68005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982f7a70c9639f50 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e7900e740 10 client.0 _ll_register_callbacks cb 0x563eb4dfecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/915980560 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68003b70 con 0x563eb4ef8750 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/915980560 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9e68034e40 con 0x563eb4ef8750 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.593+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563eb4b567e0 con 0x563eb4ef8750 2024-03-12T04:16:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/915980560 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e68003cd0 con 0x563eb4ef8750 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x563eb4e556a0 con 0x563eb4ef8750 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/915980560 <== mon.0 v2:172.21.15.42:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e68051070 con 0x563eb4ef8750 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 10 client.15789 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 20 client.15789 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f0ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 4 client.15789 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563eb4b567e0 con 0x563eb4f088c0 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.594+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f0ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.595+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f0ad80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ca5e2472abe5e35 server_cookie=39fb1dedc2e82f92 in_seq=0 out_seq=0 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.595+0000 7f9e4f7fe640 10 client.15789 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/915980560 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563eb4b567e0 con 0x563eb4f088c0 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e4f7fe640 10 client.15789 handle_command_reply: tid=0 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 client.15789 shutdown 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 msgr2=0x563eb4ef6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4ef6220 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9e68037040 tx=0x7f9e68005c50 comp rx=0 tx=0).stop 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 shutdown_connections 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f0ad80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4ef6220 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e4f7fe640 0 client.15789 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4ef5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 --2- 172.21.15.42:0/915980560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4ef57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.596+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 >> 172.21.15.42:0/915980560 conn(0x563eb4ee9980 msgr2=0x563eb4eeb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.597+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 shutdown_connections 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.597+0000 7f9e7900e740 1 -- 172.21.15.42:0/915980560 wait complete. 2024-03-12T04:16:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.597+0000 7f9e7900e740 20 client.15789 trim_cache size 0 max 16384 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 Processor -- start 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 -- start start 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eef250 0x563eb4ef7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eefb70 0x563eb4ef2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4ef2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4ddc130 con 0x563eb4eefb70 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4ee5600 con 0x563eb4ef8750 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e7900e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4eeeef0 con 0x563eb4eef250 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eefb70 0x563eb4ef2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eefb70 0x563eb4ef2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46080/0 (socket says 172.21.15.42:46080) 2024-03-12T04:16:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1351959141 learned_addr learned my addr 172.21.15.42:0/1351959141 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eef250 0x563eb4ef7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.621+0000 7f9e74908640 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4ef2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1351959141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eef250 msgr2=0x563eb4ef7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eef250 0x563eb4ef7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1351959141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 msgr2=0x563eb4ef2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4ef2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e6f7fe640 1 -- 172.21.15.42:0/1351959141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eb4df1c20 con 0x563eb4eefb70 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eefb70 0x563eb4ef2a40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000c00 tx=0x7f9e640ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ab09e469848745 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e55ffb640 1 -- 172.21.15.42:0/1351959141 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6409a050 con 0x563eb4eefb70 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e55ffb640 1 -- 172.21.15.42:0/1351959141 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e6409adf0 con 0x563eb4eefb70 2024-03-12T04:16:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.622+0000 7f9e55ffb640 1 -- 172.21.15.42:0/1351959141 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e640850b0 con 0x563eb4eefb70 2024-03-12T04:16:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 -- 172.21.15.42:0/1351959141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eefb70 msgr2=0x563eb4ef2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eefb70 0x563eb4ef2a40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000c00 tx=0x7f9e640ba040 comp rx=0 tx=0).stop 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 -- 172.21.15.42:0/1351959141 shutdown_connections 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4ef8750 0x563eb4ef2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4eefb70 0x563eb4ef2a40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1351959141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eef250 0x563eb4ef7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 -- 172.21.15.42:0/1351959141 >> 172.21.15.42:0/1351959141 conn(0x563eb4ee9980 msgr2=0x563eb4f0f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 -- 172.21.15.42:0/1351959141 shutdown_connections 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 -- 172.21.15.42:0/1351959141 wait complete. 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.623+0000 7f9e7900e740 1 -- start start 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 1 Processor -- start 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4effaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4efdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4efe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4efedd0 con 0x563eb4ef8750 2024-03-12T04:16:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4f5eeb0 con 0x563eb4eef250 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb4f5eff0 con 0x563eb4eefb70 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4effaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4effaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:38002/0 (socket says 172.21.15.42:38002) 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e6ffff640 1 -- 172.21.15.42:0/1709812366 learned_addr learned my addr 172.21.15.42:0/1709812366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e7900e740 10 client.0 _ll_register_callbacks cb 0x7f9e6000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e74908640 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4efe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.624+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4efdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e6ffff640 1 -- 172.21.15.42:0/1709812366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 msgr2=0x563eb4efdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4efdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e6ffff640 1 -- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 msgr2=0x563eb4efe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4efe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e6ffff640 1 -- 172.21.15.42:0/1709812366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e64083e70 con 0x563eb4eef250 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e6ffff640 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4effaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000bbf0 tx=0x7f9e60005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b097036b5269c1e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/1709812366 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6001f040 con 0x563eb4eef250 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/1709812366 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9e60017070 con 0x563eb4eef250 2024-03-12T04:16:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e60015070 con 0x563eb4eef250 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/1709812366 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e6000bed0 con 0x563eb4eef250 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e7900e740 10 client.25295 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563eb4e178c0 con 0x563eb4eef250 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.625+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/1709812366 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e60015070 con 0x563eb4eef250 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/1709812366 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e60029680 con 0x563eb4eef250 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e7900e740 10 client.25295 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e7900e740 10 client.25295 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e7900e740 20 client.25295 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f62700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e7900e740 4 client.25295 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563eb4b93720 con 0x563eb4f088c0 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f62700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e4f7fe640 10 client.25295 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.626+0000 7f9e6f7fe640 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f62700 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad1760d17f4fefcc server_cookie=c98df97fe9b9fe3c in_seq=0 out_seq=0 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e4f7fe640 1 -- 172.21.15.42:0/1709812366 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563eb4b93720 con 0x563eb4f088c0 2024-03-12T04:16:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e4f7fe640 10 client.25295 handle_command_reply: tid=0 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 client.25295 shutdown 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 msgr2=0x563eb4effaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4effaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000bbf0 tx=0x7f9e60005cc0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 shutdown_connections 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x563eb4f088c0 0x563eb4f62700 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e4f7fe640 0 client.25295 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb4ef8750 0x563eb4efe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb4eefb70 0x563eb4efdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 --2- 172.21.15.42:0/1709812366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563eb4eef250 0x563eb4effaf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 >> 172.21.15.42:0/1709812366 conn(0x563eb4ee9980 msgr2=0x563eb4ef4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 shutdown_connections 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 1 -- 172.21.15.42:0/1709812366 wait complete. 2024-03-12T04:16:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.627+0000 7f9e7900e740 20 client.25295 trim_cache size 0 max 16384 2024-03-12T04:16:06.635 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e580a1d20 msgr2=0x7f9e580a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e580a1d20 0x7f9e580a41e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e68004af0 tx=0x7f9e6802e040 comp rx=0 tx=0).stop 2024-03-12T04:16:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 msgr2=0x7f9e701959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e701959c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e9c0 tx=0x7f9e6400ee80 comp rx=0 tx=0).stop 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 shutdown_connections 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e580a1d20 0x7f9e580a41e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e70199d40 0x7f9e7019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e70105330 0x7f9e701959c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 --2- 172.21.15.42:0/1996911032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e70100260 0x7f9e70195480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 >> 172.21.15.42:0/1996911032 conn(0x7f9e70077180 msgr2=0x7f9e700fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 shutdown_connections 2024-03-12T04:16:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.628+0000 7f9e7636b640 1 -- 172.21.15.42:0/1996911032 wait complete. 2024-03-12T04:16:06.645 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:06.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:16:06.735 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:06 smithi042 ceph-mon[32260]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:06.735 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:06 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1645630575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 -- 172.21.15.42:0/1015990180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300100260 msgr2=0x7ff300100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 --2- 172.21.15.42:0/1015990180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300100260 0x7ff300100640 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009930 tx=0x7ff2f402bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 -- 172.21.15.42:0/1015990180 shutdown_connections 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 --2- 172.21.15.42:0/1015990180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff300105330 0x7ff30010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 --2- 172.21.15.42:0/1015990180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff300100c10 0x7ff300104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 --2- 172.21.15.42:0/1015990180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300100260 0x7ff300100640 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 -- 172.21.15.42:0/1015990180 >> 172.21.15.42:0/1015990180 conn(0x7ff300077180 msgr2=0x7ff300077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 -- 172.21.15.42:0/1015990180 shutdown_connections 2024-03-12T04:16:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 -- 172.21.15.42:0/1015990180 wait complete. 2024-03-12T04:16:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 Processor -- start 2024-03-12T04:16:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.755+0000 7ff306a76640 1 -- start start 2024-03-12T04:16:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff306a76640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff300100260 0x7ff30019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff306a76640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300105330 0x7ff30019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff306a76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3001975a0 0x7ff300197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff306a76640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff300197f90 con 0x7ff300105330 2024-03-12T04:16:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff306a76640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff300198100 con 0x7ff3001975a0 2024-03-12T04:16:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff306a76640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3001982c0 con 0x7ff300100260 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300105330 0x7ff30019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300105330 0x7ff30019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46098/0 (socket says 172.21.15.42:46098) 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 -- 172.21.15.42:0/3805894806 learned_addr learned my addr 172.21.15.42:0/3805894806 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff305013640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3001975a0 0x7ff300197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff304812640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff300100260 0x7ff30019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 -- 172.21.15.42:0/3805894806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff300100260 msgr2=0x7ff30019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff300100260 0x7ff30019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 -- 172.21.15.42:0/3805894806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3001975a0 msgr2=0x7ff300197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3001975a0 0x7ff300197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 -- 172.21.15.42:0/3805894806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009560 con 0x7ff300105330 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff305013640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3001975a0 0x7ff300197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.756+0000 7ff2fffff640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300105330 0x7ff30019d930 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00e9f0 tx=0x7ff2ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.757+0000 7ff2fdffb640 1 -- 172.21.15.42:0/3805894806 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0096c0 con 0x7ff300105330 2024-03-12T04:16:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.757+0000 7ff2fdffb640 1 -- 172.21.15.42:0/3805894806 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2ec0054d0 con 0x7ff300105330 2024-03-12T04:16:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.757+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff300106ed0 con 0x7ff300105330 2024-03-12T04:16:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.757+0000 7ff2fdffb640 1 -- 172.21.15.42:0/3805894806 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec010430 con 0x7ff300105330 2024-03-12T04:16:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.757+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff300107390 con 0x7ff300105330 2024-03-12T04:16:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 Processor -- start 2024-03-12T04:16:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 -- start start 2024-03-12T04:16:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047da60 0x56009047de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047e380 0x560090486a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x560090489350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560090430960 con 0x560090486f60 2024-03-12T04:16:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600903e4310 con 0x56009047e380 2024-03-12T04:16:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff30964a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600903e1840 con 0x56009047da60 2024-03-12T04:16:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff2fdffb640 1 -- 172.21.15.42:0/3805894806 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7ff2ec005020 con 0x7ff300105330 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff304812640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047e380 0x560090486a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff305013640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047da60 0x56009047de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff304812640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047e380 0x560090486a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:38034/0 (socket says 172.21.15.42:38034) 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.760+0000 7ff304812640 1 -- 172.21.15.42:0/2270063363 learned_addr learned my addr 172.21.15.42:0/2270063363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.761+0000 7ff2fffff640 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x560090489350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.761+0000 7ff2fdffb640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff2f00a1cc0 0x7ff2f00a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.761+0000 7ff2fdffb640 1 -- 172.21.15.42:0/3805894806 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff2ec014070 con 0x7ff300105330 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.761+0000 7ff305013640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff2f00a1cc0 0x7ff2f00a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.761+0000 7ff305013640 1 -- 172.21.15.42:0/2270063363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047e380 msgr2=0x560090486a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.761+0000 7ff305013640 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047e380 0x560090486a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.761+0000 7ff305013640 1 -- 172.21.15.42:0/2270063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 msgr2=0x560090489350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff305013640 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x560090489350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff305013640 1 -- 172.21.15.42:0/2270063363 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56009039f060 con 0x56009047da60 2024-03-12T04:16:06.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff2fffff640 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x560090489350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff305013640 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047da60 0x56009047de40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005fd0 tx=0x7ff2f8014ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e64fd3056ec254 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff305013640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff2f00a1cc0 0x7ff2f00a4180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2f801c000 tx=0x7ff2f801f000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff2eaffd640 1 -- 172.21.15.42:0/2270063363 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8014120 con 0x56009047da60 2024-03-12T04:16:06.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff2eaffd640 1 -- 172.21.15.42:0/2270063363 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2f801a070 con 0x56009047da60 2024-03-12T04:16:06.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.762+0000 7ff2eaffd640 1 -- 172.21.15.42:0/2270063363 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8011b50 con 0x56009047da60 2024-03-12T04:16:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 -- 172.21.15.42:0/2270063363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047da60 msgr2=0x56009047de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047da60 0x56009047de40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005fd0 tx=0x7ff2f8014ef0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 -- 172.21.15.42:0/2270063363 shutdown_connections 2024-03-12T04:16:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x560090489350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047e380 0x560090486a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 --2- 172.21.15.42:0/2270063363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047da60 0x56009047de40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 -- 172.21.15.42:0/2270063363 >> 172.21.15.42:0/2270063363 conn(0x560090478190 msgr2=0x560090478570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 -- 172.21.15.42:0/2270063363 shutdown_connections 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 -- 172.21.15.42:0/2270063363 wait complete. 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.763+0000 7ff30964a740 1 -- start start 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff30964a740 1 Processor -- start 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff30964a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090484050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff30964a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56009047e380 0x560090484590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff30964a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560090486f60 0x560090484ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff30964a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600903dd020 con 0x56009047e380 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff30964a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600900b0430 con 0x56009047da60 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56009047e380 0x560090484590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff304812640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090484050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff304812640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090484050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:38048/0 (socket says 172.21.15.42:38048) 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff305013640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560090486f60 0x560090484ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56009047e380 0x560090484590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46114/0 (socket says 172.21.15.42:46114) 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.764+0000 7ff304812640 1 -- 172.21.15.42:0/577938836 learned_addr learned my addr 172.21.15.42:0/577938836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009006ce00 con 0x560090486f60 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff30964a740 10 client.0 _ll_register_callbacks cb 0x560090384cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff305013640 1 -- 172.21.15.42:0/577938836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 msgr2=0x560090484050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff305013640 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090484050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff305013640 1 -- 172.21.15.42:0/577938836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56009047e380 msgr2=0x560090484590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff305013640 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56009047e380 0x560090484590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff305013640 1 -- 172.21.15.42:0/577938836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f80146d0 con 0x560090486f60 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff2fffff640 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56009047e380 0x560090484590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff304812640 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090484050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff305013640 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560090486f60 0x560090484ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005fd0 tx=0x7ff2f8005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb10ac0f1ae0780e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff2dffff640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:16:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.765+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600900dc7e0 con 0x560090486f60 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff2dffff640 1 -- 172.21.15.42:0/577938836 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8020440 con 0x560090486f60 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff2dffff640 1 -- 172.21.15.42:0/577938836 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff2f80205a0 con 0x560090486f60 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff2dffff640 1 -- 172.21.15.42:0/577938836 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2f8020760 con 0x560090486f60 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 10 client.25198 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5600903db6a0 con 0x560090486f60 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff2dffff640 1 -- 172.21.15.42:0/577938836 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff2f80208c0 con 0x560090486f60 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 10 client.25198 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 10 client.25198 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 20 client.25198 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 4 client.25198 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.766+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600900dc7e0 con 0x5600904971c0 2024-03-12T04:16:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.767+0000 7ff304812640 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.767+0000 7ff304812640 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904995b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc6cf83bfa89e7ef server_cookie=f0cac4735d27ad04 in_seq=0 out_seq=0 2024-03-12T04:16:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.767+0000 7ff2dffff640 10 client.25198 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff2dffff640 1 -- 172.21.15.42:0/577938836 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600900dc7e0 con 0x5600904971c0 2024-03-12T04:16:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff2dffff640 10 client.25198 handle_command_reply: tid=0 2024-03-12T04:16:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 client.25198 shutdown 2024-03-12T04:16:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560090486f60 msgr2=0x560090484ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560090486f60 0x560090484ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005fd0 tx=0x7ff2f8005c80 comp rx=0 tx=0).stop 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 shutdown_connections 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904995b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff2dffff640 0 client.25198 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560090486f60 0x560090484ad0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56009047e380 0x560090484590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 --2- 172.21.15.42:0/577938836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090484050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.768+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 >> 172.21.15.42:0/577938836 conn(0x560090478190 msgr2=0x560090479650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.769+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 shutdown_connections 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.769+0000 7ff30964a740 1 -- 172.21.15.42:0/577938836 wait complete. 2024-03-12T04:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.769+0000 7ff30964a740 20 client.25198 trim_cache size 0 max 16384 2024-03-12T04:16:06.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.793+0000 7ff30964a740 1 Processor -- start 2024-03-12T04:16:06.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.793+0000 7ff30964a740 1 -- start start 2024-03-12T04:16:06.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.793+0000 7ff30964a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090486780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.793+0000 7ff30964a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.793+0000 7ff30964a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff30964a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560090362130 con 0x560090486f60 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff30964a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600904793f0 con 0x56009047da60 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff30964a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560090479190 con 0x56009047e380 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090486780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090486780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:38064/0 (socket says 172.21.15.42:38064) 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 -- 172.21.15.42:0/1206742539 learned_addr learned my addr 172.21.15.42:0/1206742539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff305013640 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff2fffff640 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 -- 172.21.15.42:0/1206742539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 msgr2=0x56009048ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 -- 172.21.15.42:0/1206742539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 msgr2=0x56009048e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff304812640 1 -- 172.21.15.42:0/1206742539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560090377c20 con 0x56009047da60 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.794+0000 7ff305013640 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.795+0000 7ff304812640 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090486780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e52f0f13546a1a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.795+0000 7ff2ea7fc640 1 -- 172.21.15.42:0/1206742539 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4048de0 con 0x56009047da60 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.795+0000 7ff2ea7fc640 1 -- 172.21.15.42:0/1206742539 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2f402de90 con 0x56009047da60 2024-03-12T04:16:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.795+0000 7ff2ea7fc640 1 -- 172.21.15.42:0/1206742539 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4048720 con 0x56009047da60 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 -- 172.21.15.42:0/1206742539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 msgr2=0x560090486780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090486780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4005a20 comp rx=0 tx=0).stop 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 -- 172.21.15.42:0/1206742539 shutdown_connections 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 --2- 172.21.15.42:0/1206742539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x560090486780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 -- 172.21.15.42:0/1206742539 >> 172.21.15.42:0/1206742539 conn(0x560090478190 msgr2=0x56009049df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 -- 172.21.15.42:0/1206742539 shutdown_connections 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.796+0000 7ff30964a740 1 -- 172.21.15.42:0/1206742539 wait complete. 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 -- start start 2024-03-12T04:16:06.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 Processor -- start 2024-03-12T04:16:06.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x56009048de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff305013640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff2fffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:49084/0 (socket says 172.21.15.42:49084) 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff304812640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x56009048de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff305013640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46126/0 (socket says 172.21.15.42:46126) 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009048e6e0 con 0x560090486f60 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff2fffff640 1 -- 172.21.15.42:0/951559678 learned_addr learned my addr 172.21.15.42:0/951559678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009048cc30 con 0x56009047da60 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.797+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56009048cda0 con 0x56009047e380 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff30964a740 10 client.0 _ll_register_callbacks cb 0x7ff2f80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2fffff640 1 -- 172.21.15.42:0/951559678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 msgr2=0x56009048de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2fffff640 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x56009048de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2fffff640 1 -- 172.21.15.42:0/951559678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 msgr2=0x56009048c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2fffff640 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2fffff640 1 -- 172.21.15.42:0/951559678 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30010d710 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff304812640 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x56009048de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2fffff640 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048c1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec018000 tx=0x7ff2ec09a550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41fc0f778ec7e953 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2dffff640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2dffff640 1 -- 172.21.15.42:0/951559678 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec09a950 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff2dffff640 1 -- 172.21.15.42:0/951559678 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff2ec09eaa0 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.798+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2f801a070 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff2dffff640 1 -- 172.21.15.42:0/951559678 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2ec066170 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 10 client.25204 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56009039d8c0 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff2dffff640 1 -- 172.21.15.42:0/951559678 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2ec07d080 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff2dffff640 1 -- 172.21.15.42:0/951559678 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff2ec01f060 con 0x56009047e380 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 10 client.25204 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:06.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 10 client.25204 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 20 client.25204 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 4 client.25204 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560090119720 con 0x5600904971c0 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.799+0000 7ff304812640 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff2dffff640 10 client.25204 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff304812640 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904f0ad0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee57cb05d248885a server_cookie=205270f987a4f913 in_seq=0 out_seq=0 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff2dffff640 1 -- 172.21.15.42:0/951559678 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560090119720 con 0x5600904971c0 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff2dffff640 10 client.25204 handle_command_reply: tid=0 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 client.25204 shutdown 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 msgr2=0x56009048c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048c1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec018000 tx=0x7ff2ec09a550 comp rx=0 tx=0).stop 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 shutdown_connections 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5600904971c0 0x5600904f0ad0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560090486f60 0x56009048c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56009047e380 0x56009048c1b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 --2- 172.21.15.42:0/951559678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56009047da60 0x56009048de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 >> 172.21.15.42:0/951559678 conn(0x560090478190 msgr2=0x56009047f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff2dffff640 0 client.25204 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 shutdown_connections 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.800+0000 7ff30964a740 1 -- 172.21.15.42:0/951559678 wait complete. 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.801+0000 7ff30964a740 20 client.25204 trim_cache size 0 max 16384 2024-03-12T04:16:06.807 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:06.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.801+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff2f00a1cc0 msgr2=0x7ff2f00a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.801+0000 7ff306a76640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff2f00a1cc0 0x7ff2f00a4180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2f801c000 tx=0x7ff2f801f000 comp rx=0 tx=0).stop 2024-03-12T04:16:06.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.801+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300105330 msgr2=0x7ff30019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.801+0000 7ff306a76640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300105330 0x7ff30019d930 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00e9f0 tx=0x7ff2ec00eeb0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 shutdown_connections 2024-03-12T04:16:06.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff2f00a1cc0 0x7ff2f00a4180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3001975a0 0x7ff300197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff300105330 0x7ff30019d930 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 --2- 172.21.15.42:0/3805894806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff300100260 0x7ff30019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 >> 172.21.15.42:0/3805894806 conn(0x7ff300077180 msgr2=0x7ff3000febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 shutdown_connections 2024-03-12T04:16:06.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.802+0000 7ff306a76640 1 -- 172.21.15.42:0/3805894806 wait complete. 2024-03-12T04:16:06.816 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:06.816 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/3299626917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a08068fa0 msgr2=0x7f5a08105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/3299626917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a08068fa0 0x7f5a08105de0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009930 tx=0x7f59f802bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/3299626917 shutdown_connections 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/3299626917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081064c0 0x7f5a0810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/3299626917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a08068fa0 0x7f5a08105de0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/3299626917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a080685f0 0x7f5a080689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/3299626917 >> 172.21.15.42:0/3299626917 conn(0x7f5a08077180 msgr2=0x7f5a08077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/3299626917 shutdown_connections 2024-03-12T04:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.921+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/3299626917 wait complete. 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 Processor -- start 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 -- start start 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a080685f0 0x7f5a08197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a081064c0 0x7f5a08197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a0819c010 0x7f5a0819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0819ca00 con 0x7f5a0819c010 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0819cb70 con 0x7f5a080685f0 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0f9a1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0819cce0 con 0x7f5a081064c0 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0df3e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a0819c010 0x7f5a0819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0df3e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a0819c010 0x7f5a0819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46154/0 (socket says 172.21.15.42:46154) 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0df3e640 1 -- 172.21.15.42:0/353722026 learned_addr learned my addr 172.21.15.42:0/353722026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a080685f0 0x7f5a08197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.922+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a081064c0 0x7f5a08197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0df3e640 1 -- 172.21.15.42:0/353722026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a081064c0 msgr2=0x7f5a08197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0df3e640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a081064c0 0x7f5a08197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0df3e640 1 -- 172.21.15.42:0/353722026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a080685f0 msgr2=0x7f5a08197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0df3e640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a080685f0 0x7f5a08197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0df3e640 1 -- 172.21.15.42:0/353722026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009560 con 0x7f5a0819c010 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a080685f0 0x7f5a08197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0df3e640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a0819c010 0x7f5a0819c4c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009fd0 tx=0x7f5a0000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f59fe7fc640 1 -- 172.21.15.42:0/353722026 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000bea0 con 0x7f5a0819c010 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a081a42c0 con 0x7f5a0819c010 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f59fe7fc640 1 -- 172.21.15.42:0/353722026 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a00009940 con 0x7f5a0819c010 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f59fe7fc640 1 -- 172.21.15.42:0/353722026 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000106c0 con 0x7f5a0819c010 2024-03-12T04:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.923+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a081a4740 con 0x7f5a0819c010 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f59fe7fc640 1 -- 172.21.15.42:0/353722026 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f5a00003970 con 0x7f5a0819c010 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 Processor -- start 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 -- start start 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 0x55ac98e26b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98dd6960 con 0x55ac98e1b250 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98d8a310 con 0x55ac98e1bb70 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a1257b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98d87840 con 0x55ac98e24750 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a0cf3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a0cf3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e1b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46168/0 (socket says 172.21.15.42:46168) 2024-03-12T04:16:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/4084319040 learned_addr learned my addr 172.21.15.42:0/4084319040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.927+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/4084319040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 msgr2=0x55ac98e26b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 0x55ac98e26b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/4084319040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 msgr2=0x55ac98e24210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f59fe7fc640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59f40a1cc0 0x7f59f40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e24210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/4084319040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac98d45060 con 0x55ac98e1b250 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0df3e640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59f40a1cc0 0x7f59f40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e1b630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f59f8005940 tx=0x7f59f8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934c51f7a203baad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f59fe7fc640 1 -- 172.21.15.42:0/353722026 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5a000cd740 con 0x7f5a0819c010 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f59f37fe640 1 -- 172.21.15.42:0/4084319040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80049c0 con 0x55ac98e1b250 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f59f37fe640 1 -- 172.21.15.42:0/4084319040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59f802de90 con 0x55ac98e1b250 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.928+0000 7f59f37fe640 1 -- 172.21.15.42:0/4084319040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8003850 con 0x55ac98e1b250 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 -- 172.21.15.42:0/4084319040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 msgr2=0x55ac98e1b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a0df3e640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59f40a1cc0 0x7f59f40a4180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a000cadc0 tx=0x7f5a0006c370 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e1b630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f59f8005940 tx=0x7f59f8005de0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 -- 172.21.15.42:0/4084319040 shutdown_connections 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 0x55ac98e26b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 --2- 172.21.15.42:0/4084319040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e1b630 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 -- 172.21.15.42:0/4084319040 >> 172.21.15.42:0/4084319040 conn(0x55ac98e15980 msgr2=0x55ac98e15d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.929+0000 7f5a1257b740 1 -- 172.21.15.42:0/4084319040 shutdown_connections 2024-03-12T04:16:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.930+0000 7f5a1257b740 1 -- 172.21.15.42:0/4084319040 wait complete. 2024-03-12T04:16:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.930+0000 7f5a1257b740 1 -- start start 2024-03-12T04:16:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.930+0000 7f5a1257b740 1 Processor -- start 2024-03-12T04:16:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e21750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e21c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 0x55ac98e221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a1257b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98d83020 con 0x55ac98e1b250 2024-03-12T04:16:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a1257b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98a56430 con 0x55ac98e1bb70 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a1257b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98a12e00 con 0x55ac98e24750 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a1257b740 10 client.0 _ll_register_callbacks cb 0x55ac98d2acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e21750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e21750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46176/0 (socket says 172.21.15.42:46176) 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/2260057681 learned_addr learned my addr 172.21.15.42:0/2260057681 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/2260057681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 msgr2=0x55ac98e221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 0x55ac98e221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 msgr2=0x55ac98e21c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e21c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e21c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/2260057681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0810e880 con 0x55ac98e1b250 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.931+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e21c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e21750 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006010 tx=0x7f59f802c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11925f5e7b1d407e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f59f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2260057681 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8004430 con 0x55ac98e1b250 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2260057681 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59f80035f0 con 0x55ac98e1b250 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac98a827e0 con 0x55ac98e1b250 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2260057681 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f80037b0 con 0x55ac98e1b250 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f5a1257b740 10 client.15822 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ac98d816a0 con 0x55ac98e1b250 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2260057681 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ac98a827e0 con 0x55ac98e1b250 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.932+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2260057681 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59f8046020 con 0x55ac98e1b250 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a1257b740 10 client.15822 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a1257b740 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a1257b740 20 client.15822 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e346d0 0x55ac98e36af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a1257b740 4 client.15822 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac98abf720 con 0x55ac98e346d0 2024-03-12T04:16:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e346d0 0x55ac98e36af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f59f0ff9640 10 client.15822 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.933+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e346d0 0x55ac98e36af0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96cee322f60af75a server_cookie=9dbba6531e5d3bfa in_seq=0 out_seq=0 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2260057681 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac98abf720 con 0x55ac98e346d0 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f59f0ff9640 10 client.15822 handle_command_reply: tid=0 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 client.15822 shutdown 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 msgr2=0x55ac98e21750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e21750 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006010 tx=0x7f59f802c9a0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 shutdown_connections 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e346d0 0x55ac98e36af0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f59f0ff9640 0 client.15822 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e24750 0x55ac98e221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e21c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2260057681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e1b250 0x55ac98e21750 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.934+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 >> 172.21.15.42:0/2260057681 conn(0x55ac98e15980 msgr2=0x55ac98e176d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.935+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 shutdown_connections 2024-03-12T04:16:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.935+0000 7f5a1257b740 1 -- 172.21.15.42:0/2260057681 wait complete. 2024-03-12T04:16:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.935+0000 7f5a1257b740 20 client.15822 trim_cache size 0 max 16384 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.958+0000 7f5a1257b740 1 Processor -- start 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.958+0000 7f5a1257b740 1 -- start start 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1b250 0x55ac98e23e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e1ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e1e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a1257b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98d08130 con 0x55ac98e24750 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a1257b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98e31640 con 0x55ac98e1bb70 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a1257b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98e3bd00 con 0x55ac98e1b250 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e1e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e1e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46192/0 (socket says 172.21.15.42:46192) 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/3521405647 learned_addr learned my addr 172.21.15.42:0/3521405647 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e1ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1b250 0x55ac98e23e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/3521405647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1b250 msgr2=0x55ac98e23e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1b250 0x55ac98e23e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/3521405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 msgr2=0x55ac98e1ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e1ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/3521405647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac98d1dc20 con 0x55ac98e24750 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.959+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e1e9b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f59f802c170 tx=0x7f59f8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a45df7a99a1c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.960+0000 7f59f2ffd640 1 -- 172.21.15.42:0/3521405647 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f802ca10 con 0x55ac98e24750 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.960+0000 7f59f2ffd640 1 -- 172.21.15.42:0/3521405647 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59f8005270 con 0x55ac98e24750 2024-03-12T04:16:06.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.960+0000 7f59f2ffd640 1 -- 172.21.15.42:0/3521405647 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f803d980 con 0x55ac98e24750 2024-03-12T04:16:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 -- 172.21.15.42:0/3521405647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 msgr2=0x55ac98e1e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e1e9b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f59f802c170 tx=0x7f59f8001e70 comp rx=0 tx=0).stop 2024-03-12T04:16:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 -- 172.21.15.42:0/3521405647 shutdown_connections 2024-03-12T04:16:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e1e9b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1bb70 0x55ac98e1ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 --2- 172.21.15.42:0/3521405647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1b250 0x55ac98e23e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 -- 172.21.15.42:0/3521405647 >> 172.21.15.42:0/3521405647 conn(0x55ac98e15980 msgr2=0x55ac98e10a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 -- 172.21.15.42:0/3521405647 shutdown_connections 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 -- 172.21.15.42:0/3521405647 wait complete. 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.961+0000 7f5a1257b740 1 -- start start 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.962+0000 7f5a1257b740 1 Processor -- start 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.962+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1b250 0x55ac98e2a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.962+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1bb70 0x55ac98e2a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.962+0000 7f5a1257b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e2acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.962+0000 7f5a1257b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98e8ad50 con 0x55ac98e24750 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.962+0000 7f5a1257b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98e8aec0 con 0x55ac98e1b250 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a1257b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac98e8b080 con 0x55ac98e1bb70 2024-03-12T04:16:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0d73d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1bb70 0x55ac98e2a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e2acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0d73d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1bb70 0x55ac98e2a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:49140/0 (socket says 172.21.15.42:49140) 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0d73d640 1 -- 172.21.15.42:0/2607106300 learned_addr learned my addr 172.21.15.42:0/2607106300 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a1257b740 10 client.0 _ll_register_callbacks cb 0x7f5a04005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1b250 0x55ac98e2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/2607106300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1bb70 msgr2=0x55ac98e2a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1bb70 0x55ac98e2a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/2607106300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 msgr2=0x55ac98e2acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e2acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 -- 172.21.15.42:0/2607106300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac98d816a0 con 0x55ac98e1b250 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f5a0cf3c640 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1b250 0x55ac98e2a240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59f8053040 tx=0x7f59f8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7045ccfe2c14f6d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f59f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.963+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2607106300 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8069070 con 0x55ac98e1b250 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2607106300 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59f8004230 con 0x55ac98e1b250 2024-03-12T04:16:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac98b0e890 con 0x55ac98e1b250 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2607106300 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f8004520 con 0x55ac98e1b250 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f5a1257b740 10 client.25328 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ac98d438c0 con 0x55ac98e1b250 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2607106300 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59f8072080 con 0x55ac98e1b250 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2607106300 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59f806c020 con 0x55ac98e1b250 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f5a1257b740 10 client.25328 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f5a1257b740 10 client.25328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f5a1257b740 20 client.25328 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.964+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e8c3f0 0x55ac98e8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f5a1257b740 4 client.25328 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac98b0e890 con 0x55ac98e8c3f0 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e8c3f0 0x55ac98e8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f59f0ff9640 10 client.25328 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f5a0d73d640 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e8c3f0 0x55ac98e8e7e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f5a8d252668c838 server_cookie=473287c5f016e0dd in_seq=0 out_seq=0 2024-03-12T04:16:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f59f0ff9640 1 -- 172.21.15.42:0/2607106300 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac98b0e890 con 0x55ac98e8c3f0 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f59f0ff9640 10 client.25328 handle_command_reply: tid=0 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.965+0000 7f5a1257b740 1 client.25328 shutdown 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1b250 msgr2=0x55ac98e2a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1b250 0x55ac98e2a240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59f8053040 tx=0x7f59f8000bc0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 shutdown_connections 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ac98e8c3f0 0x55ac98e8e7e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f59f0ff9640 0 client.25328 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac98e24750 0x55ac98e2acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac98e1bb70 0x55ac98e2a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 --2- 172.21.15.42:0/2607106300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac98e1b250 0x55ac98e2a240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 >> 172.21.15.42:0/2607106300 conn(0x55ac98e15980 msgr2=0x55ac98e1ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 shutdown_connections 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 1 -- 172.21.15.42:0/2607106300 wait complete. 2024-03-12T04:16:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.966+0000 7f5a1257b740 20 client.25328 trim_cache size 0 max 16384 2024-03-12T04:16:06.974 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59f40a1cc0 msgr2=0x7f59f40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59f40a1cc0 0x7f59f40a4180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a000cadc0 tx=0x7f5a0006c370 comp rx=0 tx=0).stop 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a0819c010 msgr2=0x7f5a0819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a0819c010 0x7f5a0819c4c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009fd0 tx=0x7f5a0000b260 comp rx=0 tx=0).stop 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 shutdown_connections 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59f40a1cc0 0x7f59f40a4180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a0819c010 0x7f5a0819c4c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a081064c0 0x7f5a08197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 --2- 172.21.15.42:0/353722026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a080685f0 0x7f5a08197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 >> 172.21.15.42:0/353722026 conn(0x7f5a08077180 msgr2=0x7f5a080fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 shutdown_connections 2024-03-12T04:16:06.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:06.967+0000 7f5a0f9a1640 1 -- 172.21.15.42:0/353722026 wait complete. 2024-03-12T04:16:06.983 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:06.983 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 -- 172.21.15.42:0/2060968958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9581064c0 msgr2=0x7fd95810e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 --2- 172.21.15.42:0/2060968958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9581064c0 0x7fd95810e070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd950009cd0 tx=0x7fd95002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 -- 172.21.15.42:0/2060968958 shutdown_connections 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 --2- 172.21.15.42:0/2060968958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9581064c0 0x7fd95810e070 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 --2- 172.21.15.42:0/2060968958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd958068fa0 0x7fd958105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 --2- 172.21.15.42:0/2060968958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9580685f0 0x7fd9580689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 -- 172.21.15.42:0/2060968958 >> 172.21.15.42:0/2060968958 conn(0x7fd958077180 msgr2=0x7fd958077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 -- 172.21.15.42:0/2060968958 shutdown_connections 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 -- 172.21.15.42:0/2060968958 wait complete. 2024-03-12T04:16:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.089+0000 7fd96010d640 1 Processor -- start 2024-03-12T04:16:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd96010d640 1 -- start start 2024-03-12T04:16:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd96010d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9580685f0 0x7fd9581976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd96010d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9581064c0 0x7fd958197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd96010d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95819bf80 0x7fd95819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd96010d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95819c970 con 0x7fd9580685f0 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd96010d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95819cae0 con 0x7fd95819bf80 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd96010d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95819cc50 con 0x7fd9581064c0 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9580685f0 0x7fd9581976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9580685f0 0x7fd9581976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46214/0 (socket says 172.21.15.42:46214) 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 -- 172.21.15.42:0/2148460219 learned_addr learned my addr 172.21.15.42:0/2148460219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9581064c0 0x7fd958197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95819bf80 0x7fd95819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 -- 172.21.15.42:0/2148460219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9581064c0 msgr2=0x7fd958197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9581064c0 0x7fd958197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 -- 172.21.15.42:0/2148460219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95819bf80 msgr2=0x7fd95819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95819bf80 0x7fd95819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95dea9640 1 -- 172.21.15.42:0/2148460219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd950009900 con 0x7fd9580685f0 2024-03-12T04:16:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.090+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95819bf80 0x7fd95819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.091+0000 7fd95dea9640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9580685f0 0x7fd9581976c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd95400d950 tx=0x7fd95400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.091+0000 7fd94effd640 1 -- 172.21.15.42:0/2148460219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954009c30 con 0x7fd9580685f0 2024-03-12T04:16:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.091+0000 7fd94effd640 1 -- 172.21.15.42:0/2148460219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd954009d90 con 0x7fd9580685f0 2024-03-12T04:16:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.091+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9580fff70 con 0x7fd9580685f0 2024-03-12T04:16:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.091+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd958100420 con 0x7fd9580685f0 2024-03-12T04:16:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.092+0000 7fd94effd640 1 -- 172.21.15.42:0/2148460219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400f6b0 con 0x7fd9580685f0 2024-03-12T04:16:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.093+0000 7fd962dba740 1 Processor -- start 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd962dba740 1 -- start start 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd962dba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd962dba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 0x55da432e0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd962dba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432e0750 0x55da432e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd962dba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43292960 con 0x55da432e0750 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd962dba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43246310 con 0x55da432d7250 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd962dba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43243840 con 0x55da432d7b70 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd95d6a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432e0750 0x55da432e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd95d6a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432e0750 0x55da432e2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46226/0 (socket says 172.21.15.42:46226) 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1741466871 learned_addr learned my addr 172.21.15.42:0/1741466871 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.094+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd95e6aa640 1 -- 172.21.15.42:0/1741466871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 msgr2=0x55da432e0210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 0x55da432e0210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd95e6aa640 1 -- 172.21.15.42:0/1741466871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432e0750 msgr2=0x55da432e2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432e0750 0x55da432e2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd95e6aa640 1 -- 172.21.15.42:0/1741466871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da43201060 con 0x55da432d7250 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432e0750 0x55da432e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432d7630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd95002c630 tx=0x7fd950032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f15ff807e46b74 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd94effd640 1 -- 172.21.15.42:0/2148460219 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fd954010aa0 con 0x7fd9580685f0 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd943fff640 1 -- 172.21.15.42:0/1741466871 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950032c90 con 0x55da432d7250 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd943fff640 1 -- 172.21.15.42:0/1741466871 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd950032df0 con 0x55da432d7250 2024-03-12T04:16:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.095+0000 7fd943fff640 1 -- 172.21.15.42:0/1741466871 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500329b0 con 0x55da432d7250 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd94effd640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9440a1cc0 0x7fd9440a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd962dba740 1 -- 172.21.15.42:0/1741466871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 msgr2=0x55da432d7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd962dba740 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432d7630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd95002c630 tx=0x7fd950032340 comp rx=0 tx=0).stop 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd94effd640 1 -- 172.21.15.42:0/2148460219 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd9540cd710 con 0x7fd9580685f0 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd95dea9640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9440a1cc0 0x7fd9440a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd962dba740 1 -- 172.21.15.42:0/1741466871 shutdown_connections 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd962dba740 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432e0750 0x55da432e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd962dba740 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 0x55da432e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd962dba740 1 --2- 172.21.15.42:0/1741466871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432d7630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.096+0000 7fd962dba740 1 -- 172.21.15.42:0/1741466871 >> 172.21.15.42:0/1741466871 conn(0x55da432d1980 msgr2=0x55da432d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.097+0000 7fd95dea9640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9440a1cc0 0x7fd9440a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9540cad90 tx=0x7fd954099d20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.097+0000 7fd962dba740 1 -- 172.21.15.42:0/1741466871 shutdown_connections 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.097+0000 7fd962dba740 1 -- 172.21.15.42:0/1741466871 wait complete. 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.097+0000 7fd962dba740 1 -- start start 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.097+0000 7fd962dba740 1 Processor -- start 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd962dba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd962dba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd962dba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd962dba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4323f020 con 0x55da432d7b70 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd962dba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42f12430 con 0x55da432d7250 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd962dba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42ecee00 con 0x55da432e0750 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd962dba740 10 client.0 _ll_register_callbacks cb 0x55da431e6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95d6a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95e6aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95d6a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46242/0 (socket says 172.21.15.42:46242) 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1283933546 learned_addr learned my addr 172.21.15.42:0/1283933546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 msgr2=0x55da432de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432de1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.098+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1283933546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 msgr2=0x55da432dd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1283933546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95810e880 con 0x55da432d7b70 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddc80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd94800baa0 tx=0x7fd94800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ad92b520410b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd9417fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd9417fa640 1 -- 172.21.15.42:0/1283933546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948016070 con 0x55da432d7b70 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd9417fa640 1 -- 172.21.15.42:0/1283933546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd948011440 con 0x55da432d7b70 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.099+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da42f3e7e0 con 0x55da432d7b70 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd9417fa640 1 -- 172.21.15.42:0/1283933546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd948003e70 con 0x55da432d7b70 2024-03-12T04:16:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 10 client.15840 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da4323d6a0 con 0x55da432d7b70 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd9417fa640 1 -- 172.21.15.42:0/1283933546 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55da42f3e7e0 con 0x55da432d7b70 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd9417fa640 1 -- 172.21.15.42:0/1283933546 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd948025770 con 0x55da432d7b70 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 10 client.15840 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 10 client.15840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 20 client.15840 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da432f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 4 client.15840 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da42f7b720 con 0x55da432f0aa0 2024-03-12T04:16:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.100+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da432f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.101+0000 7fd9417fa640 10 client.15840 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.101+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da432f2ec0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=273b6e1e4087da36 server_cookie=cc7497dac37cc3 in_seq=0 out_seq=0 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.101+0000 7fd9417fa640 1 -- 172.21.15.42:0/1283933546 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da42f7b720 con 0x55da432f0aa0 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd9417fa640 10 client.15840 handle_command_reply: tid=0 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 client.15840 shutdown 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 msgr2=0x55da432ddc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddc80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd94800baa0 tx=0x7fd94800bf60 comp rx=0 tx=0).stop 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 shutdown_connections 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da432f2ec0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd9417fa640 0 client.15840 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddc80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 --2- 172.21.15.42:0/1283933546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 >> 172.21.15.42:0/1283933546 conn(0x55da432d1980 msgr2=0x55da432d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 shutdown_connections 2024-03-12T04:16:07.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 1 -- 172.21.15.42:0/1283933546 wait complete. 2024-03-12T04:16:07.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.102+0000 7fd962dba740 20 client.15840 trim_cache size 0 max 16384 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 Processor -- start 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 -- start start 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da431c4130 con 0x55da432d7b70 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42f3e810 con 0x55da432d7250 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd962dba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da432ed5b0 con 0x55da432e0750 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd95d6a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd95d6a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46252/0 (socket says 172.21.15.42:46252) 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.114+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1936801795 learned_addr learned my addr 172.21.15.42:0/1936801795 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1936801795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 msgr2=0x55da432f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1936801795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 msgr2=0x55da432dfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95d6a8640 1 -- 172.21.15.42:0/1936801795 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da431d9c20 con 0x55da432d7b70 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddfe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd94801b440 tx=0x7fd948004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398904e9bf22788 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd9437fe640 1 -- 172.21.15.42:0/1936801795 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948016070 con 0x55da432d7b70 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd9437fe640 1 -- 172.21.15.42:0/1936801795 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd948017040 con 0x55da432d7b70 2024-03-12T04:16:07.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.115+0000 7fd9437fe640 1 -- 172.21.15.42:0/1936801795 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948005830 con 0x55da432d7b70 2024-03-12T04:16:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 -- 172.21.15.42:0/1936801795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 msgr2=0x55da432ddfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddfe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd94801b440 tx=0x7fd948004290 comp rx=0 tx=0).stop 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 -- 172.21.15.42:0/1936801795 shutdown_connections 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432e0750 0x55da432f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7b70 0x55da432ddfe0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 --2- 172.21.15.42:0/1936801795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432d7250 0x55da432dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 -- 172.21.15.42:0/1936801795 >> 172.21.15.42:0/1936801795 conn(0x55da432d1980 msgr2=0x55da432cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 -- 172.21.15.42:0/1936801795 shutdown_connections 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 -- 172.21.15.42:0/1936801795 wait complete. 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 -- start start 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.116+0000 7fd962dba740 1 Processor -- start 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd962dba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7250 0x55da432e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd962dba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 0x55da432e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd962dba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432e0750 0x55da432e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd962dba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da432e6a20 con 0x55da432d7250 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd962dba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da432e6b90 con 0x55da432e0750 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd962dba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43346fb0 con 0x55da432d7b70 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd962dba740 10 client.0 _ll_register_callbacks cb 0x7fd950037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7250 0x55da432e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7250 0x55da432e79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46262/0 (socket says 172.21.15.42:46262) 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 -- 172.21.15.42:0/3351936540 learned_addr learned my addr 172.21.15.42:0/3351936540 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 0x55da432e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 -- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 msgr2=0x55da432e5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 0x55da432e5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 -- 172.21.15.42:0/3351936540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432e0750 msgr2=0x55da432e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432e0750 0x55da432e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.117+0000 7fd95e6aa640 1 -- 172.21.15.42:0/3351936540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da4323d6a0 con 0x55da432d7250 2024-03-12T04:16:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd95e6aa640 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7250 0x55da432e79e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd950032400 tx=0x7fd950032530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1def8d22bc78157a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd9417fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd9417fa640 1 -- 172.21.15.42:0/3351936540 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95004c040 con 0x55da432d7250 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd9417fa640 1 -- 172.21.15.42:0/3351936540 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd950034b20 con 0x55da432d7250 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd950032df0 con 0x55da432d7250 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd9417fa640 1 -- 172.21.15.42:0/3351936540 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd950034ce0 con 0x55da432d7250 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd962dba740 10 client.15846 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da431ff8c0 con 0x55da432d7250 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd9417fa640 1 -- 172.21.15.42:0/3351936540 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd950063020 con 0x55da432d7250 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd9417fa640 1 -- 172.21.15.42:0/3351936540 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd95005d070 con 0x55da432d7250 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd962dba740 10 client.15846 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd962dba740 10 client.15846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.118+0000 7fd962dba740 20 client.15846 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.119+0000 7fd962dba740 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da4334a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.119+0000 7fd962dba740 4 client.15846 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.119+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd950032df0 con 0x55da432f0aa0 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.119+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da4334a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.119+0000 7fd95d6a8640 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da4334a5c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=518de58eaf7da0f1 server_cookie=24e771ab9a12473a in_seq=0 out_seq=0 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.119+0000 7fd9417fa640 10 client.15846 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd9417fa640 1 -- 172.21.15.42:0/3351936540 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd950032df0 con 0x55da432f0aa0 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd9417fa640 10 client.15846 handle_command_reply: tid=0 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 client.15846 shutdown 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7250 msgr2=0x55da432e79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7250 0x55da432e79e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd950032400 tx=0x7fd950032530 comp rx=0 tx=0).stop 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 shutdown_connections 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55da432f0aa0 0x55da4334a5c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd9417fa640 0 client.15846 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da432e0750 0x55da432e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da432d7b70 0x55da432e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 --2- 172.21.15.42:0/3351936540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da432d7250 0x55da432e79e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 >> 172.21.15.42:0/3351936540 conn(0x55da432d1980 msgr2=0x55da432f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 shutdown_connections 2024-03-12T04:16:07.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 1 -- 172.21.15.42:0/3351936540 wait complete. 2024-03-12T04:16:07.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd962dba740 20 client.15846 trim_cache size 0 max 16384 2024-03-12T04:16:07.128 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9440a1cc0 msgr2=0x7fd9440a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd96010d640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9440a1cc0 0x7fd9440a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9540cad90 tx=0x7fd954099d20 comp rx=0 tx=0).stop 2024-03-12T04:16:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9580685f0 msgr2=0x7fd9581976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.120+0000 7fd96010d640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9580685f0 0x7fd9581976c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd95400d950 tx=0x7fd95400de10 comp rx=0 tx=0).stop 2024-03-12T04:16:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 shutdown_connections 2024-03-12T04:16:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9440a1cc0 0x7fd9440a4180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95819bf80 0x7fd95819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9581064c0 0x7fd958197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 --2- 172.21.15.42:0/2148460219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9580685f0 0x7fd9581976c0 secure :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd95400d950 tx=0x7fd95400de10 comp rx=0 tx=0).stop 2024-03-12T04:16:07.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 >> 172.21.15.42:0/2148460219 conn(0x7fd958077180 msgr2=0x7fd9580fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 shutdown_connections 2024-03-12T04:16:07.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.121+0000 7fd96010d640 1 -- 172.21.15.42:0/2148460219 wait complete. 2024-03-12T04:16:07.135 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:07.135 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:16:07.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:06 smithi119 ceph-mon[36119]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:07.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:06 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1645630575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:16:07.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:06 smithi136 ceph-mon[35571]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:07.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:06 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1645630575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:16:07.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.242+0000 7f609e99a640 1 -- 172.21.15.42:0/2926979391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 msgr2=0x7f609810e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.242+0000 7f609e99a640 1 --2- 172.21.15.42:0/2926979391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f609810e030 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6090009cd0 tx=0x7f609002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 -- 172.21.15.42:0/2926979391 shutdown_connections 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 --2- 172.21.15.42:0/2926979391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f609810e030 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 --2- 172.21.15.42:0/2926979391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098101d40 0x7f6098105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 --2- 172.21.15.42:0/2926979391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6098101390 0x7f6098101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 -- 172.21.15.42:0/2926979391 >> 172.21.15.42:0/2926979391 conn(0x7f60980fcbb0 msgr2=0x7f60980fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 -- 172.21.15.42:0/2926979391 shutdown_connections 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 -- 172.21.15.42:0/2926979391 wait complete. 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 Processor -- start 2024-03-12T04:16:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.243+0000 7f609e99a640 1 -- start start 2024-03-12T04:16:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f609e99a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098101390 0x7f6098197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f609e99a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f6098197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f609e99a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609819bf90 0x7f609819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f609e99a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609819c980 con 0x7f6098106460 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f609e99a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609819caf0 con 0x7f6098101390 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f609e99a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609819cc60 con 0x7f609819bf90 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f6098197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f6098197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46274/0 (socket says 172.21.15.42:46274) 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 -- 172.21.15.42:0/1037489284 learned_addr learned my addr 172.21.15.42:0/1037489284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f609cf37640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609819bf90 0x7f609819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f6097fff640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098101390 0x7f6098197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 -- 172.21.15.42:0/1037489284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609819bf90 msgr2=0x7f609819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609819bf90 0x7f609819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 -- 172.21.15.42:0/1037489284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098101390 msgr2=0x7f6098197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098101390 0x7f6098197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 -- 172.21.15.42:0/1037489284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6090009900 con 0x7f6098106460 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f6097fff640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098101390 0x7f6098197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:16:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.244+0000 7f60977fe640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f6098197c60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f608c00d770 tx=0x7f608c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.245+0000 7f60957fa640 1 -- 172.21.15.42:0/1037489284 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c009c30 con 0x7f6098106460 2024-03-12T04:16:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.245+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60981a4240 con 0x7f6098106460 2024-03-12T04:16:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.245+0000 7f60957fa640 1 -- 172.21.15.42:0/1037489284 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f608c00de80 con 0x7f6098106460 2024-03-12T04:16:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.245+0000 7f60957fa640 1 -- 172.21.15.42:0/1037489284 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c00f600 con 0x7f6098106460 2024-03-12T04:16:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.245+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981a4740 con 0x7f6098106460 2024-03-12T04:16:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.248+0000 7f60957fa640 1 -- 172.21.15.42:0/1037489284 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f608c010460 con 0x7f6098106460 2024-03-12T04:16:07.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.248+0000 7f60a1571740 1 Processor -- start 2024-03-12T04:16:07.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.248+0000 7f60a1571740 1 -- start start 2024-03-12T04:16:07.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60a1571740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761590630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60a1571740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x562761599210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60a1571740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761599750 0x56276159bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60a1571740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276154b960 con 0x562761590250 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60a1571740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627614ff310 con 0x562761590b70 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60a1571740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627614fc840 con 0x562761599750 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60957fa640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60800a1d40 0x7f60800a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f609cf37640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761590630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f609cf37640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761590630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46276/0 (socket says 172.21.15.42:46276) 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f609cf37640 1 -- 172.21.15.42:0/2561155017 learned_addr learned my addr 172.21.15.42:0/2561155017 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60977fe640 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761599750 0x56276159bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f609cf37640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60800a1d40 0x7f60800a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f6097fff640 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x562761599210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60957fa640 1 -- 172.21.15.42:0/1037489284 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f608c0cd6e0 con 0x7f6098106460 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60977fe640 1 -- 172.21.15.42:0/2561155017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 msgr2=0x562761599210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60977fe640 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x562761599210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60977fe640 1 -- 172.21.15.42:0/2561155017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 msgr2=0x562761590630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60977fe640 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761590630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.249+0000 7f60977fe640 1 -- 172.21.15.42:0/2561155017 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627614ba060 con 0x562761599750 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.250+0000 7f60977fe640 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761599750 0x56276159bb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f608c0cad00 tx=0x7f608c016310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270ce162610ef7ae server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.250+0000 7f609cf37640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60800a1d40 0x7f60800a4200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6090006010 tx=0x7f60900093f0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.250+0000 7f607e7fc640 1 -- 172.21.15.42:0/2561155017 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c013ea0 con 0x562761599750 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.250+0000 7f607e7fc640 1 -- 172.21.15.42:0/2561155017 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f608c099a70 con 0x562761599750 2024-03-12T04:16:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.250+0000 7f607e7fc640 1 -- 172.21.15.42:0/2561155017 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c096780 con 0x562761599750 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 -- 172.21.15.42:0/2561155017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761599750 msgr2=0x56276159bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761599750 0x56276159bb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f608c0cad00 tx=0x7f608c016310 comp rx=0 tx=0).stop 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 -- 172.21.15.42:0/2561155017 shutdown_connections 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761599750 0x56276159bb40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x562761599210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 --2- 172.21.15.42:0/2561155017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761590630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 -- 172.21.15.42:0/2561155017 >> 172.21.15.42:0/2561155017 conn(0x56276158a980 msgr2=0x56276158ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 -- 172.21.15.42:0/2561155017 shutdown_connections 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 -- 172.21.15.42:0/2561155017 wait complete. 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 -- start start 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.251+0000 7f60a1571740 1 Processor -- start 2024-03-12T04:16:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60a1571740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590250 0x562761599020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60a1571740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761592370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60a1571740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60a1571740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627614f8020 con 0x562761599750 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60a1571740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627611cb430 con 0x562761590250 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60a1571740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562761187e00 con 0x562761590b70 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60a1571740 10 client.0 _ll_register_callbacks cb 0x56276149fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60977fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761592370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60977fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761592370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:49216/0 (socket says 172.21.15.42:49216) 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60977fe640 1 -- 172.21.15.42:0/2147266189 learned_addr learned my addr 172.21.15.42:0/2147266189 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f609cf37640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f6097fff640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590250 0x562761599020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60977fe640 1 -- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590250 msgr2=0x562761599020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60977fe640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590250 0x562761599020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.252+0000 7f60977fe640 1 -- 172.21.15.42:0/2147266189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 msgr2=0x5627615928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60977fe640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615928b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60977fe640 1 -- 172.21.15.42:0/2147266189 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981a4740 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f609cf37640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60977fe640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761592370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f608c096ae0 tx=0x7f608c0a2a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5edd4bfc29db9465 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60777fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60777fe640 1 -- 172.21.15.42:0/2147266189 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c09a810 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60777fe640 1 -- 172.21.15.42:0/2147266189 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f608c0b2e40 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627611f77e0 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60777fe640 1 -- 172.21.15.42:0/2147266189 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f608c09a110 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60a1571740 10 client.25243 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.253+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5627614f66a0 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60777fe640 1 -- 172.21.15.42:0/2147266189 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f608c04f030 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60777fe640 1 -- 172.21.15.42:0/2147266189 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f608c0b2030 con 0x562761590b70 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60a1571740 10 client.25243 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:07.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60a1571740 10 client.25243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60a1571740 20 client.25243 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60a1571740 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5627615b1ef0 0x5627615b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60a1571740 4 client.25243 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562761234720 con 0x5627615b1ef0 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.254+0000 7f6097fff640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5627615b1ef0 0x5627615b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.255+0000 7f60777fe640 10 client.25243 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.255+0000 7f6097fff640 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5627615b1ef0 0x5627615b42e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6241660ade208b90 server_cookie=ca11042b9e10d6d5 in_seq=0 out_seq=0 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.255+0000 7f60777fe640 1 -- 172.21.15.42:0/2147266189 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562761234720 con 0x5627615b1ef0 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.255+0000 7f60777fe640 10 client.25243 handle_command_reply: tid=0 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.255+0000 7f60a1571740 1 client.25243 shutdown 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.255+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 msgr2=0x562761592370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.255+0000 7f60a1571740 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761592370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f608c096ae0 tx=0x7f608c0a2a30 comp rx=0 tx=0).stop 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 shutdown_connections 2024-03-12T04:16:07.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5627615b1ef0 0x5627615b42e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761592370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 --2- 172.21.15.42:0/2147266189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590250 0x562761599020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 >> 172.21.15.42:0/2147266189 conn(0x56276158a980 msgr2=0x56276158c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60777fe640 0 client.25243 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 shutdown_connections 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 1 -- 172.21.15.42:0/2147266189 wait complete. 2024-03-12T04:16:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.256+0000 7f60a1571740 20 client.25243 trim_cache size 0 max 16384 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.268+0000 7f60a1571740 1 Processor -- start 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.268+0000 7f60a1571740 1 -- start start 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f60a1571740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761598b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f60a1571740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761590f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f60a1571740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761599750 0x562761591490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f60a1571740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276147d130 con 0x562761590250 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f60a1571740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627615979e0 con 0x562761599750 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f60a1571740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562761586100 con 0x562761590b70 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761598b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761598b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46282/0 (socket says 172.21.15.42:46282) 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 -- 172.21.15.42:0/619782803 learned_addr learned my addr 172.21.15.42:0/619782803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f60977fe640 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761590f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f609cf37640 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761599750 0x562761591490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 -- 172.21.15.42:0/619782803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 msgr2=0x562761590f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761590f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 -- 172.21.15.42:0/619782803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761599750 msgr2=0x562761591490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761599750 0x562761591490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 -- 172.21.15.42:0/619782803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562761492c20 con 0x562761590250 2024-03-12T04:16:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.269+0000 7f6097fff640 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761598b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x56276158fd20 tx=0x7f6088009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef704977ac2945c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.270+0000 7f607dffb640 1 -- 172.21.15.42:0/619782803 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088018040 con 0x562761590250 2024-03-12T04:16:07.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.270+0000 7f607dffb640 1 -- 172.21.15.42:0/619782803 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60880142d0 con 0x562761590250 2024-03-12T04:16:07.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.270+0000 7f607dffb640 1 -- 172.21.15.42:0/619782803 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60880126b0 con 0x562761590250 2024-03-12T04:16:07.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.270+0000 7f60a1571740 1 -- 172.21.15.42:0/619782803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 msgr2=0x562761598b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.270+0000 7f60a1571740 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761598b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x56276158fd20 tx=0x7f6088009500 comp rx=0 tx=0).stop 2024-03-12T04:16:07.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 -- 172.21.15.42:0/619782803 shutdown_connections 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761599750 0x562761591490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590b70 0x562761590f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 --2- 172.21.15.42:0/619782803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761590250 0x562761598b50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 -- 172.21.15.42:0/619782803 >> 172.21.15.42:0/619782803 conn(0x56276158a980 msgr2=0x56276159cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 -- 172.21.15.42:0/619782803 shutdown_connections 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 -- 172.21.15.42:0/619782803 wait complete. 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 -- start start 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 Processor -- start 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.271+0000 7f60a1571740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590250 0x5627615ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60a1571740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x5627615aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60a1571740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60a1571740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627615ab7d0 con 0x562761599750 2024-03-12T04:16:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60a1571740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627615ab940 con 0x562761590b70 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60a1571740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627615abb00 con 0x562761590250 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60a1571740 10 client.0 _ll_register_callbacks cb 0x7f608c010b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f609cf37640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f6097fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590250 0x5627615ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f609cf37640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46292/0 (socket says 172.21.15.42:46292) 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x5627615aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f609cf37640 1 -- 172.21.15.42:0/1469622350 learned_addr learned my addr 172.21.15.42:0/1469622350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f6097fff640 1 -- 172.21.15.42:0/1469622350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 msgr2=0x5627615aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f6097fff640 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x5627615aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f6097fff640 1 -- 172.21.15.42:0/1469622350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 msgr2=0x5627615ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f6097fff640 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f6097fff640 1 -- 172.21.15.42:0/1469622350 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6088005740 con 0x562761590250 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f6097fff640 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590250 0x5627615ae4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6088021000 tx=0x7f6088014c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad5680765ec084bf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.272+0000 7f60777fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:16:07.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60777fe640 1 -- 172.21.15.42:0/1469622350 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608801a6c0 con 0x562761590250 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60777fe640 1 -- 172.21.15.42:0/1469622350 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f60880140d0 con 0x562761590250 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f608c099a70 con 0x562761590250 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60777fe640 1 -- 172.21.15.42:0/1469622350 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f608801a820 con 0x562761590250 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 10 client.25249 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5627614b88c0 con 0x562761590250 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60777fe640 1 -- 172.21.15.42:0/1469622350 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f608c099a70 con 0x562761590250 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60777fe640 1 -- 172.21.15.42:0/1469622350 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60880235e0 con 0x562761590250 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 10 client.25249 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 10 client.25249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 20 client.25249 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562761609480 0x56276160b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 4 client.25249 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.273+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562761283890 con 0x562761609480 2024-03-12T04:16:07.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60977fe640 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562761609480 0x56276160b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60777fe640 10 client.25249 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60977fe640 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562761609480 0x56276160b8a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214c47632894f112 server_cookie=2192d28fd630f1b7 in_seq=0 out_seq=0 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60777fe640 1 -- 172.21.15.42:0/1469622350 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562761283890 con 0x562761609480 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60777fe640 10 client.25249 handle_command_reply: tid=0 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 client.25249 shutdown 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590250 msgr2=0x5627615ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590250 0x5627615ae4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6088021000 tx=0x7f6088014c50 comp rx=0 tx=0).stop 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 shutdown_connections 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562761609480 0x56276160b8a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562761599750 0x5627615ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60777fe640 0 client.25249 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562761590b70 0x5627615aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 --2- 172.21.15.42:0/1469622350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562761590250 0x5627615ae4c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.274+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 >> 172.21.15.42:0/1469622350 conn(0x56276158a980 msgr2=0x562761590f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 shutdown_connections 2024-03-12T04:16:07.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f60a1571740 1 -- 172.21.15.42:0/1469622350 wait complete. 2024-03-12T04:16:07.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f60a1571740 20 client.25249 trim_cache size 0 max 16384 2024-03-12T04:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60800a1d40 msgr2=0x7f60800a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60800a1d40 0x7f60800a4200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6090006010 tx=0x7f60900093f0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 msgr2=0x7f6098197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f6098197c60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f608c00d770 tx=0x7f608c00dc30 comp rx=0 tx=0).stop 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 shutdown_connections 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60800a1d40 0x7f60800a4200 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609819bf90 0x7f609819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6098106460 0x7f6098197c60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 --2- 172.21.15.42:0/1037489284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098101390 0x7f6098197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 >> 172.21.15.42:0/1037489284 conn(0x7f60980fcbb0 msgr2=0x7f60980fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 shutdown_connections 2024-03-12T04:16:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:07.275+0000 7f609e99a640 1 -- 172.21.15.42:0/1037489284 wait complete. 2024-03-12T04:16:08.293 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:16:08.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:16:08.293 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.409+0000 7fb3830b5640 1 -- 172.21.15.42:0/1422950673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c106430 msgr2=0x7fb37c10e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.409+0000 7fb3830b5640 1 --2- 172.21.15.42:0/1422950673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c106430 0x7fb37c10e000 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb374009cd0 tx=0x7fb37402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.409+0000 7fb3830b5640 1 -- 172.21.15.42:0/1422950673 shutdown_connections 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.409+0000 7fb3830b5640 1 --2- 172.21.15.42:0/1422950673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c106430 0x7fb37c10e000 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 --2- 172.21.15.42:0/1422950673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c068f80 0x7fb37c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 --2- 172.21.15.42:0/1422950673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb37c0685d0 0x7fb37c0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 -- 172.21.15.42:0/1422950673 >> 172.21.15.42:0/1422950673 conn(0x7fb37c077180 msgr2=0x7fb37c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 -- 172.21.15.42:0/1422950673 shutdown_connections 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 -- 172.21.15.42:0/1422950673 wait complete. 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 Processor -- start 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 -- start start 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c0685d0 0x7fb37c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb37c106430 0x7fb37c197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.410+0000 7fb3830b5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c19bf90 0x7fb37c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb3830b5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c19c980 con 0x7fb37c0685d0 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb3830b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c19caf0 con 0x7fb37c106430 2024-03-12T04:16:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb3830b5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c19cc60 con 0x7fb37c19bf90 2024-03-12T04:16:08.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c0685d0 0x7fb37c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c0685d0 0x7fb37c1976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46312/0 (socket says 172.21.15.42:46312) 2024-03-12T04:16:08.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 -- 172.21.15.42:0/967585520 learned_addr learned my addr 172.21.15.42:0/967585520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb37bfff640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb37c106430 0x7fb37c197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb381652640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c19bf90 0x7fb37c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 -- 172.21.15.42:0/967585520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c19bf90 msgr2=0x7fb37c19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c19bf90 0x7fb37c19c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 -- 172.21.15.42:0/967585520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb37c106430 msgr2=0x7fb37c197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb37c106430 0x7fb37c197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 -- 172.21.15.42:0/967585520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb374009900 con 0x7fb37c0685d0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.411+0000 7fb380e51640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c0685d0 0x7fb37c1976d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e9f0 tx=0x7fb37000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.412+0000 7fb379ffb640 1 -- 172.21.15.42:0/967585520 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370009700 con 0x7fb37c0685d0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.412+0000 7fb379ffb640 1 -- 172.21.15.42:0/967585520 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb37000cd80 con 0x7fb37c0685d0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.412+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37c1a4240 con 0x7fb37c0685d0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.412+0000 7fb379ffb640 1 -- 172.21.15.42:0/967585520 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370018840 con 0x7fb37c0685d0 2024-03-12T04:16:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.412+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37c1a4740 con 0x7fb37c0685d0 2024-03-12T04:16:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb379ffb640 1 -- 172.21.15.42:0/967585520 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fb3700189a0 con 0x7fb37c0685d0 2024-03-12T04:16:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 Processor -- start 2024-03-12T04:16:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 -- start start 2024-03-12T04:16:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecaa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ec18f90 con 0x55ea1ecaa380 2024-03-12T04:16:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ec164c0 con 0x55ea1ecb38b0 2024-03-12T04:16:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.415+0000 7fb385d63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ec11ca0 con 0x55ea1ecaad30 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecaa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecaa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46318/0 (socket says 172.21.15.42:46318) 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 -- 172.21.15.42:0/890901001 learned_addr learned my addr 172.21.15.42:0/890901001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb380e51640 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb37bfff640 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 -- 172.21.15.42:0/890901001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaad30 msgr2=0x55ea1ecb32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 -- 172.21.15.42:0/890901001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecb38b0 msgr2=0x55ea1ecb5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 -- 172.21.15.42:0/890901001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea1ebd40b0 con 0x55ea1ecaa380 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.416+0000 7fb381652640 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecaa760 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb37402c700 tx=0x7fb374032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b898c9bf8d9860 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.417+0000 7fb366ffd640 1 -- 172.21.15.42:0/890901001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374032c90 con 0x55ea1ecaa380 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.417+0000 7fb366ffd640 1 -- 172.21.15.42:0/890901001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb374032df0 con 0x55ea1ecaa380 2024-03-12T04:16:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.417+0000 7fb366ffd640 1 -- 172.21.15.42:0/890901001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374032a00 con 0x55ea1ecaa380 2024-03-12T04:16:08.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.417+0000 7fb379ffb640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb36c0a1d40 0x7fb36c0a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.417+0000 7fb380e51640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb36c0a1d40 0x7fb36c0a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.417+0000 7fb385d63740 1 -- 172.21.15.42:0/890901001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 msgr2=0x55ea1ecaa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.417+0000 7fb385d63740 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecaa760 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb37402c700 tx=0x7fb374032340 comp rx=0 tx=0).stop 2024-03-12T04:16:08.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 -- 172.21.15.42:0/890901001 shutdown_connections 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 --2- 172.21.15.42:0/890901001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecaa760 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 -- 172.21.15.42:0/890901001 >> 172.21.15.42:0/890901001 conn(0x55ea1eca4840 msgr2=0x55ea1eca4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb379ffb640 1 -- 172.21.15.42:0/967585520 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb370014070 con 0x7fb37c0685d0 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb380e51640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb36c0a1d40 0x7fb36c0a4200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3700cb340 tx=0x7fb3700d15b0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 -- 172.21.15.42:0/890901001 shutdown_connections 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 -- 172.21.15.42:0/890901001 wait complete. 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 -- start start 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.418+0000 7fb385d63740 1 Processor -- start 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb385d63740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb385d63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb385d63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb385d63740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ebb6f40 con 0x55ea1ecaa380 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb385d63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1e8e54b0 con 0x55ea1ecaad30 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb385d63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1eb97670 con 0x55ea1ecb38b0 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb385d63740 10 client.0 _ll_register_callbacks cb 0x55ea1ec04f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:38230/0 (socket says 172.21.15.42:38230) 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 -- 172.21.15.42:0/223412869 learned_addr learned my addr 172.21.15.42:0/223412869 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb381652640 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb381652640 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 -- 172.21.15.42:0/223412869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 msgr2=0x55ea1ecb1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 -- 172.21.15.42:0/223412869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 msgr2=0x55ea1ecb08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.419+0000 7fb37bfff640 1 -- 172.21.15.42:0/223412869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37c10e810 con 0x55ea1ecaad30 2024-03-12T04:16:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb37bfff640 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb0de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb368009a60 tx=0x7fb368009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf32a9903180024 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb35bfff640 1 -- 172.21.15.42:0/223412869 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368004050 con 0x55ea1ecaad30 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea1ebebf40 con 0x55ea1ecaad30 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb35bfff640 1 -- 172.21.15.42:0/223412869 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb3680041b0 con 0x55ea1ecaad30 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb35bfff640 1 -- 172.21.15.42:0/223412869 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb368004370 con 0x55ea1ecaad30 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 10 client.25370 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea1ec10320 con 0x55ea1ecaad30 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb35bfff640 1 -- 172.21.15.42:0/223412869 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ea1ebebf40 con 0x55ea1ecaad30 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb35bfff640 1 -- 172.21.15.42:0/223412869 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb368026480 con 0x55ea1ecaad30 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 10 client.25370 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 10 client.25370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 20 client.25370 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ecc3e20 0x55ea1ecc6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 4 client.25370 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.420+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea1ec08310 con 0x55ea1ecc3e20 2024-03-12T04:16:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.421+0000 7fb381652640 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ecc3e20 0x55ea1ecc6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.421+0000 7fb35bfff640 10 client.25370 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.421+0000 7fb381652640 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ecc3e20 0x55ea1ecc6240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd9fa58201ebdf1 server_cookie=e9975da0faae3548 in_seq=0 out_seq=0 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb35bfff640 1 -- 172.21.15.42:0/223412869 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea1ec08310 con 0x55ea1ecc3e20 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb35bfff640 10 client.25370 handle_command_reply: tid=0 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 client.25370 shutdown 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 msgr2=0x55ea1ecb0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb0de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb368009a60 tx=0x7fb368009f20 comp rx=0 tx=0).stop 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 shutdown_connections 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ecc3e20 0x55ea1ecc6240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecb1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb0de0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 --2- 172.21.15.42:0/223412869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb35bfff640 0 client.25370 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 >> 172.21.15.42:0/223412869 conn(0x55ea1eca4840 msgr2=0x55ea1eca67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 shutdown_connections 2024-03-12T04:16:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 1 -- 172.21.15.42:0/223412869 wait complete. 2024-03-12T04:16:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.422+0000 7fb385d63740 20 client.25370 trim_cache size 0 max 16384 2024-03-12T04:16:08.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 Processor -- start 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 -- start start 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1eca3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ecbf890 con 0x55ea1ecb38b0 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ecb1bf0 con 0x55ea1ecaad30 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb385d63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ec9ecf0 con 0x55ea1ecaa380 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb381652640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb381652640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46334/0 (socket says 172.21.15.42:46334) 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb381652640 1 -- 172.21.15.42:0/3633079523 learned_addr learned my addr 172.21.15.42:0/3633079523 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb37bfff640 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1eca3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.438+0000 7fb381652640 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb381652640 1 -- 172.21.15.42:0/3633079523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaa380 msgr2=0x55ea1ecb1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb381652640 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb381652640 1 -- 172.21.15.42:0/3633079523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 msgr2=0x55ea1eca3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb381652640 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1eca3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb381652640 1 -- 172.21.15.42:0/3633079523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea1ebacc00 con 0x55ea1ecb38b0 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb381652640 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecba880 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb37402c700 tx=0x7fb37404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89142fc2f448ef85 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb3667fc640 1 -- 172.21.15.42:0/3633079523 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374032840 con 0x55ea1ecb38b0 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb3667fc640 1 -- 172.21.15.42:0/3633079523 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb3740329a0 con 0x55ea1ecb38b0 2024-03-12T04:16:08.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.439+0000 7fb3667fc640 1 -- 172.21.15.42:0/3633079523 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374034d50 con 0x55ea1ecb38b0 2024-03-12T04:16:08.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 -- 172.21.15.42:0/3633079523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecb38b0 msgr2=0x55ea1ecba880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecba880 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb37402c700 tx=0x7fb37404a040 comp rx=0 tx=0).stop 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 -- 172.21.15.42:0/3633079523 shutdown_connections 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecb38b0 0x55ea1ecba880 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaad30 0x55ea1eca3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 --2- 172.21.15.42:0/3633079523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 -- 172.21.15.42:0/3633079523 >> 172.21.15.42:0/3633079523 conn(0x55ea1eca4840 msgr2=0x55ea1eca3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 -- 172.21.15.42:0/3633079523 shutdown_connections 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 -- 172.21.15.42:0/3633079523 wait complete. 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 -- start start 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.440+0000 7fb385d63740 1 Processor -- start 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb385d63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb385d63740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb385d63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ed1a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb385d63740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ed1a890 con 0x55ea1ecaad30 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb385d63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ed1a9d0 con 0x55ea1ecaa380 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb385d63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1ed1ab10 con 0x55ea1ecb38b0 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb385d63740 10 client.0 _ll_register_callbacks cb 0x7fb370005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46338/0 (socket says 172.21.15.42:46338) 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 -- 172.21.15.42:0/2726824628 learned_addr learned my addr 172.21.15.42:0/2726824628 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb381652640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb381652640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ed1a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 -- 172.21.15.42:0/2726824628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 msgr2=0x55ea1ed1a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ed1a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 -- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaa380 msgr2=0x55ea1ecb9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb37bfff640 1 -- 172.21.15.42:0/2726824628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea1ebd40b0 con 0x55ea1ecaad30 2024-03-12T04:16:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.441+0000 7fb381652640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ed1a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb37bfff640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb9f90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb36800a010 tx=0x7fb368026a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b142a5bb58159db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb35bfff640 1 -- 172.21.15.42:0/2726824628 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368013ce0 con 0x55ea1ecaad30 2024-03-12T04:16:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb35bfff640 1 -- 172.21.15.42:0/2726824628 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb3680050c0 con 0x55ea1ecaad30 2024-03-12T04:16:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea1ec37900 con 0x55ea1ecaad30 2024-03-12T04:16:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb35bfff640 1 -- 172.21.15.42:0/2726824628 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb368005280 con 0x55ea1ecaad30 2024-03-12T04:16:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb385d63740 10 client.15882 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea1ebd2910 con 0x55ea1ecaad30 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb35bfff640 1 -- 172.21.15.42:0/2726824628 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ea1ec37900 con 0x55ea1ecaad30 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb35bfff640 1 -- 172.21.15.42:0/2726824628 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb36802f070 con 0x55ea1ecaad30 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb385d63740 10 client.15882 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.442+0000 7fb385d63740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.443+0000 7fb385d63740 20 client.15882 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.443+0000 7fb385d63740 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ed1c3f0 0x55ea1ed1e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.443+0000 7fb385d63740 4 client.15882 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:16:08.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.443+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ea1e911860 con 0x55ea1ed1c3f0 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.443+0000 7fb381652640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ed1c3f0 0x55ea1ed1e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.443+0000 7fb35bfff640 10 client.15882 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.443+0000 7fb381652640 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ed1c3f0 0x55ea1ed1e810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af350fdacc8755 server_cookie=6c0783771276f1f in_seq=0 out_seq=0 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.444+0000 7fb35bfff640 1 -- 172.21.15.42:0/2726824628 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ea1e911860 con 0x55ea1ed1c3f0 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.444+0000 7fb35bfff640 10 client.15882 handle_command_reply: tid=0 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.444+0000 7fb385d63740 1 client.15882 shutdown 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.444+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaad30 msgr2=0x55ea1ecb9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.444+0000 7fb385d63740 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb9f90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb36800a010 tx=0x7fb368026a60 comp rx=0 tx=0).stop 2024-03-12T04:16:08.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 shutdown_connections 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ea1ed1c3f0 0x55ea1ed1e810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1ecb38b0 0x55ea1ed1a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ea1ecaad30 0x55ea1ecb9f90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb35bfff640 0 client.15882 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 --2- 172.21.15.42:0/2726824628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1ecaa380 0x55ea1ecb9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 >> 172.21.15.42:0/2726824628 conn(0x55ea1eca4840 msgr2=0x55ea1eca38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 shutdown_connections 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 1 -- 172.21.15.42:0/2726824628 wait complete. 2024-03-12T04:16:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.445+0000 7fb385d63740 20 client.15882 trim_cache size 0 max 16384 2024-03-12T04:16:08.454 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:16:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:16:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "32af558a-5e39-462a-a660-b0b126f360ec", 2024-03-12T04:16:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:16:08.454 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb36c0a1d40 msgr2=0x7fb36c0a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb36c0a1d40 0x7fb36c0a4200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3700cb340 tx=0x7fb3700d15b0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c0685d0 msgr2=0x7fb37c1976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c0685d0 0x7fb37c1976d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e9f0 tx=0x7fb37000eeb0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 shutdown_connections 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb36c0a1d40 0x7fb36c0a4200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c19bf90 0x7fb37c19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb37c106430 0x7fb37c197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 --2- 172.21.15.42:0/967585520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb37c0685d0 0x7fb37c1976d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 >> 172.21.15.42:0/967585520 conn(0x7fb37c077180 msgr2=0x7fb37c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 shutdown_connections 2024-03-12T04:16:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.446+0000 7fb3830b5640 1 -- 172.21.15.42:0/967585520 wait complete. 2024-03-12T04:16:08.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:16:08.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 -- 172.21.15.42:0/3718344728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd7010b6e0 msgr2=0x7fbd7010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 --2- 172.21.15.42:0/3718344728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd7010b6e0 0x7fbd7010db80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ca0 tx=0x7fbd6802c180 comp rx=0 tx=0).stop 2024-03-12T04:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 -- 172.21.15.42:0/3718344728 shutdown_connections 2024-03-12T04:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 --2- 172.21.15.42:0/3718344728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd7010b6e0 0x7fbd7010db80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 --2- 172.21.15.42:0/3718344728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd70102cc0 0x7fbd7010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 --2- 172.21.15.42:0/3718344728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd701023a0 0x7fbd70102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 -- 172.21.15.42:0/3718344728 >> 172.21.15.42:0/3718344728 conn(0x7fbd70077230 msgr2=0x7fbd70077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 -- 172.21.15.42:0/3718344728 shutdown_connections 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.576+0000 7fbd77623640 1 -- 172.21.15.42:0/3718344728 wait complete. 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 Processor -- start 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 -- start start 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd701023a0 0x7fbd70197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd70102cc0 0x7fbd70197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7010b6e0 0x7fbd7019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd753bf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd701023a0 0x7fbd70197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7019c4f0 con 0x7fbd701023a0 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7019c660 con 0x7fbd7010b6e0 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd753bf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd701023a0 0x7fbd70197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46354/0 (socket says 172.21.15.42:46354) 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd77623640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7019c820 con 0x7fbd70102cc0 2024-03-12T04:16:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd753bf640 1 -- 172.21.15.42:0/4077133766 learned_addr learned my addr 172.21.15.42:0/4077133766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.577+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd70102cc0 0x7fbd70197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7010b6e0 0x7fbd7019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd753bf640 1 -- 172.21.15.42:0/4077133766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd70102cc0 msgr2=0x7fbd70197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd753bf640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd70102cc0 0x7fbd70197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd753bf640 1 -- 172.21.15.42:0/4077133766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7010b6e0 msgr2=0x7fbd7019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd753bf640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7010b6e0 0x7fbd7019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd753bf640 1 -- 172.21.15.42:0/4077133766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68009900 con 0x7fbd701023a0 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd70102cc0 0x7fbd70197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7010b6e0 0x7fbd7019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd753bf640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd701023a0 0x7fbd70197630 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e9f0 tx=0x7fbd6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd667fc640 1 -- 172.21.15.42:0/4077133766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c009720 con 0x7fbd701023a0 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7019cb00 con 0x7fbd701023a0 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd667fc640 1 -- 172.21.15.42:0/4077133766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd6c00cd80 con 0x7fbd701023a0 2024-03-12T04:16:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd667fc640 1 -- 172.21.15.42:0/4077133766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c010430 con 0x7fbd701023a0 2024-03-12T04:16:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.578+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd701a4380 con 0x7fbd701023a0 2024-03-12T04:16:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd667fc640 1 -- 172.21.15.42:0/4077133766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fbd6c005000 con 0x7fbd701023a0 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 Processor -- start 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 -- start start 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64558f60 0x562d64559340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64559880 0x562d64566680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64566bc0 0x562d64568fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d644c8280 con 0x562d64558f60 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d644c57b0 con 0x562d64566bc0 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d644c0f90 con 0x562d64559880 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd75bc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64558f60 0x562d64559340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd75bc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64558f60 0x562d64559340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46370/0 (socket says 172.21.15.42:46370) 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd75bc0640 1 -- 172.21.15.42:0/2310414259 learned_addr learned my addr 172.21.15.42:0/2310414259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd753bf640 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64559880 0x562d64566680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64566bc0 0x562d64568fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd75bc0640 1 -- 172.21.15.42:0/2310414259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64559880 msgr2=0x562d64566680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64559880 0x562d64566680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd75bc0640 1 -- 172.21.15.42:0/2310414259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64566bc0 msgr2=0x562d64568fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.582+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64566bc0 0x562d64568fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd75bc0640 1 -- 172.21.15.42:0/2310414259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6447f570 con 0x562d64558f60 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64558f60 0x562d64559340 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ca0 tx=0x7fbd68032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcff6f37d7c4ec2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd667fc640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd5c0a1cc0 0x7fbd5c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd5b7fe640 1 -- 172.21.15.42:0/2310414259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68034bc0 con 0x562d64558f60 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd753bf640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd5c0a1cc0 0x7fbd5c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd667fc640 1 -- 172.21.15.42:0/4077133766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbd6c014070 con 0x7fbd701023a0 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd5b7fe640 1 -- 172.21.15.42:0/2310414259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd68034d20 con 0x562d64558f60 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.583+0000 7fbd5b7fe640 1 -- 172.21.15.42:0/2310414259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68032680 con 0x562d64558f60 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/2310414259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64558f60 msgr2=0x562d64559340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd753bf640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd5c0a1cc0 0x7fbd5c0a4180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0cb560 tx=0x7fbd6c0d1360 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64558f60 0x562d64559340 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ca0 tx=0x7fbd68032340 comp rx=0 tx=0).stop 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/2310414259 shutdown_connections 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64566bc0 0x562d64568fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64559880 0x562d64566680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/2310414259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64558f60 0x562d64559340 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/2310414259 >> 172.21.15.42:0/2310414259 conn(0x562d64553990 msgr2=0x562d64553d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/2310414259 shutdown_connections 2024-03-12T04:16:08.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/2310414259 wait complete. 2024-03-12T04:16:08.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 -- start start 2024-03-12T04:16:08.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.584+0000 7fbd7a1f7740 1 Processor -- start 2024-03-12T04:16:08.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64559880 0x562d64563920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64566bc0 0x562d64563f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d64194430 con 0x562d64559880 2024-03-12T04:16:08.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d64150e00 con 0x562d64558f60 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d64445fd0 con 0x562d64566bc0 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd7a1f7740 10 client.0 _ll_register_callbacks cb 0x562d6449bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd74bbe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64559880 0x562d64563920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd74bbe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64559880 0x562d64563920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46384/0 (socket says 172.21.15.42:46384) 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd74bbe640 1 -- 172.21.15.42:0/615161063 learned_addr learned my addr 172.21.15.42:0/615161063 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64566bc0 0x562d64563f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd74bbe640 1 -- 172.21.15.42:0/615161063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64566bc0 msgr2=0x562d64563f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.585+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64566bc0 0x562d64563f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd74bbe640 1 -- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 msgr2=0x562d64566590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd74bbe640 1 -- 172.21.15.42:0/615161063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7010e800 con 0x562d64559880 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64559880 0x562d64563920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbd60012780 tx=0x7fbd60012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd383fc07aedb82 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd58ff9640 1 -- 172.21.15.42:0/615161063 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60010e00 con 0x562d64559880 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd58ff9640 1 -- 172.21.15.42:0/615161063 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbd6001c3f0 con 0x562d64559880 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d641c07e0 con 0x562d64559880 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd58ff9640 1 -- 172.21.15.42:0/615161063 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd6001c5b0 con 0x562d64559880 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd7a1f7740 10 client.15894 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562d644bf610 con 0x562d64559880 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.586+0000 7fbd58ff9640 1 -- 172.21.15.42:0/615161063 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd60029020 con 0x562d64559880 2024-03-12T04:16:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd58ff9640 1 -- 172.21.15.42:0/615161063 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbd600256b0 con 0x562d64559880 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd7a1f7740 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd7a1f7740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd7a1f7740 20 client.15894 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d64576e20 0x562d64579240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd7a1f7740 4 client.15894 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d641fd720 con 0x562d64576e20 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d64576e20 0x562d64579240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd58ff9640 10 client.15894 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.587+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d64576e20 0x562d64579240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1e3265e8889a48 server_cookie=69fdf3a697185698 in_seq=0 out_seq=0 2024-03-12T04:16:08.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd58ff9640 1 -- 172.21.15.42:0/615161063 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d641fd720 con 0x562d64576e20 2024-03-12T04:16:08.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd58ff9640 10 client.15894 handle_command_reply: tid=0 2024-03-12T04:16:08.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 client.15894 shutdown 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64559880 msgr2=0x562d64563920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64559880 0x562d64563920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbd60012780 tx=0x7fbd60012c40 comp rx=0 tx=0).stop 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 shutdown_connections 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d64576e20 0x562d64579240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64566bc0 0x562d64563f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd58ff9640 0 client.15894 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64559880 0x562d64563920 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/615161063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 >> 172.21.15.42:0/615161063 conn(0x562d64553990 msgr2=0x562d64555730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 shutdown_connections 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/615161063 wait complete. 2024-03-12T04:16:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.588+0000 7fbd7a1f7740 20 client.15894 trim_cache size 0 max 16384 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 Processor -- start 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 -- start start 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64566bc0 0x562d64566590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d645631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d641fd750 con 0x562d64566bc0 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d64554810 con 0x562d64558f60 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d64558bf0 con 0x562d64576e20 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd74bbe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64566bc0 0x562d64566590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd74bbe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64566bc0 0x562d64566590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46388/0 (socket says 172.21.15.42:46388) 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.604+0000 7fbd74bbe640 1 -- 172.21.15.42:0/69812367 learned_addr learned my addr 172.21.15.42:0/69812367 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d645631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd74bbe640 1 -- 172.21.15.42:0/69812367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 msgr2=0x562d645631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d645631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd74bbe640 1 -- 172.21.15.42:0/69812367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 msgr2=0x562d64566050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd74bbe640 1 -- 172.21.15.42:0/69812367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6445bac0 con 0x562d64566bc0 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64566bc0 0x562d64566590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbd60025a90 tx=0x7fbd6000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecac08abf22e4a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd5affd640 1 -- 172.21.15.42:0/69812367 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60033070 con 0x562d64566bc0 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd5affd640 1 -- 172.21.15.42:0/69812367 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd60029040 con 0x562d64566bc0 2024-03-12T04:16:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.605+0000 7fbd5affd640 1 -- 172.21.15.42:0/69812367 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60005040 con 0x562d64566bc0 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/69812367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64566bc0 msgr2=0x562d64566590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64566bc0 0x562d64566590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbd60025a90 tx=0x7fbd6000bd30 comp rx=0 tx=0).stop 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/69812367 shutdown_connections 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d645631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d64566bc0 0x562d64566590 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/69812367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d64566050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/69812367 >> 172.21.15.42:0/69812367 conn(0x562d64553990 msgr2=0x562d645621f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/69812367 shutdown_connections 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/69812367 wait complete. 2024-03-12T04:16:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 -- start start 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.606+0000 7fbd7a1f7740 1 Processor -- start 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d6456ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d6456b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd7a1f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d645c72f0 0x562d645c7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d645c7cb0 con 0x562d645c72f0 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d645c7e20 con 0x562d64558f60 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd7a1f7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d645c7fe0 con 0x562d64576e20 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd7a1f7740 10 client.0 _ll_register_callbacks cb 0x7fbd68037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d645c72f0 0x562d645c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d645c72f0 0x562d645c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46392/0 (socket says 172.21.15.42:46392) 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 -- 172.21.15.42:0/1082717631 learned_addr learned my addr 172.21.15.42:0/1082717631 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d6456b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d6456ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 -- 172.21.15.42:0/1082717631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 msgr2=0x562d6456b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d6456b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 -- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 msgr2=0x562d6456ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d6456ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd75bc0640 1 -- 172.21.15.42:0/1082717631 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d644bf610 con 0x562d645c72f0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.607+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d6456b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd75bc0640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d645c72f0 0x562d645c7770 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbd6802c6c0 tx=0x7fbd6802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d283a86a3b063b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd58ff9640 1 -- 172.21.15.42:0/1082717631 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68034650 con 0x562d645c72f0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd58ff9640 1 -- 172.21.15.42:0/1082717631 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbd6805d030 con 0x562d645c72f0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd68034d20 con 0x562d645c72f0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd58ff9640 1 -- 172.21.15.42:0/1082717631 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd680347b0 con 0x562d645c72f0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd7a1f7740 10 client.15900 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562d644816a0 con 0x562d645c72f0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.608+0000 7fbd58ff9640 1 -- 172.21.15.42:0/1082717631 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd6805c020 con 0x562d645c72f0 2024-03-12T04:16:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd58ff9640 1 -- 172.21.15.42:0/1082717631 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbd68053070 con 0x562d645c72f0 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd7a1f7740 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd7a1f7740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd7a1f7740 20 client.15900 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d645cf440 0x562d645d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd7a1f7740 4 client.15900 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fbd68034d20 con 0x562d645cf440 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d645cf440 0x562d645d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd58ff9640 10 client.15900 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.609+0000 7fbd74bbe640 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d645cf440 0x562d645d1830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114e97d8b86c017c server_cookie=95dea52ec0e531dc in_seq=0 out_seq=0 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd58ff9640 1 -- 172.21.15.42:0/1082717631 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fbd68034d20 con 0x562d645cf440 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd58ff9640 10 client.15900 handle_command_reply: tid=0 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 client.15900 shutdown 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d645c72f0 msgr2=0x562d645c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d645c72f0 0x562d645c7770 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbd6802c6c0 tx=0x7fbd6802e040 comp rx=0 tx=0).stop 2024-03-12T04:16:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 shutdown_connections 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562d645cf440 0x562d645d1830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562d645c72f0 0x562d645c7770 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d64576e20 0x562d6456b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd58ff9640 0 client.15900 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 --2- 172.21.15.42:0/1082717631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d64558f60 0x562d6456ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 >> 172.21.15.42:0/1082717631 conn(0x562d64553990 msgr2=0x562d645631b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 shutdown_connections 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 1 -- 172.21.15.42:0/1082717631 wait complete. 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.610+0000 7fbd7a1f7740 20 client.15900 trim_cache size 0 max 16384 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:16:08.617 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stdout: "32af558a-5e39-462a-a660-b0b126f360ec": { 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "32af558a-5e39-462a-a660-b0b126f360ec", 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:16:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd5c0a1cc0 msgr2=0x7fbd5c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd5c0a1cc0 0x7fbd5c0a4180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0cb560 tx=0x7fbd6c0d1360 comp rx=0 tx=0).stop 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd701023a0 msgr2=0x7fbd70197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd701023a0 0x7fbd70197630 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e9f0 tx=0x7fbd6c00edf0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 shutdown_connections 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd5c0a1cc0 0x7fbd5c0a4180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7010b6e0 0x7fbd7019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd70102cc0 0x7fbd70197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 --2- 172.21.15.42:0/4077133766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd701023a0 0x7fbd70197630 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 >> 172.21.15.42:0/4077133766 conn(0x7fbd70077230 msgr2=0x7fbd700fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 shutdown_connections 2024-03-12T04:16:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:08.611+0000 7fbd77623640 1 -- 172.21.15.42:0/4077133766 wait complete. 2024-03-12T04:16:08.624 INFO:tasks.cephfs.filesystem:scrub status for tag:32af558a-5e39-462a-a660-b0b126f360ec - {'path': '/', 'tag': '32af558a-5e39-462a-a660-b0b126f360ec', 'options': 'recursive,force'} 2024-03-12T04:16:09.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:08 smithi042 ceph-mon[32260]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:09.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:08 smithi119 ceph-mon[36119]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:09.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:08 smithi136 ceph-mon[35571]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:09 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:16:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:09 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:09 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:16:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:09 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:10.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:09 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:16:10.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:09 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:16:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:09 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:09 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:16:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:09 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:10.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:09 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:16:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:09 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:16:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:09 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:09 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:16:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:09 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:10.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:09 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:16:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:10 smithi042 ceph-mon[32260]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:11.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:10 smithi119 ceph-mon[36119]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:11.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:10 smithi136 ceph-mon[35571]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:12 smithi042 ceph-mon[32260]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:13.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:12 smithi119 ceph-mon[36119]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:13.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:12 smithi136 ceph-mon[35571]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-12T04:16:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:14 smithi042 ceph-mon[32260]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:15.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:14 smithi119 ceph-mon[36119]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:15.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:14 smithi136 ceph-mon[35571]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-12T04:16:16.879 INFO:tasks.workunit.client.0.smithi042.stderr:Updating files: 96% (11923/12344) Updating files: 97% (11974/12344) Updating files: 98% (12098/12344) Updating files: 99% (12221/12344) Updating files: 100% (12344/12344) Updating files: 100% (12344/12344), done. 2024-03-12T04:16:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:16 smithi042 ceph-mon[32260]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:16:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:16:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:16 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:16 smithi119 ceph-mon[36119]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:16:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:16:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:16 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:16 smithi136 ceph-mon[35571]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:16:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:16:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:16 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr:Note: switching to 'b6a4997be2b8bb74411268cc42753f29ae9a78ad'. 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr:state without impacting any branches by switching back to a branch. 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-12T04:16:17.606 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr: git switch -c 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr:Or undo this operation with: 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr: git switch - 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:17.607 INFO:tasks.workunit.client.0.smithi042.stderr:HEAD is now at b6a4997be2b Merge pull request #56060 from rhcs-dashboard/wip-64808-squid 2024-03-12T04:16:17.609 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-12T04:16:17.643 INFO:tasks.workunit.client.0.smithi042.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-12T04:16:17.645 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-12T04:16:17.645 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-12T04:16:18.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:17 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:16:18.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:17 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:16:18.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:17 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:18.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:17 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:16:18.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:17 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:16:18.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:17 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:18.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:17 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:16:18.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:17 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:16:18.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:17 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:16:18.378 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-12T04:16:18.452 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-12T04:16:18.493 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-12T04:16:18.494 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-12T04:16:18.494 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-12T04:16:18.533 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-12T04:16:18.536 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:16:18.536 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-12T04:16:18.592 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-03-12T04:16:18.593 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-03-12T04:16:18.593 DEBUG:teuthology.orchestra.run.smithi042:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-12T04:16:18.656 INFO:tasks.workunit.client.0.smithi042.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-03-12T04:16:18.679 INFO:tasks.workunit.client.0.smithi042.stderr:--2024-03-12 04:16:18-- http://download.ceph.com/qa/fsync-tester.c 2024-03-12T04:16:18.680 INFO:tasks.workunit.client.0.smithi042.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-12T04:16:18.701 INFO:tasks.workunit.client.0.smithi042.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-12T04:16:18.722 INFO:tasks.workunit.client.0.smithi042.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-12T04:16:18.722 INFO:tasks.workunit.client.0.smithi042.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-03-12T04:16:18.725 INFO:tasks.workunit.client.0.smithi042.stderr:Saving to: ‘fsync-tester.c’ 2024-03-12T04:16:18.728 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:18.729 INFO:tasks.workunit.client.0.smithi042.stderr: 0K .. 100% 589M=0s 2024-03-12T04:16:18.729 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:18.729 INFO:tasks.workunit.client.0.smithi042.stderr:2024-03-12 04:16:18 (589 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-03-12T04:16:18.729 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-03-12T04:16:18.729 INFO:tasks.workunit.client.0.smithi042.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-03-12T04:16:18.804 INFO:tasks.workunit.client.0.smithi042.stderr:+ ./fsync-tester 2024-03-12T04:16:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:18 smithi042 ceph-mon[32260]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:16:19.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:18 smithi119 ceph-mon[36119]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:16:19.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:18 smithi136 ceph-mon[35571]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:16:21.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:20 smithi042 ceph-mon[32260]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:16:21.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:20 smithi119 ceph-mon[36119]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:16:21.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:20 smithi136 ceph-mon[35571]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:16:23.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:22 smithi042 ceph-mon[32260]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:16:23.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:22 smithi119 ceph-mon[36119]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:16:23.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:22 smithi136 ceph-mon[35571]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:16:25.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:24 smithi042 ceph-mon[32260]: pgmap v39: 97 pgs: 97 active+clean; 93 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 17 op/s 2024-03-12T04:16:25.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:24 smithi119 ceph-mon[36119]: pgmap v39: 97 pgs: 97 active+clean; 93 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 17 op/s 2024-03-12T04:16:25.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:24 smithi136 ceph-mon[35571]: pgmap v39: 97 pgs: 97 active+clean; 93 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 17 op/s 2024-03-12T04:16:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:26 smithi042 ceph-mon[32260]: pgmap v40: 97 pgs: 97 active+clean; 93 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 12 op/s 2024-03-12T04:16:27.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:26 smithi119 ceph-mon[36119]: pgmap v40: 97 pgs: 97 active+clean; 93 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 12 op/s 2024-03-12T04:16:27.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:26 smithi136 ceph-mon[35571]: pgmap v40: 97 pgs: 97 active+clean; 93 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 12 op/s 2024-03-12T04:16:29.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:28 smithi042 ceph-mon[32260]: pgmap v41: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-03-12T04:16:29.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:28 smithi119 ceph-mon[36119]: pgmap v41: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-03-12T04:16:29.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:28 smithi136 ceph-mon[35571]: pgmap v41: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-03-12T04:16:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:30 smithi042 ceph-mon[32260]: pgmap v42: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-03-12T04:16:31.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:30 smithi119 ceph-mon[36119]: pgmap v42: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-03-12T04:16:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:30 smithi136 ceph-mon[35571]: pgmap v42: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-03-12T04:16:33.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:32 smithi042 ceph-mon[32260]: pgmap v43: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-03-12T04:16:33.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:32 smithi119 ceph-mon[36119]: pgmap v43: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-03-12T04:16:33.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:32 smithi136 ceph-mon[35571]: pgmap v43: 97 pgs: 97 active+clean; 237 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-03-12T04:16:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:34 smithi042 ceph-mon[32260]: pgmap v44: 97 pgs: 97 active+clean; 258 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 1.61k op/s 2024-03-12T04:16:35.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:34 smithi119 ceph-mon[36119]: pgmap v44: 97 pgs: 97 active+clean; 258 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 1.61k op/s 2024-03-12T04:16:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:34 smithi136 ceph-mon[35571]: pgmap v44: 97 pgs: 97 active+clean; 258 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 1.61k op/s 2024-03-12T04:16:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:36 smithi042 ceph-mon[32260]: pgmap v45: 97 pgs: 97 active+clean; 258 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 1.60k op/s 2024-03-12T04:16:37.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:36 smithi119 ceph-mon[36119]: pgmap v45: 97 pgs: 97 active+clean; 258 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 1.60k op/s 2024-03-12T04:16:37.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:36 smithi136 ceph-mon[35571]: pgmap v45: 97 pgs: 97 active+clean; 258 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 1.60k op/s 2024-03-12T04:16:38.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:16:38.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:16:38.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 -- 172.21.15.42:0/3988288846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac0ff770 msgr2=0x7fcaac0ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/3988288846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac0ff770 0x7fcaac0ffbf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009be0 tx=0x7fcaa402bef0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 -- 172.21.15.42:0/3988288846 shutdown_connections 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/3988288846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac100130 0x7fcaac10c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/3988288846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac0ff770 0x7fcaac0ffbf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/3988288846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac105780 0x7fcaac105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 -- 172.21.15.42:0/3988288846 >> 172.21.15.42:0/3988288846 conn(0x7fcaac0765b0 msgr2=0x7fcaac0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 -- 172.21.15.42:0/3988288846 shutdown_connections 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.783+0000 7fcab3cc3640 1 -- 172.21.15.42:0/3988288846 wait complete. 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.784+0000 7fcab3cc3640 1 Processor -- start 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.784+0000 7fcab3cc3640 1 -- start start 2024-03-12T04:16:38.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.784+0000 7fcab3cc3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac0ff770 0x7fcaac195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.784+0000 7fcab3cc3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac100130 0x7fcaac1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac0ff770 0x7fcaac195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab3cc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac105780 0x7fcaac199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab3cc3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac19a2e0 con 0x7fcaac100130 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab3cc3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac19a450 con 0x7fcaac105780 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab3cc3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac19a610 con 0x7fcaac0ff770 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac0ff770 0x7fcaac195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33702/0 (socket says 172.21.15.42:33702) 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 -- 172.21.15.42:0/428315630 learned_addr learned my addr 172.21.15.42:0/428315630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab125e640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac100130 0x7fcaac1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab2260640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac105780 0x7fcaac199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 -- 172.21.15.42:0/428315630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac105780 msgr2=0x7fcaac199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac105780 0x7fcaac199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 -- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac100130 msgr2=0x7fcaac1959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac100130 0x7fcaac1959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.785+0000 7fcab1a5f640 1 -- 172.21.15.42:0/428315630 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4009840 con 0x7fcaac0ff770 2024-03-12T04:16:38.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.786+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac0ff770 0x7fcaac195470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00e9c0 tx=0x7fca9c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:38.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.786+0000 7fcaa2ffd640 1 -- 172.21.15.42:0/428315630 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00ccc0 con 0x7fcaac0ff770 2024-03-12T04:16:38.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.787+0000 7fcaa2ffd640 1 -- 172.21.15.42:0/428315630 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca9c00ce20 con 0x7fcaac0ff770 2024-03-12T04:16:38.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.787+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac19a8f0 con 0x7fcaac0ff770 2024-03-12T04:16:38.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.787+0000 7fcaa2ffd640 1 -- 172.21.15.42:0/428315630 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c010430 con 0x7fcaac0ff770 2024-03-12T04:16:38.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.787+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac1a2170 con 0x7fcaac0ff770 2024-03-12T04:16:38.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.791+0000 7fcab6972740 1 Processor -- start 2024-03-12T04:16:38.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.791+0000 7fcab6972740 1 -- start start 2024-03-12T04:16:38.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab6972740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcaa2ffd640 1 -- 172.21.15.42:0/428315630 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fca9c004ee0 con 0x7fcaac0ff770 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab6972740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a035a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab6972740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a0383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab6972740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269f93280 con 0x55a26a0240c0 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab6972740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269f907b0 con 0x55a26a024a70 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab6972740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269f8bf90 con 0x55a26a035fc0 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab2260640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab2260640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0244a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44976/0 (socket says 172.21.15.42:44976) 2024-03-12T04:16:38.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.792+0000 7fcab2260640 1 -- 172.21.15.42:0/2440577720 learned_addr learned my addr 172.21.15.42:0/2440577720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:38.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab2260640 1 -- 172.21.15.42:0/2440577720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 msgr2=0x55a26a0383b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T04:16:38.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a035a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcaa2ffd640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fca980a1ea0 0x7fca980a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab2260640 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a0383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab2260640 1 -- 172.21.15.42:0/2440577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 msgr2=0x55a26a035a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab2260640 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a035a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab2260640 1 -- 172.21.15.42:0/2440577720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a269f4a570 con 0x55a26a0240c0 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a035a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcaa2ffd640 1 -- 172.21.15.42:0/428315630 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fca9c014070 con 0x7fcaac0ff770 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab2260640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fca980a1ea0 0x7fca980a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.793+0000 7fcab2260640 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0244a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009c10 tx=0x7fcaa80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71e4dfa70ac82ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.794+0000 7fca97fff640 1 -- 172.21.15.42:0/2440577720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8021070 con 0x55a26a0240c0 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.794+0000 7fca97fff640 1 -- 172.21.15.42:0/2440577720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcaa8012b40 con 0x55a26a0240c0 2024-03-12T04:16:38.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.794+0000 7fca97fff640 1 -- 172.21.15.42:0/2440577720 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa801cd90 con 0x55a26a0240c0 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 -- 172.21.15.42:0/2440577720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 msgr2=0x55a26a0244a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0244a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009c10 tx=0x7fcaa80117b0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab2260640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fca980a1ea0 0x7fca980a4360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8020000 tx=0x7fcaa800f620 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 -- 172.21.15.42:0/2440577720 shutdown_connections 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a0383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a035a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 --2- 172.21.15.42:0/2440577720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0244a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 -- 172.21.15.42:0/2440577720 >> 172.21.15.42:0/2440577720 conn(0x55a26a01e990 msgr2=0x55a26a01ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 -- 172.21.15.42:0/2440577720 shutdown_connections 2024-03-12T04:16:38.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 -- 172.21.15.42:0/2440577720 wait complete. 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.795+0000 7fcab6972740 1 -- start start 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 1 Processor -- start 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a0240c0 0x55a26a02f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a024a70 0x55a26a02fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a035fc0 0x55a26a0353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269c5f430 con 0x55a26a035fc0 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269c1be00 con 0x55a26a0240c0 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269f10fd0 con 0x55a26a024a70 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab6972740 10 client.0 _ll_register_callbacks cb 0x55a269f66fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab2260640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a035fc0 0x55a26a0353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab2260640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a035fc0 0x55a26a0353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44980/0 (socket says 172.21.15.42:44980) 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab2260640 1 -- 172.21.15.42:0/2043135161 learned_addr learned my addr 172.21.15.42:0/2043135161 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a0240c0 0x55a26a02f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.796+0000 7fcab125e640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a024a70 0x55a26a02fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab1a5f640 1 -- 172.21.15.42:0/2043135161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a024a70 msgr2=0x55a26a02fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a024a70 0x55a26a02fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab1a5f640 1 -- 172.21.15.42:0/2043135161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a035fc0 msgr2=0x55a26a0353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a035fc0 0x55a26a0353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab1a5f640 1 -- 172.21.15.42:0/2043135161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa80098c0 con 0x55a26a0240c0 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab2260640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a035fc0 0x55a26a0353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a0240c0 0x55a26a02f630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0cb210 tx=0x7fca9c0094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c2dcc37f4c881f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fca957fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fca957fa640 1 -- 172.21.15.42:0/2043135161 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c09a620 con 0x55a26a0240c0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a269c8b7e0 con 0x55a26a0240c0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.797+0000 7fca957fa640 1 -- 172.21.15.42:0/2043135161 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fca9c096040 con 0x55a26a0240c0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fca957fa640 1 -- 172.21.15.42:0/2043135161 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c09e770 con 0x55a26a0240c0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 10 client.25403 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a269f8a610 con 0x55a26a0240c0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fca957fa640 1 -- 172.21.15.42:0/2043135161 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a269c8b7e0 con 0x55a26a0240c0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fca957fa640 1 -- 172.21.15.42:0/2043135161 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca9c01f060 con 0x55a26a0240c0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 10 client.25403 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 20 client.25403 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a04e8e0 0x55a26a050d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 4 client.25403 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a269cc8720 con 0x55a26a04e8e0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.798+0000 7fcab125e640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a04e8e0 0x55a26a050d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.799+0000 7fcab125e640 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a04e8e0 0x55a26a050d00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1efcedf0631a35a server_cookie=990f8061ad2faae0 in_seq=0 out_seq=0 2024-03-12T04:16:38.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.799+0000 7fca957fa640 10 client.25403 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fca957fa640 1 -- 172.21.15.42:0/2043135161 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a269cc8720 con 0x55a26a04e8e0 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fca957fa640 10 client.25403 handle_command_reply: tid=0 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 client.25403 shutdown 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a0240c0 msgr2=0x55a26a02f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a0240c0 0x55a26a02f630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0cb210 tx=0x7fca9c0094c0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 shutdown_connections 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a04e8e0 0x55a26a050d00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a035fc0 0x55a26a0353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a024a70 0x55a26a02fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fca957fa640 0 client.25403 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 --2- 172.21.15.42:0/2043135161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a0240c0 0x55a26a02f630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.800+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 >> 172.21.15.42:0/2043135161 conn(0x55a26a01e990 msgr2=0x55a26a020730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.801+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 shutdown_connections 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.801+0000 7fcab6972740 1 -- 172.21.15.42:0/2043135161 wait complete. 2024-03-12T04:16:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.801+0000 7fcab6972740 20 client.25403 trim_cache size 0 max 16384 2024-03-12T04:16:38.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 Processor -- start 2024-03-12T04:16:38.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 -- start start 2024-03-12T04:16:38.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a0240c0 0x55a26a035310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a024a70 0x55a26a029350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a035fc0 0x55a26a029890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26a029dd0 con 0x55a26a024a70 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26a01f9e0 con 0x55a26a035fc0 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab6972740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26a055ed0 con 0x55a26a0240c0 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab125e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a024a70 0x55a26a029350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab125e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a024a70 0x55a26a029350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44994/0 (socket says 172.21.15.42:44994) 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab125e640 1 -- 172.21.15.42:0/2799776541 learned_addr learned my addr 172.21.15.42:0/2799776541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab2260640 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a035fc0 0x55a26a029890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.824+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a0240c0 0x55a26a035310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fcab125e640 1 -- 172.21.15.42:0/2799776541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a0240c0 msgr2=0x55a26a035310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fcab125e640 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a0240c0 0x55a26a035310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fcab125e640 1 -- 172.21.15.42:0/2799776541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a035fc0 msgr2=0x55a26a029890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fcab125e640 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a035fc0 0x55a26a029890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fcab125e640 1 -- 172.21.15.42:0/2799776541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a269f26ac0 con 0x55a26a024a70 2024-03-12T04:16:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a0240c0 0x55a26a035310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:38.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fcab125e640 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a024a70 0x55a26a029350 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005cc0 tx=0x7fcaa4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25bc62b24f2a148 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:38.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fca977fe640 1 -- 172.21.15.42:0/2799776541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4047d70 con 0x55a26a024a70 2024-03-12T04:16:38.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fca977fe640 1 -- 172.21.15.42:0/2799776541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcaa4033a10 con 0x55a26a024a70 2024-03-12T04:16:38.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.825+0000 7fca977fe640 1 -- 172.21.15.42:0/2799776541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4003dd0 con 0x55a26a024a70 2024-03-12T04:16:38.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 -- 172.21.15.42:0/2799776541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a024a70 msgr2=0x55a26a029350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a024a70 0x55a26a029350 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005cc0 tx=0x7fcaa4005e30 comp rx=0 tx=0).stop 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 -- 172.21.15.42:0/2799776541 shutdown_connections 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a035fc0 0x55a26a029890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a024a70 0x55a26a029350 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 --2- 172.21.15.42:0/2799776541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a0240c0 0x55a26a035310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 -- 172.21.15.42:0/2799776541 >> 172.21.15.42:0/2799776541 conn(0x55a26a01e990 msgr2=0x55a26a025d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 -- 172.21.15.42:0/2799776541 shutdown_connections 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 -- 172.21.15.42:0/2799776541 wait complete. 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.826+0000 7fcab6972740 1 -- start start 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 1 Processor -- start 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a031f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a032470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26a0a4e90 con 0x55a26a0240c0 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26a0a5000 con 0x55a26a024a70 2024-03-12T04:16:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26a0a51c0 con 0x55a26a035fc0 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab6972740 10 client.0 _ll_register_callbacks cb 0x7fcaa80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab1a5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab1a5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0319f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45004/0 (socket says 172.21.15.42:45004) 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab1a5f640 1 -- 172.21.15.42:0/2423350206 learned_addr learned my addr 172.21.15.42:0/2423350206 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab125e640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a031f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.827+0000 7fcab2260640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab1a5f640 1 -- 172.21.15.42:0/2423350206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 msgr2=0x55a26a032470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a032470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab1a5f640 1 -- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 msgr2=0x55a26a031f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a031f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab1a5f640 1 -- 172.21.15.42:0/2423350206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac067dd0 con 0x55a26a0240c0 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab125e640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a031f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab2260640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a032470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab1a5f640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0319f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca9c096ff0 tx=0x7fca9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc39d9b6032e20d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fca957fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fca957fa640 1 -- 172.21.15.42:0/2423350206 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0a7080 con 0x55a26a0240c0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fca957fa640 1 -- 172.21.15.42:0/2423350206 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fca9c096040 con 0x55a26a0240c0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaa8012b40 con 0x55a26a0240c0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fca957fa640 1 -- 172.21.15.42:0/2423350206 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c0968c0 con 0x55a26a0240c0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab6972740 10 client.15918 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a269f4c6a0 con 0x55a26a0240c0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.828+0000 7fca957fa640 1 -- 172.21.15.42:0/2423350206 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcaa8012b40 con 0x55a26a0240c0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fca957fa640 1 -- 172.21.15.42:0/2423350206 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca9c01f070 con 0x55a26a0240c0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab6972740 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab6972740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab6972740 20 client.15918 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab6972740 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a0a65a0 0x55a26a0a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab6972740 4 client.15918 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a269d17890 con 0x55a26a0a65a0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab125e640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a0a65a0 0x55a26a0a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fca957fa640 10 client.15918 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab125e640 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a0a65a0 0x55a26a0a89c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e263157a24c6725 server_cookie=b49dcb2af49681e3 in_seq=0 out_seq=0 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fca957fa640 1 -- 172.21.15.42:0/2423350206 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a269d17890 con 0x55a26a0a65a0 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fca957fa640 10 client.15918 handle_command_reply: tid=0 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.829+0000 7fcab6972740 1 client.15918 shutdown 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 msgr2=0x55a26a0319f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0319f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca9c096ff0 tx=0x7fca9c005880 comp rx=0 tx=0).stop 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 shutdown_connections 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a26a0a65a0 0x55a26a0a89c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26a035fc0 0x55a26a032470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a26a024a70 0x55a26a031f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 --2- 172.21.15.42:0/2423350206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a26a0240c0 0x55a26a0319f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 >> 172.21.15.42:0/2423350206 conn(0x55a26a01e990 msgr2=0x55a26a025d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fca957fa640 0 client.15918 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 shutdown_connections 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 1 -- 172.21.15.42:0/2423350206 wait complete. 2024-03-12T04:16:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab6972740 20 client.15918 trim_cache size 0 max 16384 2024-03-12T04:16:38.837 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:16:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:16:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:16:38.837 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fca980a1ea0 msgr2=0x7fca980a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fca980a1ea0 0x7fca980a4360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8020000 tx=0x7fcaa800f620 comp rx=0 tx=0).stop 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac0ff770 msgr2=0x7fcaac195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac0ff770 0x7fcaac195470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00e9c0 tx=0x7fca9c00ee80 comp rx=0 tx=0).stop 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 shutdown_connections 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fca980a1ea0 0x7fca980a4360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac105780 0x7fcaac199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaac100130 0x7fcaac1959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 --2- 172.21.15.42:0/428315630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaac0ff770 0x7fcaac195470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.830+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 >> 172.21.15.42:0/428315630 conn(0x7fcaac0765b0 msgr2=0x7fcaac1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:38.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.831+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 shutdown_connections 2024-03-12T04:16:38.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.831+0000 7fcab3cc3640 1 -- 172.21.15.42:0/428315630 wait complete. 2024-03-12T04:16:38.844 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:16:38.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:16:38.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.991+0000 7f0c7496b640 1 -- 172.21.15.42:0/3284150557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70069ac0 msgr2=0x7f0c70069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.991+0000 7f0c7496b640 1 --2- 172.21.15.42:0/3284150557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70069ac0 0x7f0c70069ea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009930 tx=0x7f0c6002bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.992+0000 7f0c7496b640 1 -- 172.21.15.42:0/3284150557 shutdown_connections 2024-03-12T04:16:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.992+0000 7f0c7496b640 1 --2- 172.21.15.42:0/3284150557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c701064e0 0x7f0c7010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.992+0000 7f0c7496b640 1 --2- 172.21.15.42:0/3284150557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c7006a470 0x7f0c70105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.992+0000 7f0c7496b640 1 --2- 172.21.15.42:0/3284150557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70069ac0 0x7f0c70069ea0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.992+0000 7f0c7496b640 1 -- 172.21.15.42:0/3284150557 >> 172.21.15.42:0/3284150557 conn(0x7f0c70077150 msgr2=0x7f0c70077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.992+0000 7f0c7496b640 1 -- 172.21.15.42:0/3284150557 shutdown_connections 2024-03-12T04:16:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.993+0000 7f0c7496b640 1 -- 172.21.15.42:0/3284150557 wait complete. 2024-03-12T04:16:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 Processor -- start 2024-03-12T04:16:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 -- start start 2024-03-12T04:16:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c70069ac0 0x7f0c70100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c701064e0 0x7f0c701005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70100ae0 0x7f0c701a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c701a5cf0 con 0x7f0c70100ae0 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c701a5e60 con 0x7f0c701064e0 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c7496b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c701a6020 con 0x7f0c70069ac0 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c6e59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c70069ac0 0x7f0c70100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c6e59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c70069ac0 0x7f0c70100060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33734/0 (socket says 172.21.15.42:33734) 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c6e59c640 1 -- 172.21.15.42:0/1581753892 learned_addr learned my addr 172.21.15.42:0/1581753892 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c6ed9d640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70100ae0 0x7f0c701a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.994+0000 7f0c67fff640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c701064e0 0x7f0c701005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c67fff640 1 -- 172.21.15.42:0/1581753892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c70069ac0 msgr2=0x7f0c70100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c67fff640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c70069ac0 0x7f0c70100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c67fff640 1 -- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70100ae0 msgr2=0x7f0c701a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c67fff640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70100ae0 0x7f0c701a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c67fff640 1 -- 172.21.15.42:0/1581753892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60009560 con 0x7f0c701064e0 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c6e59c640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c70069ac0 0x7f0c70100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c67fff640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c701064e0 0x7f0c701005a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c701022b0 tx=0x7f0c5800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c677fe640 1 -- 172.21.15.42:0/1581753892 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800cc40 con 0x7f0c701064e0 2024-03-12T04:16:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c677fe640 1 -- 172.21.15.42:0/1581753892 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0c5800cda0 con 0x7f0c701064e0 2024-03-12T04:16:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.995+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c701a6300 con 0x7f0c701064e0 2024-03-12T04:16:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.996+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c701a67a0 con 0x7f0c701064e0 2024-03-12T04:16:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.996+0000 7f0c677fe640 1 -- 172.21.15.42:0/1581753892 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580187a0 con 0x7f0c701064e0 2024-03-12T04:16:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.997+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c50005290 con 0x7f0c701064e0 2024-03-12T04:16:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.998+0000 7f0c677fe640 1 -- 172.21.15.42:0/1581753892 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f0c58018900 con 0x7f0c701064e0 2024-03-12T04:16:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.999+0000 7f0c677fe640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0c5c0a1d70 0x7f0c5c0a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.999+0000 7f0c677fe640 1 -- 172.21.15.42:0/1581753892 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0c58014070 con 0x7f0c701064e0 2024-03-12T04:16:39.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:38.999+0000 7f0c6e59c640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0c5c0a1d70 0x7f0c5c0a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.000+0000 7f0c6e59c640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0c5c0a1d70 0x7f0c5c0a4230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c60005ab0 tx=0x7f0c60000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.003+0000 7f0c677fe640 1 -- 172.21.15.42:0/1581753892 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c58052d20 con 0x7f0c701064e0 2024-03-12T04:16:39.065 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:38 smithi042 ceph-mon[32260]: pgmap v46: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 3.45k op/s 2024-03-12T04:16:39.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:38 smithi119 ceph-mon[36119]: pgmap v46: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 3.45k op/s 2024-03-12T04:16:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:38 smithi136 ceph-mon[35571]: pgmap v46: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 3.45k op/s 2024-03-12T04:16:39.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.383+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c50005810 con 0x7f0c701064e0 2024-03-12T04:16:39.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.384+0000 7f0c677fe640 1 -- 172.21.15.42:0/1581753892 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f0c5809a4d0 con 0x7f0c701064e0 2024-03-12T04:16:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:16:39.387 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:16:39.387 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:16:39.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0c5c0a1d70 msgr2=0x7f0c5c0a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0c5c0a1d70 0x7f0c5c0a4230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c60005ab0 tx=0x7f0c60000bc0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c701064e0 msgr2=0x7f0c701005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c701064e0 0x7f0c701005a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c701022b0 tx=0x7f0c5800ee50 comp rx=0 tx=0).stop 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 shutdown_connections 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0c5c0a1d70 0x7f0c5c0a4230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c70100ae0 0x7f0c701a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c701064e0 0x7f0c701005a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 --2- 172.21.15.42:0/1581753892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c70069ac0 0x7f0c70100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 >> 172.21.15.42:0/1581753892 conn(0x7f0c70077150 msgr2=0x7f0c700fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 shutdown_connections 2024-03-12T04:16:39.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.387+0000 7f0c7496b640 1 -- 172.21.15.42:0/1581753892 wait complete. 2024-03-12T04:16:39.405 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:39.405 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 -- 172.21.15.42:0/3015662108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04101d30 msgr2=0x7fbe04105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 --2- 172.21.15.42:0/3015662108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04101d30 0x7fbe04105d70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009930 tx=0x7fbdf802bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 -- 172.21.15.42:0/3015662108 shutdown_connections 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 --2- 172.21.15.42:0/3015662108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe04106450 0x7fbe0410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 --2- 172.21.15.42:0/3015662108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04101d30 0x7fbe04105d70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 --2- 172.21.15.42:0/3015662108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe04101380 0x7fbe04101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 -- 172.21.15.42:0/3015662108 >> 172.21.15.42:0/3015662108 conn(0x7fbe04078540 msgr2=0x7fbe04078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 -- 172.21.15.42:0/3015662108 shutdown_connections 2024-03-12T04:16:39.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.562+0000 7fbe08817640 1 -- 172.21.15.42:0/3015662108 wait complete. 2024-03-12T04:16:39.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 Processor -- start 2024-03-12T04:16:39.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 -- start start 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe04101380 0x7fbe04075080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04106450 0x7fbe04073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe04073a10 0x7fbe04073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041aa110 con 0x7fbe04106450 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041aa280 con 0x7fbe04073a10 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe08817640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041aa440 con 0x7fbe04101380 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04106450 0x7fbe04073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04106450 0x7fbe04073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45048/0 (socket says 172.21.15.42:45048) 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe01d9b640 1 -- 172.21.15.42:0/505694713 learned_addr learned my addr 172.21.15.42:0/505694713 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe02d9d640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe04073a10 0x7fbe04073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.563+0000 7fbe0259c640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe04101380 0x7fbe04075080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbe01d9b640 1 -- 172.21.15.42:0/505694713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe04101380 msgr2=0x7fbe04075080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe04101380 0x7fbe04075080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbe01d9b640 1 -- 172.21.15.42:0/505694713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe04073a10 msgr2=0x7fbe04073ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe04073a10 0x7fbe04073ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbe01d9b640 1 -- 172.21.15.42:0/505694713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8009560 con 0x7fbe04106450 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbe02d9d640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe04073a10 0x7fbe04073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04106450 0x7fbe04073480 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009900 tx=0x7fbdf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.564+0000 7fbdf37fe640 1 -- 172.21.15.42:0/505694713 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8004520 con 0x7fbe04106450 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.565+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe041aa6c0 con 0x7fbe04106450 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.565+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe041aac20 con 0x7fbe04106450 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.565+0000 7fbdf37fe640 1 -- 172.21.15.42:0/505694713 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdf802de90 con 0x7fbe04106450 2024-03-12T04:16:39.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.565+0000 7fbdf37fe640 1 -- 172.21.15.42:0/505694713 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80054e0 con 0x7fbe04106450 2024-03-12T04:16:39.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.568+0000 7fbdf37fe640 1 -- 172.21.15.42:0/505694713 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fbdf8003800 con 0x7fbe04106450 2024-03-12T04:16:39.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.568+0000 7fbe0b3df740 1 Processor -- start 2024-03-12T04:16:39.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0b3df740 1 -- start start 2024-03-12T04:16:39.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37250 0x560d5ec37630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37b70 0x560d5ec40210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec42b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbdf37fe640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbdec0a1dc0 0x7fbdec0a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5ebf2960 con 0x560d5ec37250 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5eba6310 con 0x560d5ec37b70 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37b70 0x560d5ec40210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5eba3840 con 0x560d5ec40750 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.569+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37250 0x560d5ec37630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37250 0x560d5ec37630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45056/0 (socket says 172.21.15.42:45056) 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe02d9d640 1 -- 172.21.15.42:0/3354746517 learned_addr learned my addr 172.21.15.42:0/3354746517 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec42b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe02d9d640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbdec0a1dc0 0x7fbdec0a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbdf37fe640 1 -- 172.21.15.42:0/505694713 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbdf80effd0 con 0x7fbe04106450 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe0259c640 1 -- 172.21.15.42:0/3354746517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 msgr2=0x560d5ec42b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe0259c640 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec42b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe0259c640 1 -- 172.21.15.42:0/3354746517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37250 msgr2=0x560d5ec37630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.570+0000 7fbe0259c640 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37250 0x560d5ec37630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbe0259c640 1 -- 172.21.15.42:0/3354746517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d5eb61060 con 0x560d5ec37b70 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbe02d9d640 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37250 0x560d5ec37630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec42b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbe02d9d640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbdec0a1dc0 0x7fbdec0a4280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc010370 tx=0x7fbdfc0123f0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbe0259c640 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37b70 0x560d5ec40210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005fd0 tx=0x7fbdf40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be23aed9b8cce93e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbdf0ff9640 1 -- 172.21.15.42:0/3354746517 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf401c480 con 0x560d5ec37b70 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbdf0ff9640 1 -- 172.21.15.42:0/3354746517 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdf4015070 con 0x560d5ec37b70 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.571+0000 7fbdf0ff9640 1 -- 172.21.15.42:0/3354746517 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4011950 con 0x560d5ec37b70 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 -- 172.21.15.42:0/3354746517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37b70 msgr2=0x560d5ec40210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37b70 0x560d5ec40210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005fd0 tx=0x7fbdf40107a0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 -- 172.21.15.42:0/3354746517 shutdown_connections 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec42b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37b70 0x560d5ec40210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/3354746517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37250 0x560d5ec37630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 -- 172.21.15.42:0/3354746517 >> 172.21.15.42:0/3354746517 conn(0x560d5ec31980 msgr2=0x560d5ec31d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.572+0000 7fbe0b3df740 1 -- 172.21.15.42:0/3354746517 shutdown_connections 2024-03-12T04:16:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.573+0000 7fbe0b3df740 1 -- 172.21.15.42:0/3354746517 wait complete. 2024-03-12T04:16:39.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.573+0000 7fbe0b3df740 1 -- start start 2024-03-12T04:16:39.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.573+0000 7fbe0b3df740 1 Processor -- start 2024-03-12T04:16:39.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.573+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec3d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.573+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec3e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5eb9f020 con 0x560d5ec37b70 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e872430 con 0x560d5ec37250 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e82ee00 con 0x560d5ec40750 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0b3df740 10 client.0 _ll_register_callbacks cb 0x560d5eb46cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec3e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec3d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec3d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49800/0 (socket says 172.21.15.42:49800) 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 -- 172.21.15.42:0/2178949252 learned_addr learned my addr 172.21.15.42:0/2178949252 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 -- 172.21.15.42:0/2178949252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 msgr2=0x560d5ec3e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec3e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 -- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 msgr2=0x560d5ec3dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 -- 172.21.15.42:0/2178949252 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf400b8c0 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbe0259c640 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec3d670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400bf80 tx=0x7fbdf400bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa37826b9bcb715d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.574+0000 7fbde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbde9ffb640 1 -- 172.21.15.42:0/2178949252 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4021070 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbde9ffb640 1 -- 172.21.15.42:0/2178949252 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbdf4017070 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d5e89e7e0 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbde9ffb640 1 -- 172.21.15.42:0/2178949252 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdf401c7d0 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 10 client.25427 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560d5eb9d6a0 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbde9ffb640 1 -- 172.21.15.42:0/2178949252 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x560d5e89e7e0 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbde9ffb640 1 -- 172.21.15.42:0/2178949252 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbdf4026470 con 0x560d5ec37250 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 10 client.25427 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 10 client.25427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 20 client.25427 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5ec50ac0 0x560d5ec52ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 4 client.25427 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.575+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d5e8db720 con 0x560d5ec50ac0 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.576+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5ec50ac0 0x560d5ec52ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.576+0000 7fbde9ffb640 10 client.25427 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.576+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5ec50ac0 0x560d5ec52ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fbda740d168c48e1 server_cookie=d35c9202777ce584 in_seq=0 out_seq=0 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.576+0000 7fbde9ffb640 1 -- 172.21.15.42:0/2178949252 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d5e8db720 con 0x560d5ec50ac0 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.576+0000 7fbde9ffb640 10 client.25427 handle_command_reply: tid=0 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 client.25427 shutdown 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 msgr2=0x560d5ec3d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec3d670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400bf80 tx=0x7fbdf400bfb0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 shutdown_connections 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec40750 0x560d5ec3e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5ec50ac0 0x560d5ec52ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/2178949252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec3d670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 >> 172.21.15.42:0/2178949252 conn(0x560d5ec31980 msgr2=0x560d5ec336d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbde9ffb640 0 client.25427 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 shutdown_connections 2024-03-12T04:16:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 1 -- 172.21.15.42:0/2178949252 wait complete. 2024-03-12T04:16:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.577+0000 7fbe0b3df740 20 client.25427 trim_cache size 0 max 16384 2024-03-12T04:16:39.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.595+0000 7fbe0b3df740 1 Processor -- start 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.595+0000 7fbe0b3df740 1 -- start start 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37250 0x560d5ec3fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37250 0x560d5ec3fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37250 0x560d5ec3fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33772/0 (socket says 172.21.15.42:33772) 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec40750 0x560d5ec3df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0259c640 1 -- 172.21.15.42:0/151300355 learned_addr learned my addr 172.21.15.42:0/151300355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5eb24130 con 0x560d5ec37b70 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5ec42ec0 con 0x560d5ec40750 2024-03-12T04:16:39.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5ec317b0 con 0x560d5ec37250 2024-03-12T04:16:39.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe02d9d640 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec40750 0x560d5ec3df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0259c640 1 -- 172.21.15.42:0/151300355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec40750 msgr2=0x560d5ec3df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0259c640 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec40750 0x560d5ec3df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.596+0000 7fbe0259c640 1 -- 172.21.15.42:0/151300355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 msgr2=0x560d5ec3c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.597+0000 7fbe0259c640 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.597+0000 7fbe0259c640 1 -- 172.21.15.42:0/151300355 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d5eb39c20 con 0x560d5ec37250 2024-03-12T04:16:39.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.597+0000 7fbe0259c640 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37250 0x560d5ec3fda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005d00 tx=0x7fbdf40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66531c2c1b7259ec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.597+0000 7fbdebfff640 1 -- 172.21.15.42:0/151300355 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4021070 con 0x560d5ec37250 2024-03-12T04:16:39.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.597+0000 7fbdebfff640 1 -- 172.21.15.42:0/151300355 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdf400bae0 con 0x560d5ec37250 2024-03-12T04:16:39.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.597+0000 7fbdebfff640 1 -- 172.21.15.42:0/151300355 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40055b0 con 0x560d5ec37250 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.598+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37250 msgr2=0x560d5ec3fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.598+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37250 0x560d5ec3fda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005d00 tx=0x7fbdf40107a0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.599+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 shutdown_connections 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.599+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec40750 0x560d5ec3df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.599+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec37b70 0x560d5ec3c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.599+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/151300355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37250 0x560d5ec3fda0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.599+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 >> 172.21.15.42:0/151300355 conn(0x560d5ec31980 msgr2=0x560d5ec3bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.599+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 shutdown_connections 2024-03-12T04:16:39.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.599+0000 7fbe0b3df740 1 -- 172.21.15.42:0/151300355 wait complete. 2024-03-12T04:16:39.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.600+0000 7fbe0b3df740 1 -- start start 2024-03-12T04:16:39.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.600+0000 7fbe0b3df740 1 Processor -- start 2024-03-12T04:16:39.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec47730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37b70 0x560d5ec45ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0b3df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec40750 0x560d5ec46010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5ec46550 con 0x560d5ec40750 2024-03-12T04:16:39.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5ec466c0 con 0x560d5ec37250 2024-03-12T04:16:39.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0b3df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5ec46880 con 0x560d5ec37b70 2024-03-12T04:16:39.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0b3df740 10 client.0 _ll_register_callbacks cb 0x7fbdf400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37b70 0x560d5ec45ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec40750 0x560d5ec46010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37b70 0x560d5ec45ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33778/0 (socket says 172.21.15.42:33778) 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe01d9b640 1 -- 172.21.15.42:0/4196968979 learned_addr learned my addr 172.21.15.42:0/4196968979 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.601+0000 7fbe0259c640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec47730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe0259c640 1 -- 172.21.15.42:0/4196968979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37b70 msgr2=0x560d5ec45ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe0259c640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37b70 0x560d5ec45ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe0259c640 1 -- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec40750 msgr2=0x560d5ec46010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe0259c640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec40750 0x560d5ec46010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe0259c640 1 -- 172.21.15.42:0/4196968979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d5eb9d6a0 con 0x560d5ec37250 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37b70 0x560d5ec45ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe02d9d640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec40750 0x560d5ec46010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe0259c640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec47730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4034000 tx=0x7fbdf401ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6c46467f43b8de server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbde9ffb640 1 -- 172.21.15.42:0/4196968979 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4021040 con 0x560d5ec37250 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbde9ffb640 1 -- 172.21.15.42:0/4196968979 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbdf4000e40 con 0x560d5ec37250 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.602+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d5e92a890 con 0x560d5ec37250 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbde9ffb640 1 -- 172.21.15.42:0/4196968979 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdf4005ed0 con 0x560d5ec37250 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 10 client.25433 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560d5eb5f8c0 con 0x560d5ec37250 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbde9ffb640 1 -- 172.21.15.42:0/4196968979 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdf401cca0 con 0x560d5ec37250 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbde9ffb640 1 -- 172.21.15.42:0/4196968979 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbdf403a640 con 0x560d5ec37250 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 10 client.25433 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 10 client.25433 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 20 client.25433 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5eca8020 0x560d5ecaa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 4 client.25433 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d5e92a890 con 0x560d5eca8020 2024-03-12T04:16:39.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.603+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5eca8020 0x560d5ecaa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbde9ffb640 10 client.25433 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe01d9b640 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5eca8020 0x560d5ecaa440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d1a22f12970c259 server_cookie=6ea60ba273f53e37 in_seq=0 out_seq=0 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbde9ffb640 1 -- 172.21.15.42:0/4196968979 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d5e92a890 con 0x560d5eca8020 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbde9ffb640 10 client.25433 handle_command_reply: tid=0 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 client.25433 shutdown 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 msgr2=0x560d5ec47730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec47730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4034000 tx=0x7fbdf401ca80 comp rx=0 tx=0).stop 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 shutdown_connections 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x560d5eca8020 0x560d5ecaa440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d5ec40750 0x560d5ec46010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d5ec37b70 0x560d5ec45ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 --2- 172.21.15.42:0/4196968979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5ec37250 0x560d5ec47730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbde9ffb640 0 client.25433 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.604+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 >> 172.21.15.42:0/4196968979 conn(0x560d5ec31980 msgr2=0x560d5ec30b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 shutdown_connections 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe0b3df740 1 -- 172.21.15.42:0/4196968979 wait complete. 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe0b3df740 20 client.25433 trim_cache size 0 max 16384 2024-03-12T04:16:39.610 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbdec0a1dc0 msgr2=0x7fbdec0a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbdec0a1dc0 0x7fbdec0a4280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc010370 tx=0x7fbdfc0123f0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04106450 msgr2=0x7fbe04073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04106450 0x7fbe04073480 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009900 tx=0x7fbdf8004270 comp rx=0 tx=0).stop 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 shutdown_connections 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbdec0a1dc0 0x7fbdec0a4280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe04073a10 0x7fbe04073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe04106450 0x7fbe04073480 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 --2- 172.21.15.42:0/505694713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe04101380 0x7fbe04075080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 >> 172.21.15.42:0/505694713 conn(0x7fbe04078540 msgr2=0x7fbe040ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 shutdown_connections 2024-03-12T04:16:39.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.605+0000 7fbe08817640 1 -- 172.21.15.42:0/505694713 wait complete. 2024-03-12T04:16:39.623 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:39.623 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.775+0000 7f2ae1935640 1 -- 172.21.15.42:0/1458919755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100c10 msgr2=0x7f2adc104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.775+0000 7f2ae1935640 1 --2- 172.21.15.42:0/1458919755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100c10 0x7f2adc104c50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009930 tx=0x7f2ad002bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.776+0000 7f2ae1935640 1 -- 172.21.15.42:0/1458919755 shutdown_connections 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.776+0000 7f2ae1935640 1 --2- 172.21.15.42:0/1458919755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc105330 0x7f2adc10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.776+0000 7f2ae1935640 1 --2- 172.21.15.42:0/1458919755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100c10 0x7f2adc104c50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.776+0000 7f2ae1935640 1 --2- 172.21.15.42:0/1458919755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2adc100260 0x7f2adc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.776+0000 7f2ae1935640 1 -- 172.21.15.42:0/1458919755 >> 172.21.15.42:0/1458919755 conn(0x7f2adc077180 msgr2=0x7f2adc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.776+0000 7f2ae1935640 1 -- 172.21.15.42:0/1458919755 shutdown_connections 2024-03-12T04:16:39.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.776+0000 7f2ae1935640 1 -- 172.21.15.42:0/1458919755 wait complete. 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 Processor -- start 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 -- start start 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100260 0x7f2adc10ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2adc105330 0x7f2adc108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc108ff0 0x7f2adc1094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc10d1d0 con 0x7f2adc100260 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1aa4e0 con 0x7f2adc108ff0 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.777+0000 7f2ae1935640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1aa620 con 0x7f2adc105330 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2ada7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2adc105330 0x7f2adc108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc108ff0 0x7f2adc1094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100260 0x7f2adc10ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc108ff0 0x7f2adc1094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49844/0 (socket says 172.21.15.42:49844) 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 -- 172.21.15.42:0/4294628017 learned_addr learned my addr 172.21.15.42:0/4294628017 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100260 0x7f2adc10ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45104/0 (socket says 172.21.15.42:45104) 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2adc105330 0x7f2adc108ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33796/0 (socket says 172.21.15.42:33796) 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 -- 172.21.15.42:0/4294628017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2adc105330 msgr2=0x7f2adc108ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2adc105330 0x7f2adc108ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 -- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100260 msgr2=0x7f2adc10ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100260 0x7f2adc10ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 -- 172.21.15.42:0/4294628017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0009560 con 0x7f2adc108ff0 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adaffd640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100260 0x7f2adc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.778+0000 7f2adb7fe640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc108ff0 0x7f2adc1094a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400e970 tx=0x7f2ad400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.779+0000 7f2ae0933640 1 -- 172.21.15.42:0/4294628017 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4009720 con 0x7f2adc108ff0 2024-03-12T04:16:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.779+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc1aa880 con 0x7f2adc108ff0 2024-03-12T04:16:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.779+0000 7f2ae0933640 1 -- 172.21.15.42:0/4294628017 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ad40054d0 con 0x7f2adc108ff0 2024-03-12T04:16:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.779+0000 7f2ae0933640 1 -- 172.21.15.42:0/4294628017 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4010430 con 0x7f2adc108ff0 2024-03-12T04:16:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.779+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc1aad10 con 0x7f2adc108ff0 2024-03-12T04:16:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae0933640 1 -- 172.21.15.42:0/4294628017 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f2ad4004f30 con 0x7f2adc108ff0 2024-03-12T04:16:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 Processor -- start 2024-03-12T04:16:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 -- start start 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a130630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a139210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a0eb960 con 0x55927a130250 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a09f310 con 0x55927a130b70 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a09c840 con 0x55927a139750 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.782+0000 7f2ae0933640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2ac40a1cc0 0x7f2ac40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a130630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a130630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45116/0 (socket says 172.21.15.42:45116) 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 -- 172.21.15.42:0/2287156492 learned_addr learned my addr 172.21.15.42:0/2287156492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2adb7fe640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2ac40a1cc0 0x7f2ac40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2adaffd640 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ae0933640 1 -- 172.21.15.42:0/4294628017 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2ad4014070 con 0x7f2adc108ff0 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 -- 172.21.15.42:0/2287156492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 msgr2=0x55927a13bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 -- 172.21.15.42:0/2287156492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 msgr2=0x55927a139210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a139210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 -- 172.21.15.42:0/2287156492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55927a05a060 con 0x55927a130250 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2adaffd640 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2adb7fe640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2ac40a1cc0 0x7f2ac40a4180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40cb3f0 tx=0x7f2ad406da50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.783+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a130630 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0004c20 tx=0x7f2ad002c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffa6d0f24364db0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.784+0000 7f2ac1ffb640 1 -- 172.21.15.42:0/2287156492 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad002c8d0 con 0x55927a130250 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.784+0000 7f2ac1ffb640 1 -- 172.21.15.42:0/2287156492 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ad002de90 con 0x55927a130250 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.784+0000 7f2ac1ffb640 1 -- 172.21.15.42:0/2287156492 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0003ab0 con 0x55927a130250 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 -- 172.21.15.42:0/2287156492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 msgr2=0x55927a130630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a130630 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0004c20 tx=0x7f2ad002c610 comp rx=0 tx=0).stop 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 -- 172.21.15.42:0/2287156492 shutdown_connections 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 --2- 172.21.15.42:0/2287156492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a130630 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 -- 172.21.15.42:0/2287156492 >> 172.21.15.42:0/2287156492 conn(0x55927a12a980 msgr2=0x55927a12ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 -- 172.21.15.42:0/2287156492 shutdown_connections 2024-03-12T04:16:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.785+0000 7f2ae45df740 1 -- 172.21.15.42:0/2287156492 wait complete. 2024-03-12T04:16:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.786+0000 7f2ae45df740 1 -- start start 2024-03-12T04:16:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.786+0000 7f2ae45df740 1 Processor -- start 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a136670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a1370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2ae45df740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a098020 con 0x55927a130250 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2ae45df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559279d6b430 con 0x55927a130b70 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2ae45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559279d27e00 con 0x55927a139750 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2ae45df740 10 client.0 _ll_register_callbacks cb 0x55927a03fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2adaffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2adaffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a136bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49864/0 (socket says 172.21.15.42:49864) 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2adaffd640 1 -- 172.21.15.42:0/3204193174 learned_addr learned my addr 172.21.15.42:0/3204193174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2adaffd640 1 -- 172.21.15.42:0/3204193174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 msgr2=0x55927a1370f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T04:16:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2adaffd640 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a1370f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.787+0000 7f2adaffd640 1 -- 172.21.15.42:0/3204193174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 msgr2=0x55927a136670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T04:16:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.788+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.788+0000 7f2adaffd640 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a136670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.788+0000 7f2adaffd640 1 -- 172.21.15.42:0/3204193174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc10d710 con 0x55927a130b70 2024-03-12T04:16:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.788+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a136670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:16:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.788+0000 7f2adaffd640 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a136bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2acc011e50 tx=0x7f2acc013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29860127938f76a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.788+0000 7f2abaffd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.792+0000 7f2abaffd640 1 -- 172.21.15.42:0/3204193174 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0107e0 con 0x55927a130b70 2024-03-12T04:16:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.792+0000 7f2abaffd640 1 -- 172.21.15.42:0/3204193174 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2acc010940 con 0x55927a130b70 2024-03-12T04:16:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.792+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559279d977e0 con 0x55927a130b70 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.792+0000 7f2abaffd640 1 -- 172.21.15.42:0/3204193174 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2acc010b00 con 0x55927a130b70 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.792+0000 7f2ae45df740 10 client.25445 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.792+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55927a0966a0 con 0x55927a130b70 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2abaffd640 1 -- 172.21.15.42:0/3204193174 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x559279d977e0 con 0x55927a130b70 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2abaffd640 1 -- 172.21.15.42:0/3204193174 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2acc01b3e0 con 0x55927a130b70 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2ae45df740 10 client.25445 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2ae45df740 10 client.25445 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2ae45df740 20 client.25445 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2ae45df740 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a15abb0 0x55927a15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2ae45df740 4 client.25445 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:39.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559279dd4720 con 0x55927a15abb0 2024-03-12T04:16:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.793+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a15abb0 0x55927a15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.794+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a15abb0 0x55927a15cff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57feb64dc6ebfd68 server_cookie=93cb4a9b3ff38698 in_seq=0 out_seq=0 2024-03-12T04:16:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.794+0000 7f2abaffd640 10 client.25445 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2abaffd640 1 -- 172.21.15.42:0/3204193174 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559279dd4720 con 0x55927a15abb0 2024-03-12T04:16:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2abaffd640 10 client.25445 handle_command_reply: tid=0 2024-03-12T04:16:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 client.25445 shutdown 2024-03-12T04:16:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 msgr2=0x55927a136bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a136bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2acc011e50 tx=0x7f2acc013830 comp rx=0 tx=0).stop 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 shutdown_connections 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a15abb0 0x55927a15cff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a1370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130b70 0x55927a136bb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 --2- 172.21.15.42:0/3204193174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130250 0x55927a136670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.795+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 >> 172.21.15.42:0/3204193174 conn(0x55927a12a980 msgr2=0x55927a12c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.796+0000 7f2abaffd640 0 client.25445 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.796+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 shutdown_connections 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.796+0000 7f2ae45df740 1 -- 172.21.15.42:0/3204193174 wait complete. 2024-03-12T04:16:39.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.796+0000 7f2ae45df740 20 client.25445 trim_cache size 0 max 16384 2024-03-12T04:16:39.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.813+0000 7f2ae45df740 1 Processor -- start 2024-03-12T04:16:39.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.813+0000 7f2ae45df740 1 -- start start 2024-03-12T04:16:39.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a138da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a130b70 0x55927a1319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a139750 0x55927a161990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ae45df740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a01d130 con 0x55927a139750 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ae45df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a157570 con 0x55927a130250 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ae45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a13bb40 con 0x55927a130b70 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a139750 0x55927a161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a139750 0x55927a161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45134/0 (socket says 172.21.15.42:45134) 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2adaffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a130b70 0x55927a1319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 -- 172.21.15.42:0/1259435671 learned_addr learned my addr 172.21.15.42:0/1259435671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 -- 172.21.15.42:0/1259435671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a130b70 msgr2=0x55927a1319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a130b70 0x55927a1319f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 -- 172.21.15.42:0/1259435671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 msgr2=0x55927a138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a138da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.814+0000 7f2ada7fc640 1 -- 172.21.15.42:0/1259435671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55927a032c20 con 0x55927a139750 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.815+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a139750 0x55927a161990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2ad002c710 tx=0x7f2ad0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f80a899b1824f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.815+0000 7f2ac17fa640 1 -- 172.21.15.42:0/1259435671 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0052070 con 0x55927a139750 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.815+0000 7f2ac17fa640 1 -- 172.21.15.42:0/1259435671 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ad00049f0 con 0x55927a139750 2024-03-12T04:16:39.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.815+0000 7f2ac17fa640 1 -- 172.21.15.42:0/1259435671 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0003ab0 con 0x55927a139750 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 -- 172.21.15.42:0/1259435671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a139750 msgr2=0x55927a161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a139750 0x55927a161990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2ad002c710 tx=0x7f2ad0000bc0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 -- 172.21.15.42:0/1259435671 shutdown_connections 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a139750 0x55927a161990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a130b70 0x55927a1319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 --2- 172.21.15.42:0/1259435671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 -- 172.21.15.42:0/1259435671 >> 172.21.15.42:0/1259435671 conn(0x55927a12a980 msgr2=0x55927a12ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 -- 172.21.15.42:0/1259435671 shutdown_connections 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 -- 172.21.15.42:0/1259435671 wait complete. 2024-03-12T04:16:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.816+0000 7f2ae45df740 1 -- start start 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 1 Processor -- start 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130b70 0x55927a1408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a13ef20 con 0x55927a130b70 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a13f090 con 0x55927a130250 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927a13f250 con 0x55927a139750 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ae45df740 10 client.0 _ll_register_callbacks cb 0x7f2ad0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ada7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2adaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130b70 0x55927a1408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ada7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49878/0 (socket says 172.21.15.42:49878) 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ada7fc640 1 -- 172.21.15.42:0/840521576 learned_addr learned my addr 172.21.15.42:0/840521576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.817+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ada7fc640 1 -- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 msgr2=0x55927a13e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ada7fc640 1 -- 172.21.15.42:0/840521576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130b70 msgr2=0x55927a1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130b70 0x55927a1408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ada7fc640 1 -- 172.21.15.42:0/840521576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad0058040 con 0x55927a130250 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ada7fc640 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a140390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0001eb0 tx=0x7f2ad0004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e56ac5e1f449cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2abaffd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2abaffd640 1 -- 172.21.15.42:0/840521576 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0052070 con 0x55927a130250 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2abaffd640 1 -- 172.21.15.42:0/840521576 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2ad0000e40 con 0x55927a130250 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ad002de90 con 0x55927a130250 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2abaffd640 1 -- 172.21.15.42:0/840521576 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ad002c8d0 con 0x55927a130250 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ae45df740 10 client.25451 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.818+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55927a0588c0 con 0x55927a130250 2024-03-12T04:16:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2abaffd640 1 -- 172.21.15.42:0/840521576 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ad002de90 con 0x55927a130250 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2abaffd640 1 -- 172.21.15.42:0/840521576 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2ad004b040 con 0x55927a130250 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2ae45df740 10 client.25451 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2ae45df740 10 client.25451 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2ae45df740 20 client.25451 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2ae45df740 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a1b1f30 0x55927a1b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2ae45df740 4 client.25451 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.819+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559279e23890 con 0x55927a1b1f30 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.820+0000 7f2adaffd640 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a1b1f30 0x55927a1b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.820+0000 7f2adaffd640 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a1b1f30 0x55927a1b4350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=747da358f6e98276 server_cookie=44caa0f512f132fb in_seq=0 out_seq=0 2024-03-12T04:16:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.820+0000 7f2abaffd640 10 client.25451 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.820+0000 7f2abaffd640 1 -- 172.21.15.42:0/840521576 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559279e23890 con 0x55927a1b1f30 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.820+0000 7f2abaffd640 10 client.25451 handle_command_reply: tid=0 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.820+0000 7f2ae45df740 1 client.25451 shutdown 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 msgr2=0x55927a140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a140390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0001eb0 tx=0x7f2ad0004b60 comp rx=0 tx=0).stop 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 shutdown_connections 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55927a1b1f30 0x55927a1b4350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55927a139750 0x55927a13e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55927a130b70 0x55927a1408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 --2- 172.21.15.42:0/840521576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55927a130250 0x55927a140390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 >> 172.21.15.42:0/840521576 conn(0x55927a12a980 msgr2=0x55927a1319f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2abaffd640 0 client.25451 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 shutdown_connections 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 1 -- 172.21.15.42:0/840521576 wait complete. 2024-03-12T04:16:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.821+0000 7f2ae45df740 20 client.25451 trim_cache size 0 max 16384 2024-03-12T04:16:39.827 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:39.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2ac40a1cc0 msgr2=0x7f2ac40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2ac40a1cc0 0x7f2ac40a4180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40cb3f0 tx=0x7f2ad406da50 comp rx=0 tx=0).stop 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc108ff0 msgr2=0x7f2adc1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc108ff0 0x7f2adc1094a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400e970 tx=0x7f2ad400ed70 comp rx=0 tx=0).stop 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 shutdown_connections 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2ac40a1cc0 0x7f2ac40a4180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc108ff0 0x7f2adc1094a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2adc105330 0x7f2adc108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 --2- 172.21.15.42:0/4294628017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2adc100260 0x7f2adc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 >> 172.21.15.42:0/4294628017 conn(0x7f2adc077180 msgr2=0x7f2adc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 shutdown_connections 2024-03-12T04:16:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.822+0000 7f2ae1935640 1 -- 172.21.15.42:0/4294628017 wait complete. 2024-03-12T04:16:39.845 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:39.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 -- 172.21.15.42:0/3816927076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6948101150 msgr2=0x7f694810ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 --2- 172.21.15.42:0/3816927076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6948101150 0x7f694810ce80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6940009ca0 tx=0x7f694002c140 comp rx=0 tx=0).stop 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 -- 172.21.15.42:0/3816927076 shutdown_connections 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 --2- 172.21.15.42:0/3816927076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6948101150 0x7f694810ce80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 --2- 172.21.15.42:0/3816927076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69481007b0 0x7f6948100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 --2- 172.21.15.42:0/3816927076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6948105010 0x7f69481053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 -- 172.21.15.42:0/3816927076 >> 172.21.15.42:0/3816927076 conn(0x7f69480765a0 msgr2=0x7f69480fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.990+0000 7f694d5c7640 1 -- 172.21.15.42:0/3816927076 shutdown_connections 2024-03-12T04:16:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.991+0000 7f694d5c7640 1 -- 172.21.15.42:0/3816927076 wait complete. 2024-03-12T04:16:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.992+0000 7f694d5c7640 1 Processor -- start 2024-03-12T04:16:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.992+0000 7f694d5c7640 1 -- start start 2024-03-12T04:16:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f694d5c7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69481007b0 0x7f69481975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f694d5c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6948101150 0x7f6948197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f694d5c7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6948105010 0x7f694819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f694d5c7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694819c430 con 0x7f69481007b0 2024-03-12T04:16:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f694d5c7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694819c5a0 con 0x7f6948101150 2024-03-12T04:16:39.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f694d5c7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694819c760 con 0x7f6948105010 2024-03-12T04:16:39.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69481007b0 0x7f69481975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69481007b0 0x7f69481975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45176/0 (socket says 172.21.15.42:45176) 2024-03-12T04:16:39.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f6946ffd640 1 -- 172.21.15.42:0/2123135934 learned_addr learned my addr 172.21.15.42:0/2123135934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f69477fe640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6948105010 0x7f694819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f69467fc640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6948101150 0x7f6948197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f69477fe640 1 -- 172.21.15.42:0/2123135934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6948101150 msgr2=0x7f6948197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f69477fe640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6948101150 0x7f6948197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f69477fe640 1 -- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69481007b0 msgr2=0x7f69481975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f69477fe640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69481007b0 0x7f69481975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.993+0000 7f69477fe640 1 -- 172.21.15.42:0/2123135934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940009900 con 0x7f6948105010 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.994+0000 7f69477fe640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6948105010 0x7f694819be30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6940009bf0 tx=0x7f6940034c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.994+0000 7f692ffff640 1 -- 172.21.15.42:0/2123135934 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940032a30 con 0x7f6948105010 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.994+0000 7f692ffff640 1 -- 172.21.15.42:0/2123135934 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6940003e20 con 0x7f6948105010 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.994+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694819c9e0 con 0x7f6948105010 2024-03-12T04:16:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.994+0000 7f692ffff640 1 -- 172.21.15.42:0/2123135934 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940004800 con 0x7f6948105010 2024-03-12T04:16:39.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.994+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694810da10 con 0x7f6948105010 2024-03-12T04:16:40.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.997+0000 7f692ffff640 1 -- 172.21.15.42:0/2123135934 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f694000a040 con 0x7f6948105010 2024-03-12T04:16:40.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 Processor -- start 2024-03-12T04:16:40.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 -- start start 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d7a60 0x55eb830d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d8380 0x55eb830e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830e3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8308a960 con 0x55eb830e0f60 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8303e310 con 0x55eb830d8380 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f6950198740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8303b840 con 0x55eb830d7a60 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f69467fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d7a60 0x55eb830d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.998+0000 7f69467fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d7a60 0x55eb830d7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33840/0 (socket says 172.21.15.42:33840) 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 -- 172.21.15.42:0/2610410834 learned_addr learned my addr 172.21.15.42:0/2610410834 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f6946ffd640 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d8380 0x55eb830e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f692ffff640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f69300a1dc0 0x7f69300a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 -- 172.21.15.42:0/2610410834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d7a60 msgr2=0x55eb830d7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d7a60 0x55eb830d7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 -- 172.21.15.42:0/2610410834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d8380 msgr2=0x55eb830e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d8380 0x55eb830e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 -- 172.21.15.42:0/2610410834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb82ff9060 con 0x55eb830e0f60 2024-03-12T04:16:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f6946ffd640 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d8380 0x55eb830e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f692ffff640 1 -- 172.21.15.42:0/2123135934 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f69400f0140 con 0x7f6948105010 2024-03-12T04:16:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69477fe640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f69300a1dc0 0x7f69300a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:39.999+0000 7f69467fc640 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830e3350 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f693c010960 tx=0x7f693c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a7aeb17adf055a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.000+0000 7f692d7fa640 1 -- 172.21.15.42:0/2610410834 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00f9c0 con 0x55eb830e0f60 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.000+0000 7f692d7fa640 1 -- 172.21.15.42:0/2610410834 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f693c01e3b0 con 0x55eb830e0f60 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.000+0000 7f692d7fa640 1 -- 172.21.15.42:0/2610410834 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00e610 con 0x55eb830e0f60 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 -- 172.21.15.42:0/2610410834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 msgr2=0x55eb830e3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830e3350 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f693c010960 tx=0x7f693c005f90 comp rx=0 tx=0).stop 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f69477fe640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f69300a1dc0 0x7f69300a4280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69400ed870 tx=0x7f6940032b90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 -- 172.21.15.42:0/2610410834 shutdown_connections 2024-03-12T04:16:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830e3350 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d8380 0x55eb830e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 --2- 172.21.15.42:0/2610410834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d7a60 0x55eb830d7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 -- 172.21.15.42:0/2610410834 >> 172.21.15.42:0/2610410834 conn(0x55eb830d2190 msgr2=0x55eb830d2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 -- 172.21.15.42:0/2610410834 shutdown_connections 2024-03-12T04:16:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.001+0000 7f6950198740 1 -- 172.21.15.42:0/2610410834 wait complete. 2024-03-12T04:16:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 -- start start 2024-03-12T04:16:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 Processor -- start 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830dea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb83037020 con 0x55eb830d8380 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb82d0a430 con 0x55eb830d7a60 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb82cc6e00 con 0x55eb830e0f60 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f6950198740 10 client.0 _ll_register_callbacks cb 0x55eb82fdecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f69467fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f69467fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830ddfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49912/0 (socket says 172.21.15.42:49912) 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f69467fc640 1 -- 172.21.15.42:0/2605551113 learned_addr learned my addr 172.21.15.42:0/2605551113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.002+0000 7f69467fc640 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830dea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f6946ffd640 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f69467fc640 1 -- 172.21.15.42:0/2605551113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 msgr2=0x55eb830dea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f69467fc640 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830dea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f69467fc640 1 -- 172.21.15.42:0/2605551113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 msgr2=0x55eb830de4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f69467fc640 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830de4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f69467fc640 1 -- 172.21.15.42:0/2605551113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c010640 con 0x55eb830d7a60 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f69467fc640 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830ddfb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f693c01ec80 tx=0x7f693c01eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82adb3ce293ab522 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f692aff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.003+0000 7f692aff5640 1 -- 172.21.15.42:0/2605551113 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c005220 con 0x55eb830d7a60 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.004+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb82d367e0 con 0x55eb830d7a60 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.004+0000 7f692aff5640 1 -- 172.21.15.42:0/2605551113 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f693c005380 con 0x55eb830d7a60 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.004+0000 7f692aff5640 1 -- 172.21.15.42:0/2605551113 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f693c01eea0 con 0x55eb830d7a60 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.004+0000 7f6950198740 10 client.25463 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.004+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eb830356a0 con 0x55eb830d7a60 2024-03-12T04:16:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.004+0000 7f692aff5640 1 -- 172.21.15.42:0/2605551113 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55eb82d367e0 con 0x55eb830d7a60 2024-03-12T04:16:40.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f692aff5640 1 -- 172.21.15.42:0/2605551113 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f693c01d030 con 0x55eb830d7a60 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f6950198740 10 client.25463 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f6950198740 10 client.25463 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f6950198740 20 client.25463 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f6950198740 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83102370 0x55eb83104790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f6950198740 4 client.25463 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb82d73720 con 0x55eb83102370 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.005+0000 7f6946ffd640 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83102370 0x55eb83104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.006+0000 7f6946ffd640 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83102370 0x55eb83104790 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=934eeac476f4ccf2 server_cookie=12d388c0b0205e1b in_seq=0 out_seq=0 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.006+0000 7f692aff5640 10 client.25463 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.006+0000 7f692aff5640 1 -- 172.21.15.42:0/2605551113 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb82d73720 con 0x55eb83102370 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.006+0000 7f692aff5640 10 client.25463 handle_command_reply: tid=0 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.006+0000 7f6950198740 1 client.25463 shutdown 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 msgr2=0x55eb830ddfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830ddfb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f693c01ec80 tx=0x7f693c01eda0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 shutdown_connections 2024-03-12T04:16:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83102370 0x55eb83104790 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830dea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830de4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 --2- 172.21.15.42:0/2605551113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830ddfb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 >> 172.21.15.42:0/2605551113 conn(0x55eb830d2190 msgr2=0x55eb830d3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f692aff5640 0 client.25463 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.007+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 shutdown_connections 2024-03-12T04:16:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.008+0000 7f6950198740 1 -- 172.21.15.42:0/2605551113 wait complete. 2024-03-12T04:16:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.008+0000 7f6950198740 20 client.25463 trim_cache size 0 max 16384 2024-03-12T04:16:40.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 Processor -- start 2024-03-12T04:16:40.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 -- start start 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d8380 0x55eb830f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb82fbc130 con 0x55eb830e0f60 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb830d32e0 con 0x55eb830d7a60 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f6950198740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb830d4500 con 0x55eb830d8380 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.028+0000 7f69467fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d8380 0x55eb830f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45210/0 (socket says 172.21.15.42:45210) 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 -- 172.21.15.42:0/2028846294 learned_addr learned my addr 172.21.15.42:0/2028846294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 -- 172.21.15.42:0/2028846294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d8380 msgr2=0x55eb830f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d8380 0x55eb830f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 -- 172.21.15.42:0/2028846294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 msgr2=0x55eb830de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 -- 172.21.15.42:0/2028846294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb82fd1c20 con 0x55eb830e0f60 2024-03-12T04:16:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f69467fc640 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830f9390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f693c013040 tx=0x7f693c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411b0127236693a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.029+0000 7f692cff9640 1 -- 172.21.15.42:0/2028846294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c009c00 con 0x55eb830e0f60 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.030+0000 7f692cff9640 1 -- 172.21.15.42:0/2028846294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f693c01ee10 con 0x55eb830e0f60 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.030+0000 7f692cff9640 1 -- 172.21.15.42:0/2028846294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c004070 con 0x55eb830e0f60 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 -- 172.21.15.42:0/2028846294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 msgr2=0x55eb830f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830f9390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f693c013040 tx=0x7f693c005f90 comp rx=0 tx=0).stop 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 -- 172.21.15.42:0/2028846294 shutdown_connections 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830e0f60 0x55eb830f9390 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830d8380 0x55eb830f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 --2- 172.21.15.42:0/2028846294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 -- 172.21.15.42:0/2028846294 >> 172.21.15.42:0/2028846294 conn(0x55eb830d2190 msgr2=0x55eb830d92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 -- 172.21.15.42:0/2028846294 shutdown_connections 2024-03-12T04:16:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.031+0000 7f6950198740 1 -- 172.21.15.42:0/2028846294 wait complete. 2024-03-12T04:16:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.032+0000 7f6950198740 1 -- start start 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.032+0000 7f6950198740 1 Processor -- start 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.032+0000 7f6950198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6950198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6950198740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f69467fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f69467fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49936/0 (socket says 172.21.15.42:49936) 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f69467fc640 1 -- 172.21.15.42:0/227667383 learned_addr learned my addr 172.21.15.42:0/227667383 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb830f8100 con 0x55eb830d8380 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6946ffd640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb830f8270 con 0x55eb830d7a60 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb83158840 con 0x55eb830e0f60 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6950198740 10 client.0 _ll_register_callbacks cb 0x7f693c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f69467fc640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6946ffd640 1 -- 172.21.15.42:0/227667383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 msgr2=0x55eb830f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6946ffd640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6946ffd640 1 -- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 msgr2=0x55eb830f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6946ffd640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f6946ffd640 1 -- 172.21.15.42:0/227667383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb830356a0 con 0x55eb830d8380 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.033+0000 7f69467fc640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f6946ffd640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830f74f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x55eb83105030 tx=0x7f6938021410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aab0e4b7dc383cd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f692aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f692aff5640 1 -- 172.21.15.42:0/227667383 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938017040 con 0x55eb830d8380 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f692aff5640 1 -- 172.21.15.42:0/227667383 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6938005220 con 0x55eb830d8380 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f693c01e3b0 con 0x55eb830d8380 2024-03-12T04:16:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f692aff5640 1 -- 172.21.15.42:0/227667383 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69380053e0 con 0x55eb830d8380 2024-03-12T04:16:40.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f6950198740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:40.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.034+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eb82ff78c0 con 0x55eb830d8380 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f692aff5640 1 -- 172.21.15.42:0/227667383 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f693c01e3b0 con 0x55eb830d8380 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f692aff5640 1 -- 172.21.15.42:0/227667383 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6938012690 con 0x55eb830d8380 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f6950198740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f6950198740 10 client.15975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f6950198740 20 client.15975 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f6950198740 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83159f00 0x55eb8315c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f6950198740 4 client.15975 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb82dc2890 con 0x55eb83159f00 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.035+0000 7f69467fc640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83159f00 0x55eb8315c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f69467fc640 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83159f00 0x55eb8315c320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c36dd09faf8c3f4 server_cookie=561420352e4e509 in_seq=0 out_seq=0 2024-03-12T04:16:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f692aff5640 10 client.15975 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f692aff5640 1 -- 172.21.15.42:0/227667383 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb82dc2890 con 0x55eb83159f00 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f692aff5640 10 client.15975 handle_command_reply: tid=0 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f6950198740 1 client.15975 shutdown 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 msgr2=0x55eb830f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f6950198740 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830f74f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x55eb83105030 tx=0x7f6938021410 comp rx=0 tx=0).stop 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.036+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 shutdown_connections 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f6950198740 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55eb83159f00 0x55eb8315c320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f6950198740 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb830e0f60 0x55eb830f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f6950198740 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb830d8380 0x55eb830f74f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f6950198740 1 --2- 172.21.15.42:0/227667383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eb830d7a60 0x55eb830f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 >> 172.21.15.42:0/227667383 conn(0x55eb830d2190 msgr2=0x55eb830c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f692aff5640 0 client.15975 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 shutdown_connections 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.037+0000 7f6950198740 1 -- 172.21.15.42:0/227667383 wait complete. 2024-03-12T04:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.038+0000 7f6950198740 20 client.15975 trim_cache size 0 max 16384 2024-03-12T04:16:40.043 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:40.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f69300a1dc0 msgr2=0x7f69300a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f69300a1dc0 0x7f69300a4280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69400ed870 tx=0x7f6940032b90 comp rx=0 tx=0).stop 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6948105010 msgr2=0x7f694819be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6948105010 0x7f694819be30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6940009bf0 tx=0x7f6940034c90 comp rx=0 tx=0).stop 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 shutdown_connections 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f69300a1dc0 0x7f69300a4280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6948105010 0x7f694819be30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6948101150 0x7f6948197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 --2- 172.21.15.42:0/2123135934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69481007b0 0x7f69481975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 >> 172.21.15.42:0/2123135934 conn(0x7f69480765a0 msgr2=0x7f6948102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 shutdown_connections 2024-03-12T04:16:40.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.039+0000 7f694d5c7640 1 -- 172.21.15.42:0/2123135934 wait complete. 2024-03-12T04:16:40.057 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:39 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1581753892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:16:40.058 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:40.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:16:40.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:39 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1581753892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:16:40.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:39 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1581753892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.215+0000 7f6fc8f77640 1 -- 172.21.15.42:0/2498708675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc41060d0 msgr2=0x7f6fc4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.215+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/2498708675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc41060d0 0x7f6fc4068380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009930 tx=0x7f6fb402bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.215+0000 7f6fc8f77640 1 -- 172.21.15.42:0/2498708675 shutdown_connections 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.215+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/2498708675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc4103c40 0x7f6fc41040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.215+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/2498708675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fc40688c0 0x7f6fc4068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.215+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/2498708675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc41060d0 0x7f6fc4068380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.215+0000 7f6fc8f77640 1 -- 172.21.15.42:0/2498708675 >> 172.21.15.42:0/2498708675 conn(0x7f6fc4077180 msgr2=0x7f6fc4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.216+0000 7f6fc8f77640 1 -- 172.21.15.42:0/2498708675 shutdown_connections 2024-03-12T04:16:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.216+0000 7f6fc8f77640 1 -- 172.21.15.42:0/2498708675 wait complete. 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.216+0000 7f6fc8f77640 1 Processor -- start 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.216+0000 7f6fc8f77640 1 -- start start 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc8f77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc40688c0 0x7f6fc41a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc8f77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fc4103c40 0x7f6fc41a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc8f77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc41060d0 0x7f6fc419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc40688c0 0x7f6fc41a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc40688c0 0x7f6fc41a3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49954/0 (socket says 172.21.15.42:49954) 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc259c640 1 -- 172.21.15.42:0/237983572 learned_addr learned my addr 172.21.15.42:0/237983572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc419df50 con 0x7f6fc41060d0 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc419e0c0 con 0x7f6fc40688c0 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc419e280 con 0x7f6fc4103c40 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc2d9d640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc41060d0 0x7f6fc419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.217+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fc4103c40 0x7f6fc41a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc1d9b640 1 -- 172.21.15.42:0/237983572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc40688c0 msgr2=0x7f6fc41a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc40688c0 0x7f6fc41a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc1d9b640 1 -- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc41060d0 msgr2=0x7f6fc419d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc41060d0 0x7f6fc419d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc1d9b640 1 -- 172.21.15.42:0/237983572 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc009840 con 0x7f6fc4103c40 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc2d9d640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc41060d0 0x7f6fc419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fc4103c40 0x7f6fc41a3c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800e9f0 tx=0x7f6fb800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fb37fe640 1 -- 172.21.15.42:0/237983572 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8009640 con 0x7f6fc4103c40 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb4009560 con 0x7f6fc4103c40 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fb37fe640 1 -- 172.21.15.42:0/237983572 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fb800ccc0 con 0x7f6fc4103c40 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fb37fe640 1 -- 172.21.15.42:0/237983572 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8018640 con 0x7f6fc4103c40 2024-03-12T04:16:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.218+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc419e750 con 0x7f6fc4103c40 2024-03-12T04:16:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fb37fe640 1 -- 172.21.15.42:0/237983572 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f6fb8010430 con 0x7f6fc4103c40 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 Processor -- start 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 -- start start 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943fa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618943ac960 con 0x5618943fa750 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561894360310 con 0x5618943f1250 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.222+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189435d840 con 0x5618943f1b70 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943fa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33908/0 (socket says 172.21.15.42:33908) 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 -- 172.21.15.42:0/797700528 learned_addr learned my addr 172.21.15.42:0/797700528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fb37fe640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6fac0a1d40 0x7f6fac0a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc2d9d640 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 -- 172.21.15.42:0/797700528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 msgr2=0x5618943f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 -- 172.21.15.42:0/797700528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 msgr2=0x5618943fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc2d9d640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6fac0a1d40 0x7f6fac0a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc259c640 1 -- 172.21.15.42:0/797700528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56189431b060 con 0x5618943f1b70 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fb37fe640 1 -- 172.21.15.42:0/237983572 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fb8014070 con 0x7f6fc4103c40 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.223+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.224+0000 7f6fc259c640 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943fa210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4000b60 tx=0x7f6fb40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf4ed4e004173c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.224+0000 7f6fc2d9d640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6fac0a1d40 0x7f6fac0a4200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005fd0 tx=0x7f6fbc004ab0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.224+0000 7f6fb0ff9640 1 -- 172.21.15.42:0/797700528 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4004af0 con 0x5618943f1b70 2024-03-12T04:16:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.224+0000 7f6fb0ff9640 1 -- 172.21.15.42:0/797700528 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fb402cc20 con 0x5618943f1b70 2024-03-12T04:16:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.225+0000 7f6fb0ff9640 1 -- 172.21.15.42:0/797700528 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4003940 con 0x5618943f1b70 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 -- 172.21.15.42:0/797700528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 msgr2=0x5618943fa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943fa210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4000b60 tx=0x7f6fb40046c0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 -- 172.21.15.42:0/797700528 shutdown_connections 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943fa210 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/797700528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 -- 172.21.15.42:0/797700528 >> 172.21.15.42:0/797700528 conn(0x5618943eb980 msgr2=0x5618943ebd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.226+0000 7f6fcbc22740 1 -- 172.21.15.42:0/797700528 shutdown_connections 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.227+0000 7f6fcbc22740 1 -- 172.21.15.42:0/797700528 wait complete. 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.227+0000 7f6fcbc22740 1 -- start start 2024-03-12T04:16:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.227+0000 7f6fcbc22740 1 Processor -- start 2024-03-12T04:16:40.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561894359020 con 0x5618943fa750 2024-03-12T04:16:40.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189402c430 con 0x5618943f1250 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561893fe8e00 con 0x5618943f1b70 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fcbc22740 10 client.0 _ll_register_callbacks cb 0x561894300cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:49982/0 (socket says 172.21.15.42:49982) 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 -- 172.21.15.42:0/4168171691 learned_addr learned my addr 172.21.15.42:0/4168171691 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc2d9d640 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 -- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 msgr2=0x5618943f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 -- 172.21.15.42:0/4168171691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 msgr2=0x5618943f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.228+0000 7f6fc259c640 1 -- 172.21.15.42:0/4168171691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc4114f10 con 0x5618943f1250 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fc259c640 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f77a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005e80 tx=0x7f6fb4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67df6ceddb54b7f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:40.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/4168171691 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4004af0 con 0x5618943f1250 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/4168171691 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fb402de40 con 0x5618943f1250 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618940587e0 con 0x5618943f1250 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/4168171691 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fb402cac0 con 0x5618943f1250 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fcbc22740 10 client.25481 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5618943576a0 con 0x5618943f1250 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.229+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/4168171691 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5618940587e0 con 0x5618943f1250 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/4168171691 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fb403c050 con 0x5618943f1250 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fcbc22740 10 client.25481 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fcbc22740 10 client.25481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fcbc22740 20 client.25481 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56189440a6d0 0x56189440cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fcbc22740 4 client.25481 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561894095720 con 0x56189440a6d0 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.230+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56189440a6d0 0x56189440cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.231+0000 7f6fa9ffb640 10 client.25481 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:40.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.231+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56189440a6d0 0x56189440cac0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3917f1f441bccc0a server_cookie=16c14b337d5b7061 in_seq=0 out_seq=0 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.231+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/4168171691 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561894095720 con 0x56189440a6d0 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.231+0000 7f6fa9ffb640 10 client.25481 handle_command_reply: tid=0 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.231+0000 7f6fcbc22740 1 client.25481 shutdown 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 msgr2=0x5618943f77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f77a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005e80 tx=0x7f6fb4005fd0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 shutdown_connections 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56189440a6d0 0x56189440cac0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1b70 0x5618943f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/4168171691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1250 0x5618943f77a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 >> 172.21.15.42:0/4168171691 conn(0x5618943eb980 msgr2=0x5618943ed6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fa9ffb640 0 client.25481 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:40.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 shutdown_connections 2024-03-12T04:16:40.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 1 -- 172.21.15.42:0/4168171691 wait complete. 2024-03-12T04:16:40.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.232+0000 7f6fcbc22740 20 client.25481 trim_cache size 0 max 16384 2024-03-12T04:16:40.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.256+0000 7f6fcbc22740 1 Processor -- start 2024-03-12T04:16:40.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.256+0000 7f6fcbc22740 1 -- start start 2024-03-12T04:16:40.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.256+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.256+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.256+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618944122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618942de130 con 0x5618943fa750 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618943ec9e0 con 0x5618943f1b70 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fcbc22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618943f0650 con 0x5618943f1250 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33938/0 (socket says 172.21.15.42:33938) 2024-03-12T04:16:40.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 -- 172.21.15.42:0/2175916840 learned_addr learned my addr 172.21.15.42:0/2175916840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc2d9d640 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618944122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 -- 172.21.15.42:0/2175916840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 msgr2=0x5618943f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 -- 172.21.15.42:0/2175916840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 msgr2=0x5618944122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618944122d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 -- 172.21.15.42:0/2175916840 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618942f3c20 con 0x5618943f1250 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fc259c640 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f9ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4061bd0 tx=0x7f6fb4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dafeb57fa485f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fabfff640 1 -- 172.21.15.42:0/2175916840 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4004af0 con 0x5618943f1250 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fabfff640 1 -- 172.21.15.42:0/2175916840 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fb4004fd0 con 0x5618943f1250 2024-03-12T04:16:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.257+0000 7f6fabfff640 1 -- 172.21.15.42:0/2175916840 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4061910 con 0x5618943f1250 2024-03-12T04:16:40.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 -- 172.21.15.42:0/2175916840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 msgr2=0x5618943f9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f9ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4061bd0 tx=0x7f6fb4009780 comp rx=0 tx=0).stop 2024-03-12T04:16:40.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 -- 172.21.15.42:0/2175916840 shutdown_connections 2024-03-12T04:16:40.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618944122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/2175916840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f9ed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 -- 172.21.15.42:0/2175916840 >> 172.21.15.42:0/2175916840 conn(0x5618943eb980 msgr2=0x5618943f6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 -- 172.21.15.42:0/2175916840 shutdown_connections 2024-03-12T04:16:40.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.260+0000 7f6fcbc22740 1 -- 172.21.15.42:0/2175916840 wait complete. 2024-03-12T04:16:40.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.261+0000 7f6fcbc22740 1 -- start start 2024-03-12T04:16:40.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.261+0000 7f6fcbc22740 1 Processor -- start 2024-03-12T04:16:40.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.261+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.261+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.261+0000 7f6fcbc22740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33952/0 (socket says 172.21.15.42:33952) 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 -- 172.21.15.42:0/862741565 learned_addr learned my addr 172.21.15.42:0/862741565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189445fbc0 con 0x5618943fa750 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189445fd30 con 0x5618943f1b70 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189445fef0 con 0x5618943f1250 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 -- 172.21.15.42:0/862741565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 msgr2=0x5618943f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fcbc22740 10 client.0 _ll_register_callbacks cb 0x7f6fb8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 -- 172.21.15.42:0/862741565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 msgr2=0x5618943f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fc259c640 1 -- 172.21.15.42:0/862741565 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618943576a0 con 0x5618943f1250 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.262+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618943e70f0 con 0x5618943f1250 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fc259c640 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f4710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6fb402c630 tx=0x7f6fb4034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34eb7141c7043ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fa9ffb640 10 client.25375 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/862741565 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4005590 con 0x5618943f1250 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/862741565 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fb4000e40 con 0x5618943f1250 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/862741565 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fb40056f0 con 0x5618943f1250 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 10 client.25375 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5618943198c0 con 0x5618943f1250 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/862741565 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fb403c040 con 0x5618943f1250 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 10 client.25375 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 10 client.25375 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 20 client.25375 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5618944610e0 0x5618944634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 4 client.25375 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.263+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f6fb402cc20 con 0x5618944610e0 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5618944610e0 0x5618944634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fc1d9b640 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5618944610e0 0x5618944634d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1101626e9fc55289 server_cookie=7bbd4e8b23ebddff in_seq=0 out_seq=0 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fa9ffb640 10 client.25375 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fa9ffb640 1 -- 172.21.15.42:0/862741565 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f6fb402cc20 con 0x5618944610e0 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fa9ffb640 10 client.25375 handle_command_reply: tid=0 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fcbc22740 1 client.25375 shutdown 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 msgr2=0x5618943f4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.264+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f4710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6fb402c630 tx=0x7f6fb4034440 comp rx=0 tx=0).stop 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 shutdown_connections 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5618944610e0 0x5618944634d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fa9ffb640 0 client.25375 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:16:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5618943fa750 0x5618943f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618943f1b70 0x5618943f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 --2- 172.21.15.42:0/862741565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618943f1250 0x5618943f4710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 >> 172.21.15.42:0/862741565 conn(0x5618943eb980 msgr2=0x56189440d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 shutdown_connections 2024-03-12T04:16:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 1 -- 172.21.15.42:0/862741565 wait complete. 2024-03-12T04:16:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.265+0000 7f6fcbc22740 20 client.25375 trim_cache size 0 max 16384 2024-03-12T04:16:40.275 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:40.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6fac0a1d40 msgr2=0x7f6fac0a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6fac0a1d40 0x7f6fac0a4200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005fd0 tx=0x7f6fbc004ab0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fc4103c40 msgr2=0x7f6fc41a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fc4103c40 0x7f6fc41a3c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800e9f0 tx=0x7f6fb800edf0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 shutdown_connections 2024-03-12T04:16:40.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6fac0a1d40 0x7f6fac0a4200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc41060d0 0x7f6fc419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fc4103c40 0x7f6fc41a3c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 --2- 172.21.15.42:0/237983572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc40688c0 0x7f6fc41a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 >> 172.21.15.42:0/237983572 conn(0x7f6fc4077180 msgr2=0x7f6fc40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 shutdown_connections 2024-03-12T04:16:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.266+0000 7f6fc8f77640 1 -- 172.21.15.42:0/237983572 wait complete. 2024-03-12T04:16:40.286 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:16:40.286 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:16:40.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 -- 172.21.15.42:0/1336626107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 msgr2=0x7fac2010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 --2- 172.21.15.42:0/1336626107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2010db10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac10009930 tx=0x7fac1002bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 -- 172.21.15.42:0/1336626107 shutdown_connections 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 --2- 172.21.15.42:0/1336626107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac201080e0 0x7fac201145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 --2- 172.21.15.42:0/1336626107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20107720 0x7fac20107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 --2- 172.21.15.42:0/1336626107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2010db10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 -- 172.21.15.42:0/1336626107 >> 172.21.15.42:0/1336626107 conn(0x7fac200756c0 msgr2=0x7fac20077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.441+0000 7fac25fb5640 1 -- 172.21.15.42:0/1336626107 shutdown_connections 2024-03-12T04:16:40.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.442+0000 7fac25fb5640 1 -- 172.21.15.42:0/1336626107 wait complete. 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.442+0000 7fac25fb5640 1 Processor -- start 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac25fb5640 1 -- start start 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac25fb5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac20107720 0x7fac201a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac25fb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac201080e0 0x7fac201a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac25fb5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac25fb5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2019e2a0 con 0x7fac20107720 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac25fb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2019e410 con 0x7fac201080e0 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac25fb5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2019e5d0 con 0x7fac2010d730 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2019db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33968/0 (socket says 172.21.15.42:33968) 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac1ffff640 1 -- 172.21.15.42:0/3603423310 learned_addr learned my addr 172.21.15.42:0/3603423310 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac1effd640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac201080e0 0x7fac201a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.443+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac20107720 0x7fac201a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1ffff640 1 -- 172.21.15.42:0/3603423310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac201080e0 msgr2=0x7fac201a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1ffff640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac201080e0 0x7fac201a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1ffff640 1 -- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac20107720 msgr2=0x7fac201a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1ffff640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac20107720 0x7fac201a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1ffff640 1 -- 172.21.15.42:0/3603423310 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac10009560 con 0x7fac2010d730 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1ffff640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2019db10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac1800eb10 tx=0x7fac1800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1cff9640 1 -- 172.21.15.42:0/3603423310 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18004b20 con 0x7fac2010d730 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1cff9640 1 -- 172.21.15.42:0/3603423310 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac18004c80 con 0x7fac2010d730 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac1cff9640 1 -- 172.21.15.42:0/3603423310 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18010690 con 0x7fac2010d730 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2019e8b0 con 0x7fac2010d730 2024-03-12T04:16:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.444+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201aa9d0 con 0x7fac2010d730 2024-03-12T04:16:40.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.447+0000 7fac1cff9640 1 -- 172.21.15.42:0/3603423310 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fac18005020 con 0x7fac2010d730 2024-03-12T04:16:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 Processor -- start 2024-03-12T04:16:40.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac1cff9640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fac080a1d40 0x7fac080a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 -- start start 2024-03-12T04:16:40.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba6b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba76b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac1effd640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fac080a1d40 0x7fac080a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137ba26960 con 0x56137ba6bb70 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137b9da310 con 0x56137ba74750 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac28b88740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137b9d7840 con 0x56137ba6b250 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac1effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac1cff9640 1 -- 172.21.15.42:0/3603423310 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fac18014070 con 0x7fac2010d730 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac1effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba74210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45282/0 (socket says 172.21.15.42:45282) 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac1effd640 1 -- 172.21.15.42:0/2800675025 learned_addr learned my addr 172.21.15.42:0/2800675025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.449+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba6b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1ffff640 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba76b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1f7fe640 1 -- 172.21.15.42:0/2800675025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 msgr2=0x56137ba76b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba76b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1f7fe640 1 -- 172.21.15.42:0/2800675025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 msgr2=0x56137ba74210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba74210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1f7fe640 1 -- 172.21.15.42:0/2800675025 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56137b995060 con 0x56137ba6b250 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1effd640 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba74210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1effd640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fac080a1d40 0x7fac080a4200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac14013a10 tx=0x7fac14009880 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba6b630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac10009230 tx=0x7fac10005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f0121f00ab44faa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.450+0000 7fac05ffb640 1 -- 172.21.15.42:0/2800675025 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10004a00 con 0x56137ba6b250 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.451+0000 7fac05ffb640 1 -- 172.21.15.42:0/2800675025 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac1002de90 con 0x56137ba6b250 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.451+0000 7fac05ffb640 1 -- 172.21.15.42:0/2800675025 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100038e0 con 0x56137ba6b250 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 -- 172.21.15.42:0/2800675025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 msgr2=0x56137ba6b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba6b630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac10009230 tx=0x7fac10005d50 comp rx=0 tx=0).stop 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 -- 172.21.15.42:0/2800675025 shutdown_connections 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba76b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba74210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 --2- 172.21.15.42:0/2800675025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba6b630 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 -- 172.21.15.42:0/2800675025 >> 172.21.15.42:0/2800675025 conn(0x56137ba65980 msgr2=0x56137ba65d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 -- 172.21.15.42:0/2800675025 shutdown_connections 2024-03-12T04:16:40.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.452+0000 7fac28b88740 1 -- 172.21.15.42:0/2800675025 wait complete. 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.453+0000 7fac28b88740 1 -- start start 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.453+0000 7fac28b88740 1 Processor -- start 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.453+0000 7fac28b88740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac28b88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba6bb70 0x56137ba71c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33982/0 (socket says 172.21.15.42:33982) 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3783661197 learned_addr learned my addr 172.21.15.42:0/3783661197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac28b88740 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba74750 0x56137ba72160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137b9d3020 con 0x56137ba74750 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137b6a6430 con 0x56137ba6bb70 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137b662e00 con 0x56137ba6b250 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1effd640 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba6bb70 0x56137ba71c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac28b88740 10 client.0 _ll_register_callbacks cb 0x56137b97acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:40.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba6bb70 msgr2=0x56137ba71c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba6bb70 0x56137ba71c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3783661197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba74750 msgr2=0x56137ba72160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba74750 0x56137ba72160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3783661197 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac20114ef0 con 0x56137ba6b250 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.454+0000 7fac1effd640 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba6bb70 0x56137ba71c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba716e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac10005ab0 tx=0x7fac1002c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5511db2cbd9e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fabfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fabfeffd640 1 -- 172.21.15.42:0/3783661197 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10004230 con 0x56137ba6b250 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fabfeffd640 1 -- 172.21.15.42:0/3783661197 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fac10004390 con 0x56137ba6b250 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56137b6d27e0 con 0x56137ba6b250 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fabfeffd640 1 -- 172.21.15.42:0/3783661197 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac10003660 con 0x56137ba6b250 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fac28b88740 10 client.25387 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.455+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56137b9d16a0 con 0x56137ba6b250 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fabfeffd640 1 -- 172.21.15.42:0/3783661197 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac10003c70 con 0x56137ba6b250 2024-03-12T04:16:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fabfeffd640 1 -- 172.21.15.42:0/3783661197 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac1003c040 con 0x56137ba6b250 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fac28b88740 10 client.25387 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fac28b88740 10 client.25387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fac28b88740 20 client.25387 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fac28b88740 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137baa67a0 0x56137baa8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fac28b88740 4 client.25387 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56137b6d27e0 con 0x56137baa67a0 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.456+0000 7fac1effd640 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137baa67a0 0x56137baa8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.457+0000 7fac1effd640 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137baa67a0 0x56137baa8c40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e739e057f48ba0c1 server_cookie=c55754b3bb99b14c in_seq=0 out_seq=0 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.457+0000 7fabfeffd640 10 client.25387 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.457+0000 7fabfeffd640 1 -- 172.21.15.42:0/3783661197 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56137b6d27e0 con 0x56137baa67a0 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.457+0000 7fabfeffd640 10 client.25387 handle_command_reply: tid=0 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.457+0000 7fac28b88740 1 client.25387 shutdown 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 msgr2=0x56137ba716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba716e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac10005ab0 tx=0x7fac1002c9a0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 shutdown_connections 2024-03-12T04:16:40.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137baa67a0 0x56137baa8c40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba74750 0x56137ba72160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba6bb70 0x56137ba71c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 --2- 172.21.15.42:0/3783661197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba716e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 >> 172.21.15.42:0/3783661197 conn(0x56137ba65980 msgr2=0x56137ba676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fabfeffd640 0 client.25387 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:40.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.458+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 shutdown_connections 2024-03-12T04:16:40.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.459+0000 7fac28b88740 1 -- 172.21.15.42:0/3783661197 wait complete. 2024-03-12T04:16:40.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.459+0000 7fac28b88740 20 client.25387 trim_cache size 0 max 16384 2024-03-12T04:16:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 Processor -- start 2024-03-12T04:16:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 -- start start 2024-03-12T04:16:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba73e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba6e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba6ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137b958130 con 0x56137ba6bb70 2024-03-12T04:16:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137ba76fa0 con 0x56137ba74750 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.479+0000 7fac28b88740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137baa35e0 con 0x56137ba6b250 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba73e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba6e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba73e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33986/0 (socket says 172.21.15.42:33986) 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3894695254 learned_addr learned my addr 172.21.15.42:0/3894695254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1ffff640 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba6ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3894695254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 msgr2=0x56137ba6ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba6ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3894695254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 msgr2=0x56137ba6e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba6e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 -- 172.21.15.42:0/3894695254 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56137b96dc20 con 0x56137ba6b250 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba73e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac1002c170 tx=0x7fac1002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1c5f89e51b10a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac057fa640 1 -- 172.21.15.42:0/3894695254 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1002cc50 con 0x56137ba6b250 2024-03-12T04:16:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.480+0000 7fac057fa640 1 -- 172.21.15.42:0/3894695254 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac10004440 con 0x56137ba6b250 2024-03-12T04:16:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.481+0000 7fac057fa640 1 -- 172.21.15.42:0/3894695254 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10003c60 con 0x56137ba6b250 2024-03-12T04:16:40.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.481+0000 7fac28b88740 1 -- 172.21.15.42:0/3894695254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 msgr2=0x56137ba73e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.481+0000 7fac28b88740 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba73e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac1002c170 tx=0x7fac1002bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 -- 172.21.15.42:0/3894695254 shutdown_connections 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba6ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba6e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 --2- 172.21.15.42:0/3894695254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba73e10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 -- 172.21.15.42:0/3894695254 >> 172.21.15.42:0/3894695254 conn(0x56137ba65980 msgr2=0x56137baad5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 -- 172.21.15.42:0/3894695254 shutdown_connections 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 -- 172.21.15.42:0/3894695254 wait complete. 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 -- start start 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 Processor -- start 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba9c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.482+0000 7fac28b88740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba9c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac28b88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba9caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac28b88740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137bafcd30 con 0x56137ba6bb70 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac28b88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137bafce70 con 0x56137ba74750 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac28b88740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137bafcfb0 con 0x56137ba6b250 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba9c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba9c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45324/0 (socket says 172.21.15.42:45324) 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 -- 172.21.15.42:0/2492050878 learned_addr learned my addr 172.21.15.42:0/2492050878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac28b88740 10 client.0 _ll_register_callbacks cb 0x7fac180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba9c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1ffff640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba9caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 -- 172.21.15.42:0/2492050878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 msgr2=0x56137ba9c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba9c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 -- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 msgr2=0x56137ba9caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba9caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 -- 172.21.15.42:0/2492050878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56137b9d16a0 con 0x56137ba6bb70 2024-03-12T04:16:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba9c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fac1effd640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba9c5b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fac140107a0 tx=0x7fac14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32992f1f039c974 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fabfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.483+0000 7fabfeffd640 1 -- 172.21.15.42:0/2492050878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14013720 con 0x56137ba6bb70 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fabfeffd640 1 -- 172.21.15.42:0/2492050878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fac14027050 con 0x56137ba6bb70 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac1002de90 con 0x56137ba6bb70 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fabfeffd640 1 -- 172.21.15.42:0/2492050878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac140130e0 con 0x56137ba6bb70 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 10 client.16005 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56137b9938c0 con 0x56137ba6bb70 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fabfeffd640 1 -- 172.21.15.42:0/2492050878 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac14029730 con 0x56137ba6bb70 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fabfeffd640 1 -- 172.21.15.42:0/2492050878 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac14005130 con 0x56137ba6bb70 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 20 client.16005 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137bafe820 0x56137bb00c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 4 client.16005 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fac1002de90 con 0x56137bafe820 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.484+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137bafe820 0x56137bb00c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fac1f7fe640 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137bafe820 0x56137bb00c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9c01020f9d1105 server_cookie=8f2f34668b7b0b6 in_seq=0 out_seq=0 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fabfeffd640 10 client.16005 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fabfeffd640 1 -- 172.21.15.42:0/2492050878 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fac1002de90 con 0x56137bafe820 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fabfeffd640 10 client.16005 handle_command_reply: tid=0 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fac28b88740 1 client.16005 shutdown 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 msgr2=0x56137ba9c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fac28b88740 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba9c5b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fac140107a0 tx=0x7fac14004730 comp rx=0 tx=0).stop 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 shutdown_connections 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.485+0000 7fac28b88740 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56137bafe820 0x56137bb00c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac28b88740 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56137ba74750 0x56137ba9caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac28b88740 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56137ba6bb70 0x56137ba9c5b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac28b88740 1 --2- 172.21.15.42:0/2492050878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56137ba6b250 0x56137ba9c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fabfeffd640 0 client.16005 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 >> 172.21.15.42:0/2492050878 conn(0x56137ba65980 msgr2=0x56137ba642f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 shutdown_connections 2024-03-12T04:16:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac28b88740 1 -- 172.21.15.42:0/2492050878 wait complete. 2024-03-12T04:16:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac28b88740 20 client.16005 trim_cache size 0 max 16384 2024-03-12T04:16:40.494 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:16:40.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fac080a1d40 msgr2=0x7fac080a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fac080a1d40 0x7fac080a4200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac14013a10 tx=0x7fac14009880 comp rx=0 tx=0).stop 2024-03-12T04:16:40.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 msgr2=0x7fac2019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:40.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2019db10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac1800eb10 tx=0x7fac1800c2d0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 shutdown_connections 2024-03-12T04:16:40.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fac080a1d40 0x7fac080a4200 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2010d730 0x7fac2019db10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac201080e0 0x7fac201a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 --2- 172.21.15.42:0/3603423310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac20107720 0x7fac201a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:40.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.486+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 >> 172.21.15.42:0/3603423310 conn(0x7fac200756c0 msgr2=0x7fac200760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:40.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.487+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 shutdown_connections 2024-03-12T04:16:40.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:40.487+0000 7fac25fb5640 1 -- 172.21.15.42:0/3603423310 wait complete. 2024-03-12T04:16:41.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:40 smithi042 ceph-mon[32260]: pgmap v47: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 3.44k op/s 2024-03-12T04:16:41.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:40 smithi119 ceph-mon[36119]: pgmap v47: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 3.44k op/s 2024-03-12T04:16:41.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:40 smithi136 ceph-mon[35571]: pgmap v47: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 3.44k op/s 2024-03-12T04:16:41.507 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:16:41.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:16:41.508 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:16:41.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 -- 172.21.15.42:0/1726632336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 msgr2=0x7f25d0108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 --2- 172.21.15.42:0/1726632336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d0108740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009900 tx=0x7f25c002bc30 comp rx=0 tx=0).stop 2024-03-12T04:16:41.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 -- 172.21.15.42:0/1726632336 shutdown_connections 2024-03-12T04:16:41.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 --2- 172.21.15.42:0/1726632336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01142d0 0x7f25d01166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 --2- 172.21.15.42:0/1726632336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0108c80 0x7f25d0113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 --2- 172.21.15.42:0/1726632336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d0108740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 -- 172.21.15.42:0/1726632336 >> 172.21.15.42:0/1726632336 conn(0x7f25d006c260 msgr2=0x7f25d006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 -- 172.21.15.42:0/1726632336 shutdown_connections 2024-03-12T04:16:41.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.670+0000 7f25d689d640 1 -- 172.21.15.42:0/1726632336 wait complete. 2024-03-12T04:16:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.671+0000 7f25d689d640 1 Processor -- start 2024-03-12T04:16:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.671+0000 7f25d689d640 1 -- start start 2024-03-12T04:16:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.671+0000 7f25d689d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.671+0000 7f25d689d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0108c80 0x7f25d01a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25d689d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01142d0 0x7f25d01a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25d689d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01a4dc0 con 0x7f25d0108c80 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25d689d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01a4f30 con 0x7f25d0108360 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25d689d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01a50f0 con 0x7f25d01142d0 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:34862/0 (socket says 172.21.15.42:34862) 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 -- 172.21.15.42:0/2242901124 learned_addr learned my addr 172.21.15.42:0/2242901124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0108c80 0x7f25d01a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01142d0 0x7f25d01a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 -- 172.21.15.42:0/2242901124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01142d0 msgr2=0x7f25d01a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01142d0 0x7f25d01a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 -- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0108c80 msgr2=0x7f25d01a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0108c80 0x7f25d01a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.672+0000 7f25cffff640 1 -- 172.21.15.42:0/2242901124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009490 con 0x7f25d0108360 2024-03-12T04:16:41.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.673+0000 7f25cffff640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d019ff00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25c0006010 tx=0x7f25c0005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.673+0000 7f25cd7fa640 1 -- 172.21.15.42:0/2242901124 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c00043d0 con 0x7f25d0108360 2024-03-12T04:16:41.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.673+0000 7f25cd7fa640 1 -- 172.21.15.42:0/2242901124 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25c002de90 con 0x7f25d0108360 2024-03-12T04:16:41.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.673+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c0009560 con 0x7f25d0108360 2024-03-12T04:16:41.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.673+0000 7f25cd7fa640 1 -- 172.21.15.42:0/2242901124 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0005220 con 0x7f25d0108360 2024-03-12T04:16:41.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.674+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d0109b20 con 0x7f25d0108360 2024-03-12T04:16:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.677+0000 7f25cd7fa640 1 -- 172.21.15.42:0/2242901124 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f25c0003c30 con 0x7f25d0108360 2024-03-12T04:16:41.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 Processor -- start 2024-03-12T04:16:41.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 -- start start 2024-03-12T04:16:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8380 0x55a95c1c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c1d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c136f90 con 0x55a95c1c8380 2024-03-12T04:16:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c1344c0 con 0x55a95c1d18b0 2024-03-12T04:16:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d9472740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c12fca0 con 0x55a95c1c8d30 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25cd7fa640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f25b40a1cc0 0x7f25b40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d4e3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8380 0x55a95c1c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d4e3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8380 0x55a95c1c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59892/0 (socket says 172.21.15.42:59892) 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25d4e3a640 1 -- 172.21.15.42:0/3782369082 learned_addr learned my addr 172.21.15.42:0/3782369082 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c1d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25cffff640 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.679+0000 7f25cd7fa640 1 -- 172.21.15.42:0/2242901124 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f25c00ef950 con 0x7f25d0108360 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.678+0000 7f25cf7fe640 1 -- 172.21.15.42:0/3782369082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 msgr2=0x55a95c1d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.679+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.679+0000 7f25cf7fe640 1 -- 172.21.15.42:0/3782369082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8380 msgr2=0x55a95c1c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.679+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8380 0x55a95c1c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.679+0000 7f25cf7fe640 1 -- 172.21.15.42:0/3782369082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a95c0f20b0 con 0x55a95c1d18b0 2024-03-12T04:16:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.679+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f25b40a1cc0 0x7f25b40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.679+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c1d3d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009d80 tx=0x7f25bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd43d074a9499446 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.680+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f25b40a1cc0 0x7f25b40a4180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25c8005c40 tx=0x7f25c8004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.680+0000 7f25c67fc640 1 -- 172.21.15.42:0/3782369082 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc019410 con 0x55a95c1d18b0 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.680+0000 7f25c67fc640 1 -- 172.21.15.42:0/3782369082 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25bc019a20 con 0x55a95c1d18b0 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.680+0000 7f25c67fc640 1 -- 172.21.15.42:0/3782369082 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc018a60 con 0x55a95c1d18b0 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 -- 172.21.15.42:0/3782369082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 msgr2=0x55a95c1d3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c1d3d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009d80 tx=0x7f25bc011810 comp rx=0 tx=0).stop 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 -- 172.21.15.42:0/3782369082 shutdown_connections 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c1d3d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 --2- 172.21.15.42:0/3782369082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8380 0x55a95c1c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 -- 172.21.15.42:0/3782369082 >> 172.21.15.42:0/3782369082 conn(0x55a95c1c2840 msgr2=0x55a95c1c2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 -- 172.21.15.42:0/3782369082 shutdown_connections 2024-03-12T04:16:41.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.681+0000 7f25d9472740 1 -- 172.21.15.42:0/3782369082 wait complete. 2024-03-12T04:16:41.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.682+0000 7f25d9472740 1 -- start start 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.682+0000 7f25d9472740 1 Processor -- start 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.682+0000 7f25d9472740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.682+0000 7f25d9472740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.682+0000 7f25d9472740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1d18b0 0x55a95c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.682+0000 7f25d9472740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c0d4f40 con 0x55a95c1d18b0 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25d9472740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95be034b0 con 0x55a95c1c8380 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25d9472740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c0b5670 con 0x55a95c1c8d30 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25d9472740 10 client.0 _ll_register_callbacks cb 0x55a95c122f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:34890/0 (socket says 172.21.15.42:34890) 2024-03-12T04:16:41.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 -- 172.21.15.42:0/2063671423 learned_addr learned my addr 172.21.15.42:0/2063671423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 -- 172.21.15.42:0/2063671423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 msgr2=0x55a95c1cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 -- 172.21.15.42:0/2063671423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1d18b0 msgr2=0x55a95c1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1d18b0 0x55a95c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 -- 172.21.15.42:0/2063671423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc0099e0 con 0x55a95c1c8380 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.683+0000 7f25cffff640 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1ce8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25c00bc3c0 tx=0x7f25c0049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54680cbb2498bc3b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.684+0000 7f25b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.684+0000 7f25b37fe640 1 -- 172.21.15.42:0/2063671423 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c00bc430 con 0x55a95c1c8380 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.684+0000 7f25b37fe640 1 -- 172.21.15.42:0/2063671423 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f25c00c0580 con 0x55a95c1c8380 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.684+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a95c109f40 con 0x55a95c1c8380 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.685+0000 7f25b37fe640 1 -- 172.21.15.42:0/2063671423 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25c00c4bd0 con 0x55a95c1c8380 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.685+0000 7f25d9472740 10 client.25517 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:41.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.685+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a95c12e320 con 0x55a95c1c8380 2024-03-12T04:16:41.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.685+0000 7f25b37fe640 1 -- 172.21.15.42:0/2063671423 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a95c109f40 con 0x55a95c1c8380 2024-03-12T04:16:41.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25b37fe640 1 -- 172.21.15.42:0/2063671423 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25c003c070 con 0x55a95c1c8380 2024-03-12T04:16:41.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25d9472740 10 client.25517 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:41.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25d9472740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:41.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25d9472740 20 client.25517 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:41.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25d9472740 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c1ea480 0x55a95c1ec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25d9472740 4 client.25517 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:41.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a95c126310 con 0x55a95c1ea480 2024-03-12T04:16:41.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.686+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c1ea480 0x55a95c1ec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.687+0000 7f25b37fe640 10 client.25517 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.687+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c1ea480 0x55a95c1ec890 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d172457006ef73 server_cookie=1fe7f910ed82144b in_seq=0 out_seq=0 2024-03-12T04:16:41.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25b37fe640 1 -- 172.21.15.42:0/2063671423 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a95c126310 con 0x55a95c1ea480 2024-03-12T04:16:41.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25b37fe640 10 client.25517 handle_command_reply: tid=0 2024-03-12T04:16:41.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 client.25517 shutdown 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 msgr2=0x55a95c1ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1ce8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25c00bc3c0 tx=0x7f25c0049000 comp rx=0 tx=0).stop 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 shutdown_connections 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c1ea480 0x55a95c1ec890 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1d18b0 0x55a95c1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8d30 0x55a95c1cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 --2- 172.21.15.42:0/2063671423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1ce8d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 >> 172.21.15.42:0/2063671423 conn(0x55a95c1c2840 msgr2=0x55a95c1c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25b37fe640 0 client.25517 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 shutdown_connections 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 1 -- 172.21.15.42:0/2063671423 wait complete. 2024-03-12T04:16:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.688+0000 7f25d9472740 20 client.25517 trim_cache size 0 max 16384 2024-03-12T04:16:41.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.713+0000 7f25d9472740 1 Processor -- start 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.713+0000 7f25d9472740 1 -- start start 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25d9472740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25d9472740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25d9472740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1d18b0 0x55a95c1cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25d9472740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c109f40 con 0x55a95c1c8d30 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25d9472740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c1c3640 con 0x55a95c1c8380 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59904/0 (socket says 172.21.15.42:59904) 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:34892/0 (socket says 172.21.15.42:34892) 2024-03-12T04:16:41.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25cf7fe640 1 -- 172.21.15.42:0/2638783737 learned_addr learned my addr 172.21.15.42:0/2638783737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1d18b0 0x55a95c1cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.714+0000 7f25d9472740 1 -- 172.21.15.42:0/2638783737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c1c4db0 con 0x55a95c1d18b0 2024-03-12T04:16:41.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.715+0000 7f25cf7fe640 1 -- 172.21.15.42:0/2638783737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1d18b0 msgr2=0x55a95c1cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.715+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1d18b0 0x55a95c1cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.715+0000 7f25cf7fe640 1 -- 172.21.15.42:0/2638783737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 msgr2=0x55a95c1d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.715+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.715+0000 7f25cf7fe640 1 -- 172.21.15.42:0/2638783737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a95c0cac00 con 0x55a95c1c8d30 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.715+0000 7f25cffff640 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.715+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1cd1a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009cd0 tx=0x7f25bc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847f451c6d29a18b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.716+0000 7f25c5ffb640 1 -- 172.21.15.42:0/2638783737 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0184d0 con 0x55a95c1c8d30 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.716+0000 7f25c5ffb640 1 -- 172.21.15.42:0/2638783737 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25bc018770 con 0x55a95c1c8d30 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.716+0000 7f25c5ffb640 1 -- 172.21.15.42:0/2638783737 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc003850 con 0x55a95c1c8d30 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 -- 172.21.15.42:0/2638783737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 msgr2=0x55a95c1cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1cd1a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009cd0 tx=0x7f25bc005d30 comp rx=0 tx=0).stop 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 -- 172.21.15.42:0/2638783737 shutdown_connections 2024-03-12T04:16:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1d18b0 0x55a95c1cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1cd1a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 --2- 172.21.15.42:0/2638783737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1c8380 0x55a95c1d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 -- 172.21.15.42:0/2638783737 >> 172.21.15.42:0/2638783737 conn(0x55a95c1c2840 msgr2=0x55a95c1c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 -- 172.21.15.42:0/2638783737 shutdown_connections 2024-03-12T04:16:41.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.717+0000 7f25d9472740 1 -- 172.21.15.42:0/2638783737 wait complete. 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 -- start start 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 Processor -- start 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8380 0x55a95c1dff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c240990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c1e0d90 con 0x55a95c1c8d30 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c240ed0 con 0x55a95c1d18b0 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95c241010 con 0x55a95c1c8380 2024-03-12T04:16:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25d9472740 10 client.0 _ll_register_callbacks cb 0x7f25bc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.718+0000 7f25cffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8380 0x55a95c1dff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25cffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8380 0x55a95c1dff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54786/0 (socket says 172.21.15.42:54786) 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25cffff640 1 -- 172.21.15.42:0/3290122884 learned_addr learned my addr 172.21.15.42:0/3290122884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c240990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25cf7fe640 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25d4e3a640 1 -- 172.21.15.42:0/3290122884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8380 msgr2=0x55a95c1dff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8380 0x55a95c1dff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25d4e3a640 1 -- 172.21.15.42:0/3290122884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 msgr2=0x55a95c1e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.719+0000 7f25d4e3a640 1 -- 172.21.15.42:0/3290122884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c00d6070 con 0x55a95c1d18b0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.720+0000 7f25d4e3a640 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c240990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55a95c1c2180 tx=0x7f25c8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8beaf98fb30a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.720+0000 7f25b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.720+0000 7f25b37fe640 1 -- 172.21.15.42:0/3290122884 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8024040 con 0x55a95c1d18b0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.720+0000 7f25b37fe640 1 -- 172.21.15.42:0/3290122884 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f25c8010cd0 con 0x55a95c1d18b0 2024-03-12T04:16:41.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.720+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a95c155900 con 0x55a95c1d18b0 2024-03-12T04:16:41.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25b37fe640 1 -- 172.21.15.42:0/3290122884 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25c8018070 con 0x55a95c1d18b0 2024-03-12T04:16:41.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 10 client.25523 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:41.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a95c0f20b0 con 0x55a95c1d18b0 2024-03-12T04:16:41.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25b37fe640 1 -- 172.21.15.42:0/3290122884 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a95c155900 con 0x55a95c1d18b0 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25b37fe640 1 -- 172.21.15.42:0/3290122884 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25c801c400 con 0x55a95c1d18b0 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 10 client.25523 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 20 client.25523 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c242800 0x55a95c244c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 4 client.25523 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a95be2f860 con 0x55a95c242800 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.721+0000 7f25cffff640 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c242800 0x55a95c244c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.722+0000 7f25b37fe640 10 client.25523 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.722+0000 7f25cffff640 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c242800 0x55a95c244c20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4db8abb222cd server_cookie=da9825580b1a4426 in_seq=0 out_seq=0 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.723+0000 7f25b37fe640 1 -- 172.21.15.42:0/3290122884 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a95be2f860 con 0x55a95c242800 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.723+0000 7f25b37fe640 10 client.25523 handle_command_reply: tid=0 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.723+0000 7f25d9472740 1 client.25523 shutdown 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 msgr2=0x55a95c240990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c240990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55a95c1c2180 tx=0x7f25c8009500 comp rx=0 tx=0).stop 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 shutdown_connections 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a95c242800 0x55a95c244c20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a95c1d18b0 0x55a95c240990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a95c1c8d30 0x55a95c1e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 --2- 172.21.15.42:0/3290122884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a95c1c8380 0x55a95c1dff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 >> 172.21.15.42:0/3290122884 conn(0x55a95c1c2840 msgr2=0x55a95c1cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.724+0000 7f25b37fe640 0 client.25523 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.725+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 shutdown_connections 2024-03-12T04:16:41.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.725+0000 7f25d9472740 1 -- 172.21.15.42:0/3290122884 wait complete. 2024-03-12T04:16:41.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.726+0000 7f25d9472740 20 client.25523 trim_cache size 0 max 16384 2024-03-12T04:16:41.729 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:16:41.729 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:16:41.729 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "cef52cc2-6a97-4a11-95de-3930ebd14746", 2024-03-12T04:16:41.729 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:16:41.729 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:16:41.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f25b40a1cc0 msgr2=0x7f25b40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f25b40a1cc0 0x7f25b40a4180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25c8005c40 tx=0x7f25c8004a40 comp rx=0 tx=0).stop 2024-03-12T04:16:41.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 msgr2=0x7f25d019ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d019ff00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25c0006010 tx=0x7f25c0005e10 comp rx=0 tx=0).stop 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 shutdown_connections 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f25b40a1cc0 0x7f25b40a4180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01142d0 0x7f25d01a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0108c80 0x7f25d01a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 --2- 172.21.15.42:0/2242901124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25d0108360 0x7f25d019ff00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 >> 172.21.15.42:0/2242901124 conn(0x7f25d006c260 msgr2=0x7f25d00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 shutdown_connections 2024-03-12T04:16:41.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.727+0000 7f25d689d640 1 -- 172.21.15.42:0/2242901124 wait complete. 2024-03-12T04:16:41.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:16:41.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 -- 172.21.15.42:0/2918564648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 msgr2=0x7f9c88102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/2918564648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c88102780 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009900 tx=0x7f9c7802bcd0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 -- 172.21.15.42:0/2918564648 shutdown_connections 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/2918564648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c8810b6e0 0x7f9c8810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/2918564648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c88102cc0 0x7f9c8810b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/2918564648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c88102780 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 -- 172.21.15.42:0/2918564648 >> 172.21.15.42:0/2918564648 conn(0x7f9c88077230 msgr2=0x7f9c88077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 -- 172.21.15.42:0/2918564648 shutdown_connections 2024-03-12T04:16:41.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.896+0000 7f9c8d29d640 1 -- 172.21.15.42:0/2918564648 wait complete. 2024-03-12T04:16:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 Processor -- start 2024-03-12T04:16:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 -- start start 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c880ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c88102cc0 0x7f9c88100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8810b6e0 0x7f9c88100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c881a59c0 con 0x7f9c881023a0 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c881a5b30 con 0x7f9c88102cc0 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c8d29d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c881a5cf0 con 0x7f9c8810b6e0 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c880ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c880ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59938/0 (socket says 172.21.15.42:59938) 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c88102cc0 0x7f9c88100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.897+0000 7f9c86ffd640 1 -- 172.21.15.42:0/850634856 learned_addr learned my addr 172.21.15.42:0/850634856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c877fe640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8810b6e0 0x7f9c88100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c867fc640 1 -- 172.21.15.42:0/850634856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8810b6e0 msgr2=0x7f9c88100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c867fc640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8810b6e0 0x7f9c88100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c867fc640 1 -- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 msgr2=0x7f9c880ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c867fc640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c880ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c867fc640 1 -- 172.21.15.42:0/850634856 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78009560 con 0x7f9c88102cc0 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c880ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c867fc640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c88102cc0 0x7f9c88100530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c970 tx=0x7f9c7c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c6ffff640 1 -- 172.21.15.42:0/850634856 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c007bb0 con 0x7f9c88102cc0 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c881a5fd0 con 0x7f9c88102cc0 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c6ffff640 1 -- 172.21.15.42:0/850634856 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c7c007d10 con 0x7f9c88102cc0 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.898+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c881a6450 con 0x7f9c88102cc0 2024-03-12T04:16:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.899+0000 7f9c6ffff640 1 -- 172.21.15.42:0/850634856 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c005220 con 0x7f9c88102cc0 2024-03-12T04:16:41.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c6ffff640 1 -- 172.21.15.42:0/850634856 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f9c7c005380 con 0x7f9c88102cc0 2024-03-12T04:16:41.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 Processor -- start 2024-03-12T04:16:41.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 -- start start 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e272b280 con 0x5608e27bcbd0 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e27287b0 con 0x5608e27bc2b0 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e2723f90 con 0x5608e27c57b0 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c6ffff640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9c700a1ba0 0x7f9c700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59942/0 (socket says 172.21.15.42:59942) 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c86ffd640 1 -- 172.21.15.42:0/3688174370 learned_addr learned my addr 172.21.15.42:0/3688174370 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c877fe640 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c877fe640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9c700a1ba0 0x7f9c700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c6ffff640 1 -- 172.21.15.42:0/850634856 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9c7c0cd8e0 con 0x7f9c88102cc0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.901+0000 7f9c867fc640 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c877fe640 1 -- 172.21.15.42:0/3688174370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 msgr2=0x5608e27c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c877fe640 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c877fe640 1 -- 172.21.15.42:0/3688174370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 msgr2=0x5608e27c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c877fe640 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c877fe640 1 -- 172.21.15.42:0/3688174370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e26e2570 con 0x5608e27bc2b0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c877fe640 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27bc690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9c80012930 tx=0x7f9c80012df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbf3f9376d44f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c6d7fa640 1 -- 172.21.15.42:0/3688174370 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80012250 con 0x5608e27bc2b0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c6d7fa640 1 -- 172.21.15.42:0/3688174370 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c80003e20 con 0x5608e27bc2b0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c877fe640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9c700a1ba0 0x7f9c700a4060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c80021000 tx=0x7f9c8001d000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.902+0000 7f9c6d7fa640 1 -- 172.21.15.42:0/3688174370 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80004030 con 0x5608e27bc2b0 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 -- 172.21.15.42:0/3688174370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 msgr2=0x5608e27bc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27bc690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9c80012930 tx=0x7f9c80012df0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 -- 172.21.15.42:0/3688174370 shutdown_connections 2024-03-12T04:16:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/3688174370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27bc690 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 -- 172.21.15.42:0/3688174370 >> 172.21.15.42:0/3688174370 conn(0x5608e27b6990 msgr2=0x5608e27b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 -- 172.21.15.42:0/3688174370 shutdown_connections 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 -- 172.21.15.42:0/3688174370 wait complete. 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 -- start start 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.903+0000 7f9c8fe73740 1 Processor -- start 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e23f7430 con 0x5608e27bcbd0 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e23b3e00 con 0x5608e27bc2b0 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e26a8fd0 con 0x5608e27c57b0 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c8fe73740 10 client.0 _ll_register_callbacks cb 0x5608e26fefd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59956/0 (socket says 172.21.15.42:59956) 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 -- 172.21.15.42:0/285384260 learned_addr learned my addr 172.21.15.42:0/285384260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c877fe640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 -- 172.21.15.42:0/285384260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 msgr2=0x5608e27c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 -- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 msgr2=0x5608e27c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 -- 172.21.15.42:0/285384260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c80012510 con 0x5608e27bcbd0 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c867fc640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c24d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0cb0e0 tx=0x7f9c7c099f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f815074a78d30bb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.904+0000 7f9c6aff5640 1 -- 172.21.15.42:0/285384260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c09e6b0 con 0x5608e27bcbd0 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c6aff5640 1 -- 172.21.15.42:0/285384260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9c7c0a2d00 con 0x5608e27bcbd0 2024-03-12T04:16:41.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608e24237e0 con 0x5608e27bcbd0 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c6aff5640 1 -- 172.21.15.42:0/285384260 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c7c0850c0 con 0x5608e27bcbd0 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 10 client.16032 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5608e2722610 con 0x5608e27bcbd0 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c6aff5640 1 -- 172.21.15.42:0/285384260 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5608e24237e0 con 0x5608e27bcbd0 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c6aff5640 1 -- 172.21.15.42:0/285384260 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9c7c016020 con 0x5608e27bcbd0 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 10 client.16032 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 20 client.16032 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e27d5900 0x5608e27d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 4 client.16032 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608e2460720 con 0x5608e27d5900 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.905+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e27d5900 0x5608e27d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c6aff5640 10 client.16032 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e27d5900 0x5608e27d7d20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8601b2311aaeea2a server_cookie=feb8f088c48f7a05 in_seq=0 out_seq=0 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c6aff5640 1 -- 172.21.15.42:0/285384260 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608e2460720 con 0x5608e27d5900 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c6aff5640 10 client.16032 handle_command_reply: tid=0 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 client.16032 shutdown 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 msgr2=0x5608e27c24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c24d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0cb0e0 tx=0x7f9c7c099f80 comp rx=0 tx=0).stop 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 shutdown_connections 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e27d5900 0x5608e27d7d20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27c57b0 0x5608e27c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bcbd0 0x5608e27c24d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c6aff5640 0 client.16032 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/285384260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27bc2b0 0x5608e27c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.906+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 >> 172.21.15.42:0/285384260 conn(0x5608e27b6990 msgr2=0x5608e27b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.907+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 shutdown_connections 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.907+0000 7f9c8fe73740 1 -- 172.21.15.42:0/285384260 wait complete. 2024-03-12T04:16:41.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.907+0000 7f9c8fe73740 20 client.16032 trim_cache size 0 max 16384 2024-03-12T04:16:41.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 Processor -- start 2024-03-12T04:16:41.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 -- start start 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bc2b0 0x5608e27c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c57b0 0x5608e27c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e27b7a10 con 0x5608e27bc2b0 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e27b7790 con 0x5608e27c57b0 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e27bd2e0 con 0x5608e27bcbd0 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bc2b0 0x5608e27c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bc2b0 0x5608e27c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59958/0 (socket says 172.21.15.42:59958) 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 -- 172.21.15.42:0/2610224475 learned_addr learned my addr 172.21.15.42:0/2610224475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c877fe640 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c57b0 0x5608e27c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c867fc640 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 -- 172.21.15.42:0/2610224475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 msgr2=0x5608e27c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 -- 172.21.15.42:0/2610224475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c57b0 msgr2=0x5608e27c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c57b0 0x5608e27c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.927+0000 7f9c86ffd640 1 -- 172.21.15.42:0/2610224475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e26beac0 con 0x5608e27bc2b0 2024-03-12T04:16:41.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.928+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bc2b0 0x5608e27c4c00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9c78006010 tx=0x7f9c780058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba67a1d4e9dda0c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.928+0000 7f9c6cff9640 1 -- 172.21.15.42:0/2610224475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78045040 con 0x5608e27bc2b0 2024-03-12T04:16:41.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.928+0000 7f9c6cff9640 1 -- 172.21.15.42:0/2610224475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c78047040 con 0x5608e27bc2b0 2024-03-12T04:16:41.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.928+0000 7f9c6cff9640 1 -- 172.21.15.42:0/2610224475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78003940 con 0x5608e27bc2b0 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 -- 172.21.15.42:0/2610224475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bc2b0 msgr2=0x5608e27c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bc2b0 0x5608e27c4c00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9c78006010 tx=0x7f9c780058d0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 -- 172.21.15.42:0/2610224475 shutdown_connections 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c57b0 0x5608e27c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/2610224475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27bc2b0 0x5608e27c4c00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 -- 172.21.15.42:0/2610224475 >> 172.21.15.42:0/2610224475 conn(0x5608e27b6990 msgr2=0x5608e27bd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 -- 172.21.15.42:0/2610224475 shutdown_connections 2024-03-12T04:16:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.929+0000 7f9c8fe73740 1 -- 172.21.15.42:0/2610224475 wait complete. 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 -- start start 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 Processor -- start 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27c57b0 0x5608e27c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c9ae0 0x5608e282be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e27ccf10 con 0x5608e27c57b0 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e282c340 con 0x5608e27c9ae0 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c8fe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e282c480 con 0x5608e27bcbd0 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c9ae0 0x5608e282be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27c57b0 0x5608e27c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c9ae0 0x5608e282be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:34966/0 (socket says 172.21.15.42:34966) 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.930+0000 7f9c877fe640 1 -- 172.21.15.42:0/1981109102 learned_addr learned my addr 172.21.15.42:0/1981109102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c8fe73740 10 client.0 _ll_register_callbacks cb 0x7f9c80009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c877fe640 1 -- 172.21.15.42:0/1981109102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 msgr2=0x5608e27c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c877fe640 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c877fe640 1 -- 172.21.15.42:0/1981109102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27c57b0 msgr2=0x5608e27c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c877fe640 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27c57b0 0x5608e27c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c877fe640 1 -- 172.21.15.42:0/1981109102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8810e800 con 0x5608e27c9ae0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c877fe640 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c9ae0 0x5608e282be00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009540 tx=0x7f9c80009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f5aa18bb10bc03 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c6aff5640 1 -- 172.21.15.42:0/1981109102 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80004030 con 0x5608e27c9ae0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c6aff5640 1 -- 172.21.15.42:0/1981109102 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9c8001d890 con 0x5608e27c9ae0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.931+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c80003e20 con 0x5608e27c9ae0 2024-03-12T04:16:41.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c6aff5640 1 -- 172.21.15.42:0/1981109102 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c8001da50 con 0x5608e27c9ae0 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 10 client.25541 fetch_fsmap learned FSMap version 27 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5608e26e46a0 con 0x5608e27c9ae0 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c6aff5640 1 -- 172.21.15.42:0/1981109102 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c80003e20 con 0x5608e27c9ae0 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c6aff5640 1 -- 172.21.15.42:0/1981109102 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9c8003d470 con 0x5608e27c9ae0 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 10 client.25541 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 10 client.25541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 20 client.25541 populate_metadata read hostname 'smithi042' 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e282dc00 0x5608e2830020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 4 client.25541 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.932+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608e24af890 con 0x5608e282dc00 2024-03-12T04:16:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e282dc00 0x5608e2830020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c6aff5640 10 client.25541 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c86ffd640 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e282dc00 0x5608e2830020 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80683a627435da6 server_cookie=52b95f94457ff9e1 in_seq=0 out_seq=0 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c6aff5640 1 -- 172.21.15.42:0/1981109102 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5608e24af890 con 0x5608e282dc00 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c6aff5640 10 client.25541 handle_command_reply: tid=0 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 client.25541 shutdown 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c9ae0 msgr2=0x5608e282be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c9ae0 0x5608e282be00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009540 tx=0x7f9c80009740 comp rx=0 tx=0).stop 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 shutdown_connections 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5608e282dc00 0x5608e2830020 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e27c9ae0 0x5608e282be00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c6aff5640 0 client.25541 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:16:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608e27c57b0 0x5608e27c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 --2- 172.21.15.42:0/1981109102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608e27bcbd0 0x5608e27c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.933+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 >> 172.21.15.42:0/1981109102 conn(0x5608e27b6990 msgr2=0x5608e27c0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 shutdown_connections 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8fe73740 1 -- 172.21.15.42:0/1981109102 wait complete. 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8fe73740 20 client.25541 trim_cache size 0 max 16384 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stdout: "cef52cc2-6a97-4a11-95de-3930ebd14746": { 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:16:41.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "cef52cc2-6a97-4a11-95de-3930ebd14746", 2024-03-12T04:16:41.942 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:16:41.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:16:41.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:16:41.942 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9c700a1ba0 msgr2=0x7f9c700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9c700a1ba0 0x7f9c700a4060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c80021000 tx=0x7f9c8001d000 comp rx=0 tx=0).stop 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c88102cc0 msgr2=0x7f9c88100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c88102cc0 0x7f9c88100530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c970 tx=0x7f9c7c00ce30 comp rx=0 tx=0).stop 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 shutdown_connections 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9c700a1ba0 0x7f9c700a4060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8810b6e0 0x7f9c88100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c88102cc0 0x7f9c88100530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 --2- 172.21.15.42:0/850634856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c881023a0 0x7f9c880ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.934+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 >> 172.21.15.42:0/850634856 conn(0x7f9c88077230 msgr2=0x7f9c880fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.935+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 shutdown_connections 2024-03-12T04:16:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:16:41.935+0000 7f9c8d29d640 1 -- 172.21.15.42:0/850634856 wait complete. 2024-03-12T04:16:41.955 INFO:tasks.cephfs.filesystem:scrub status for tag:cef52cc2-6a97-4a11-95de-3930ebd14746 - {'path': '/', 'tag': 'cef52cc2-6a97-4a11-95de-3930ebd14746', 'options': 'recursive,force'} 2024-03-12T04:16:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:42 smithi042 ceph-mon[32260]: pgmap v48: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 3.45k op/s 2024-03-12T04:16:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:42 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:16:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:42 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:42 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:16:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:42 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:42 smithi042 ceph-mon[32260]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-12T04:16:43.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:42 smithi119 ceph-mon[36119]: pgmap v48: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 3.45k op/s 2024-03-12T04:16:43.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:42 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:16:43.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:42 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:43.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:42 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:16:43.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:42 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:43.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:42 smithi119 ceph-mon[36119]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-12T04:16:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:42 smithi136 ceph-mon[35571]: pgmap v48: 97 pgs: 97 active+clean; 259 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 3.45k op/s 2024-03-12T04:16:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:42 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:16:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:42 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:42 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:16:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:42 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:16:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:42 smithi136 ceph-mon[35571]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-12T04:16:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:44 smithi042 ceph-mon[32260]: pgmap v49: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 5.13k op/s 2024-03-12T04:16:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:44 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:16:45.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:44 smithi119 ceph-mon[36119]: pgmap v49: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 5.13k op/s 2024-03-12T04:16:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:44 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:16:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:44 smithi136 ceph-mon[35571]: pgmap v49: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 5.13k op/s 2024-03-12T04:16:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:44 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:16:47.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:47 smithi119 ceph-mon[36119]: pgmap v50: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.54k op/s 2024-03-12T04:16:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:47 smithi136 ceph-mon[35571]: pgmap v50: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.54k op/s 2024-03-12T04:16:47.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:47 smithi042 ceph-mon[32260]: pgmap v50: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.54k op/s 2024-03-12T04:16:49.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:49 smithi119 ceph-mon[36119]: pgmap v51: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.34k op/s 2024-03-12T04:16:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:49 smithi136 ceph-mon[35571]: pgmap v51: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.34k op/s 2024-03-12T04:16:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:49 smithi042 ceph-mon[32260]: pgmap v51: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.34k op/s 2024-03-12T04:16:51.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:51 smithi119 ceph-mon[36119]: pgmap v52: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:16:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:51 smithi136 ceph-mon[35571]: pgmap v52: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:16:51.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:51 smithi042 ceph-mon[32260]: pgmap v52: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:16:53.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:53 smithi119 ceph-mon[36119]: pgmap v53: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:16:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:53 smithi136 ceph-mon[35571]: pgmap v53: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:16:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:53 smithi042 ceph-mon[32260]: pgmap v53: 97 pgs: 97 active+clean; 259 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:16:55.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:55 smithi119 ceph-mon[36119]: pgmap v54: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.26k op/s 2024-03-12T04:16:55.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:55 smithi136 ceph-mon[35571]: pgmap v54: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.26k op/s 2024-03-12T04:16:55.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:55 smithi042 ceph-mon[32260]: pgmap v54: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.26k op/s 2024-03-12T04:16:57.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:57 smithi119 ceph-mon[36119]: pgmap v55: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.58k op/s 2024-03-12T04:16:57.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:57 smithi136 ceph-mon[35571]: pgmap v55: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.58k op/s 2024-03-12T04:16:57.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:57 smithi042 ceph-mon[32260]: pgmap v55: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.58k op/s 2024-03-12T04:16:59.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:16:59 smithi119 ceph-mon[36119]: pgmap v56: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.32k op/s 2024-03-12T04:16:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:16:59 smithi136 ceph-mon[35571]: pgmap v56: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.32k op/s 2024-03-12T04:16:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:16:59 smithi042 ceph-mon[32260]: pgmap v56: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 5.32k op/s 2024-03-12T04:17:01.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:01 smithi119 ceph-mon[36119]: pgmap v57: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.53k op/s 2024-03-12T04:17:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:01 smithi136 ceph-mon[35571]: pgmap v57: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.53k op/s 2024-03-12T04:17:01.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:01 smithi042 ceph-mon[32260]: pgmap v57: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.53k op/s 2024-03-12T04:17:03.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:03 smithi119 ceph-mon[36119]: pgmap v58: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.53k op/s 2024-03-12T04:17:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:03 smithi136 ceph-mon[35571]: pgmap v58: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.53k op/s 2024-03-12T04:17:03.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:03 smithi042 ceph-mon[32260]: pgmap v58: 97 pgs: 97 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 3.53k op/s 2024-03-12T04:17:05.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:05 smithi119 ceph-mon[36119]: pgmap v59: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 5.26k op/s 2024-03-12T04:17:05.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:05 smithi136 ceph-mon[35571]: pgmap v59: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 5.26k op/s 2024-03-12T04:17:05.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:05 smithi042 ceph-mon[32260]: pgmap v59: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 5.26k op/s 2024-03-12T04:17:07.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:07 smithi119 ceph-mon[36119]: pgmap v60: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:17:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:07 smithi136 ceph-mon[35571]: pgmap v60: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:17:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:07 smithi042 ceph-mon[32260]: pgmap v60: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.48k op/s 2024-03-12T04:17:09.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:09 smithi119 ceph-mon[36119]: pgmap v61: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 5.20k op/s 2024-03-12T04:17:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:09 smithi136 ceph-mon[35571]: pgmap v61: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 5.20k op/s 2024-03-12T04:17:09.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:09 smithi042 ceph-mon[32260]: pgmap v61: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 5.20k op/s 2024-03-12T04:17:11.367 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:11 smithi042 ceph-mon[32260]: pgmap v62: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.45k op/s 2024-03-12T04:17:11.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:11 smithi119 ceph-mon[36119]: pgmap v62: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.45k op/s 2024-03-12T04:17:11.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:11 smithi136 ceph-mon[35571]: pgmap v62: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.45k op/s 2024-03-12T04:17:11.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:17:11.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:17:12.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd265ffb640 1 -- 172.21.15.42:0/3048216287 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2600329f0 con 0x7fd268106430 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd26d74f640 1 -- 172.21.15.42:0/3048216287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268106430 msgr2=0x7fd26810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd26d74f640 1 --2- 172.21.15.42:0/3048216287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268106430 0x7fd26810e000 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd260009cd0 tx=0x7fd26002c100 comp rx=0 tx=0).stop 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd26d74f640 1 -- 172.21.15.42:0/3048216287 shutdown_connections 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd26d74f640 1 --2- 172.21.15.42:0/3048216287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268106430 0x7fd26810e000 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd26d74f640 1 --2- 172.21.15.42:0/3048216287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268101d10 0x7fd268105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd26d74f640 1 --2- 172.21.15.42:0/3048216287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd268101360 0x7fd268101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.111+0000 7fd26d74f640 1 -- 172.21.15.42:0/3048216287 >> 172.21.15.42:0/3048216287 conn(0x7fd2680fcc20 msgr2=0x7fd2680ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.112+0000 7fd26d74f640 1 -- 172.21.15.42:0/3048216287 shutdown_connections 2024-03-12T04:17:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.112+0000 7fd26d74f640 1 -- 172.21.15.42:0/3048216287 wait complete. 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.112+0000 7fd26d74f640 1 Processor -- start 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd26d74f640 1 -- start start 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd26d74f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268101360 0x7fd268195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd26d74f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268106430 0x7fd268195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd26d74f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd268199e00 0x7fd26819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd26d74f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26819a7f0 con 0x7fd268101360 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd26d74f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26819a960 con 0x7fd268199e00 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd26d74f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26819aad0 con 0x7fd268106430 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268106430 0x7fd268195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268106430 0x7fd268195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59344/0 (socket says 172.21.15.42:59344) 2024-03-12T04:17:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd2667fc640 1 -- 172.21.15.42:0/1139023812 learned_addr learned my addr 172.21.15.42:0/1139023812 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.113+0000 7fd266ffd640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268101360 0x7fd268195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2677fe640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd268199e00 0x7fd26819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2667fc640 1 -- 172.21.15.42:0/1139023812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd268199e00 msgr2=0x7fd26819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2667fc640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd268199e00 0x7fd26819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2667fc640 1 -- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268101360 msgr2=0x7fd268195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2667fc640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268101360 0x7fd268195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2667fc640 1 -- 172.21.15.42:0/1139023812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c011ca0 con 0x7fd268106430 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2677fe640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd268199e00 0x7fd26819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd266ffd640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268101360 0x7fd268195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.114+0000 7fd2667fc640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268106430 0x7fd268195a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd25c014370 tx=0x7fd25c014830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.115+0000 7fd24ffff640 1 -- 172.21.15.42:0/1139023812 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c01f070 con 0x7fd268106430 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.115+0000 7fd24ffff640 1 -- 172.21.15.42:0/1139023812 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd25c017040 con 0x7fd268106430 2024-03-12T04:17:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.115+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd260009900 con 0x7fd268106430 2024-03-12T04:17:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.115+0000 7fd24ffff640 1 -- 172.21.15.42:0/1139023812 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c016050 con 0x7fd268106430 2024-03-12T04:17:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.115+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2681a23d0 con 0x7fd268106430 2024-03-12T04:17:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.118+0000 7fd24ffff640 1 -- 172.21.15.42:0/1139023812 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd25c011570 con 0x7fd268106430 2024-03-12T04:17:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.118+0000 7fd270402740 1 Processor -- start 2024-03-12T04:17:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.118+0000 7fd270402740 1 -- start start 2024-03-12T04:17:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd270402740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a8c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd270402740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24a95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd270402740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24a97ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd270402740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e249fb280 con 0x558e24a8c2b0 2024-03-12T04:17:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd270402740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e249f87b0 con 0x558e24a957b0 2024-03-12T04:17:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd270402740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e249f3f90 con 0x558e24a8cbd0 2024-03-12T04:17:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24a97ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd266ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24a95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24a97ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56110/0 (socket says 172.21.15.42:56110) 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 -- 172.21.15.42:0/376992506 learned_addr learned my addr 172.21.15.42:0/376992506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2677fe640 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a8c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 -- 172.21.15.42:0/376992506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 msgr2=0x558e24a95270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24a95270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 -- 172.21.15.42:0/376992506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 msgr2=0x558e24a8c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a8c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2667fc640 1 -- 172.21.15.42:0/376992506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e249b2570 con 0x558e24a957b0 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.119+0000 7fd2677fe640 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a8c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.120+0000 7fd24ffff640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2500a1c20 0x7fd2500a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.120+0000 7fd2677fe640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2500a1c20 0x7fd2500a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.120+0000 7fd2667fc640 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24a97ba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0d2a80 tx=0x7fd25c0dd880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c0bcdc2e5b50442 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.120+0000 7fd24ffff640 1 -- 172.21.15.42:0/1139023812 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd25c0d5400 con 0x7fd268106430 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.120+0000 7fd24d7fa640 1 -- 172.21.15.42:0/376992506 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c09e2c0 con 0x558e24a957b0 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.120+0000 7fd24d7fa640 1 -- 172.21.15.42:0/376992506 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd25c0a1df0 con 0x558e24a957b0 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.121+0000 7fd24d7fa640 1 -- 172.21.15.42:0/376992506 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c011820 con 0x558e24a957b0 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.121+0000 7fd270402740 1 -- 172.21.15.42:0/376992506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 msgr2=0x558e24a97ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.121+0000 7fd2677fe640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2500a1c20 0x7fd2500a40e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd260037040 tx=0x7fd260009800 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.121+0000 7fd270402740 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24a97ba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0d2a80 tx=0x7fd25c0dd880 comp rx=0 tx=0).stop 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 -- 172.21.15.42:0/376992506 shutdown_connections 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24a97ba0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24a95270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 --2- 172.21.15.42:0/376992506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a8c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 -- 172.21.15.42:0/376992506 >> 172.21.15.42:0/376992506 conn(0x558e24a86990 msgr2=0x558e24a86d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 -- 172.21.15.42:0/376992506 shutdown_connections 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 -- 172.21.15.42:0/376992506 wait complete. 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.122+0000 7fd270402740 1 -- start start 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 1 Processor -- start 2024-03-12T04:17:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8c2b0 0x558e24a92650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a92b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a957b0 0x558e24a930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e246c7430 con 0x558e24a957b0 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24683e00 con 0x558e24a8cbd0 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24978fd0 con 0x558e24a8c2b0 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd270402740 10 client.0 _ll_register_callbacks cb 0x558e249cefd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a957b0 0x558e24a930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a957b0 0x558e24a930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41602/0 (socket says 172.21.15.42:41602) 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.123+0000 7fd2677fe640 1 -- 172.21.15.42:0/441603551 learned_addr learned my addr 172.21.15.42:0/441603551 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2667fc640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a92b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd266ffd640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8c2b0 0x558e24a92650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2677fe640 1 -- 172.21.15.42:0/441603551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8c2b0 msgr2=0x558e24a92650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2677fe640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8c2b0 0x558e24a92650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2677fe640 1 -- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 msgr2=0x558e24a92b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2677fe640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a92b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2677fe640 1 -- 172.21.15.42:0/441603551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2681a23d0 con 0x558e24a957b0 2024-03-12T04:17:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2667fc640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a92b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd2677fe640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a957b0 0x558e24a930d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd260036000 tx=0x7fd2600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41df4d114efd4414 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.124+0000 7fd24aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.125+0000 7fd24aff5640 1 -- 172.21.15.42:0/441603551 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260045040 con 0x558e24a957b0 2024-03-12T04:17:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.125+0000 7fd24aff5640 1 -- 172.21.15.42:0/441603551 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd260003b70 con 0x558e24a957b0 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.125+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e246f37e0 con 0x558e24a957b0 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.125+0000 7fd24aff5640 1 -- 172.21.15.42:0/441603551 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd260032460 con 0x558e24a957b0 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.125+0000 7fd270402740 10 client.16047 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.125+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e249f2610 con 0x558e24a957b0 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.125+0000 7fd24aff5640 1 -- 172.21.15.42:0/441603551 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd260003d30 con 0x558e24a957b0 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.126+0000 7fd24aff5640 1 -- 172.21.15.42:0/441603551 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd26004b600 con 0x558e24a957b0 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.126+0000 7fd270402740 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.126+0000 7fd270402740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.126+0000 7fd270402740 20 client.16047 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.126+0000 7fd270402740 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24aa59a0 0x558e24aa7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.126+0000 7fd270402740 4 client.16047 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.126+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e246f37e0 con 0x558e24aa59a0 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.127+0000 7fd266ffd640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24aa59a0 0x558e24aa7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd266ffd640 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24aa59a0 0x558e24aa7db0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd8224c6e08ca95 server_cookie=77583c412658b219 in_seq=0 out_seq=0 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd24aff5640 10 client.16047 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd24aff5640 1 -- 172.21.15.42:0/441603551 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e246f37e0 con 0x558e24aa59a0 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd24aff5640 10 client.16047 handle_command_reply: tid=0 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd270402740 1 client.16047 shutdown 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a957b0 msgr2=0x558e24a930d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd270402740 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a957b0 0x558e24a930d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd260036000 tx=0x7fd2600095c0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 shutdown_connections 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.128+0000 7fd270402740 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24aa59a0 0x558e24aa7db0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd270402740 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a957b0 0x558e24a930d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd270402740 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a92b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd270402740 1 --2- 172.21.15.42:0/441603551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8c2b0 0x558e24a92650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 >> 172.21.15.42:0/441603551 conn(0x558e24a86990 msgr2=0x558e24a88730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd24aff5640 0 client.16047 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 shutdown_connections 2024-03-12T04:17:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd270402740 1 -- 172.21.15.42:0/441603551 wait complete. 2024-03-12T04:17:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.129+0000 7fd270402740 20 client.16047 trim_cache size 0 max 16384 2024-03-12T04:17:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.156+0000 7fd270402740 1 Processor -- start 2024-03-12T04:17:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.156+0000 7fd270402740 1 -- start start 2024-03-12T04:17:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd270402740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a94d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd270402740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a9c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd270402740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a957b0 0x558e24a9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd270402740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24a87c10 con 0x558e24a8c2b0 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd270402740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24a8b630 con 0x558e24a8cbd0 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd270402740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24a87790 con 0x558e24a957b0 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a9c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a9c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56124/0 (socket says 172.21.15.42:56124) 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd2667fc640 1 -- 172.21.15.42:0/314796483 learned_addr learned my addr 172.21.15.42:0/314796483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.157+0000 7fd2677fe640 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a957b0 0x558e24a9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.158+0000 7fd2677fe640 1 -- 172.21.15.42:0/314796483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 msgr2=0x558e24a9c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.158+0000 7fd2677fe640 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a9c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.158+0000 7fd2677fe640 1 -- 172.21.15.42:0/314796483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 msgr2=0x558e24a94d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.158+0000 7fd2677fe640 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a94d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.158+0000 7fd2677fe640 1 -- 172.21.15.42:0/314796483 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e2498eac0 con 0x558e24a957b0 2024-03-12T04:17:12.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.158+0000 7fd2677fe640 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a957b0 0x558e24a9cbf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd260005e30 tx=0x7fd260009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4c4c29370b8b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.160+0000 7fd24cff9640 1 -- 172.21.15.42:0/314796483 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260032460 con 0x558e24a957b0 2024-03-12T04:17:12.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.161+0000 7fd24cff9640 1 -- 172.21.15.42:0/314796483 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd260032a70 con 0x558e24a957b0 2024-03-12T04:17:12.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.161+0000 7fd24cff9640 1 -- 172.21.15.42:0/314796483 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2600042a0 con 0x558e24a957b0 2024-03-12T04:17:12.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.162+0000 7fd270402740 1 -- 172.21.15.42:0/314796483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a957b0 msgr2=0x558e24a9cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.162+0000 7fd270402740 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a957b0 0x558e24a9cbf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd260005e30 tx=0x7fd260009b20 comp rx=0 tx=0).stop 2024-03-12T04:17:12.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.162+0000 7fd270402740 1 -- 172.21.15.42:0/314796483 shutdown_connections 2024-03-12T04:17:12.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.162+0000 7fd270402740 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a957b0 0x558e24a9cbf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.162+0000 7fd270402740 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a8cbd0 0x558e24a9c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.162+0000 7fd270402740 1 --2- 172.21.15.42:0/314796483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24a94d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.162+0000 7fd270402740 1 -- 172.21.15.42:0/314796483 >> 172.21.15.42:0/314796483 conn(0x558e24a86990 msgr2=0x558e24a8d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.163+0000 7fd270402740 1 -- 172.21.15.42:0/314796483 shutdown_connections 2024-03-12T04:17:12.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.163+0000 7fd270402740 1 -- 172.21.15.42:0/314796483 wait complete. 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.163+0000 7fd270402740 1 -- start start 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.163+0000 7fd270402740 1 Processor -- start 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd270402740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24af4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd270402740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24af47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd270402740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24aefe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd270402740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24af0560 con 0x558e24a8c2b0 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd270402740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24af06d0 con 0x558e24a957b0 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd270402740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e24af0890 con 0x558e24a8cbd0 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd270402740 10 client.0 _ll_register_callbacks cb 0x7fd25c0d8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd266ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24af4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24af47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd2667fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24af47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59382/0 (socket says 172.21.15.42:59382) 2024-03-12T04:17:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd2667fc640 1 -- 172.21.15.42:0/3004528693 learned_addr learned my addr 172.21.15.42:0/3004528693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.164+0000 7fd266ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24af4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41624/0 (socket says 172.21.15.42:41624) 2024-03-12T04:17:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd2667fc640 1 -- 172.21.15.42:0/3004528693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 msgr2=0x558e24aefe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd2667fc640 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24aefe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd2667fc640 1 -- 172.21.15.42:0/3004528693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 msgr2=0x558e24af4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd2667fc640 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24af4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd2667fc640 1 -- 172.21.15.42:0/3004528693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e249f2610 con 0x558e24a8cbd0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd2667fc640 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24af47d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd25c011c70 tx=0x7fd25c011c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b30daabc405494d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd24aff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd24aff5640 1 -- 172.21.15.42:0/3004528693 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0a2480 con 0x558e24a8cbd0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd24aff5640 1 -- 172.21.15.42:0/3004528693 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd25c00fbe0 con 0x558e24a8cbd0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.165+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd25c0a1df0 con 0x558e24a8cbd0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd24aff5640 1 -- 172.21.15.42:0/3004528693 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd25c09e2c0 con 0x558e24a8cbd0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 10 client.25444 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e249b46a0 con 0x558e24a8cbd0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd24aff5640 1 -- 172.21.15.42:0/3004528693 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd25c0a1df0 con 0x558e24a8cbd0 2024-03-12T04:17:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd24aff5640 1 -- 172.21.15.42:0/3004528693 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd25c0aa330 con 0x558e24a8cbd0 2024-03-12T04:17:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 10 client.25444 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 10 client.25444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 20 client.25444 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24afcee0 0x558e24aff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 4 client.25444 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:17:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.166+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e24730720 con 0x558e24afcee0 2024-03-12T04:17:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.167+0000 7fd266ffd640 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24afcee0 0x558e24aff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.169+0000 7fd266ffd640 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24afcee0 0x558e24aff300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7869588236c020fb server_cookie=21123425dff2e88f in_seq=0 out_seq=0 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.169+0000 7fd24aff5640 10 client.25444 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.169+0000 7fd24aff5640 1 -- 172.21.15.42:0/3004528693 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558e24730720 con 0x558e24afcee0 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.169+0000 7fd24aff5640 10 client.25444 handle_command_reply: tid=0 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.169+0000 7fd270402740 1 client.25444 shutdown 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.169+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 msgr2=0x558e24af47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.169+0000 7fd270402740 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24af47d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd25c011c70 tx=0x7fd25c011c00 comp rx=0 tx=0).stop 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.170+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 shutdown_connections 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.170+0000 7fd270402740 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558e24afcee0 0x558e24aff300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.170+0000 7fd270402740 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e24a957b0 0x558e24aefe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.170+0000 7fd270402740 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e24a8cbd0 0x558e24af47d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.170+0000 7fd270402740 1 --2- 172.21.15.42:0/3004528693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e24a8c2b0 0x558e24af4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.170+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 >> 172.21.15.42:0/3004528693 conn(0x558e24a86990 msgr2=0x558e24a87e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.171+0000 7fd24aff5640 0 client.25444 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:12.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.171+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 shutdown_connections 2024-03-12T04:17:12.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.171+0000 7fd270402740 1 -- 172.21.15.42:0/3004528693 wait complete. 2024-03-12T04:17:12.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.171+0000 7fd270402740 20 client.25444 trim_cache size 0 max 16384 2024-03-12T04:17:12.175 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:17:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:17:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:17:12.175 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.172+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2500a1c20 msgr2=0x7fd2500a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.172+0000 7fd26d74f640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2500a1c20 0x7fd2500a40e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd260037040 tx=0x7fd260009800 comp rx=0 tx=0).stop 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.172+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268106430 msgr2=0x7fd268195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.172+0000 7fd26d74f640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268106430 0x7fd268195a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd25c014370 tx=0x7fd25c014830 comp rx=0 tx=0).stop 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.172+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 shutdown_connections 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.173+0000 7fd26d74f640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2500a1c20 0x7fd2500a40e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.173+0000 7fd26d74f640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd268199e00 0x7fd26819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.173+0000 7fd26d74f640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268106430 0x7fd268195a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.173+0000 7fd26d74f640 1 --2- 172.21.15.42:0/1139023812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd268101360 0x7fd268195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.173+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 >> 172.21.15.42:0/1139023812 conn(0x7fd2680fcc20 msgr2=0x7fd2680fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.173+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 shutdown_connections 2024-03-12T04:17:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.174+0000 7fd26d74f640 1 -- 172.21.15.42:0/1139023812 wait complete. 2024-03-12T04:17:12.194 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:17:12.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:17:12.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 -- 172.21.15.42:0/3832424124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0264102d10 msgr2=0x7f026410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 --2- 172.21.15.42:0/3832424124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0264102d10 0x7f026410b1f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0250009900 tx=0x7f025002bc50 comp rx=0 tx=0).stop 2024-03-12T04:17:12.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 -- 172.21.15.42:0/3832424124 shutdown_connections 2024-03-12T04:17:12.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 --2- 172.21.15.42:0/3832424124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026410b7c0 0x7f026410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 --2- 172.21.15.42:0/3832424124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0264102d10 0x7f026410b1f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 --2- 172.21.15.42:0/3832424124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 0x7f02641027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 -- 172.21.15.42:0/3832424124 >> 172.21.15.42:0/3832424124 conn(0x7f0264077180 msgr2=0x7f0264077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 -- 172.21.15.42:0/3832424124 shutdown_connections 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.352+0000 7f026b264640 1 -- 172.21.15.42:0/3832424124 wait complete. 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.353+0000 7f026b264640 1 Processor -- start 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f026b264640 1 -- start start 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f026b264640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 0x7f02641141a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f026b264640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0264102d10 0x7f0264110320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f026b264640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f026410b7c0 0x7f0264110a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f026b264640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0264110f80 con 0x7f026410b7c0 2024-03-12T04:17:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f026b264640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02641110f0 con 0x7f0264102d10 2024-03-12T04:17:12.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f026b264640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02641b2c60 con 0x7f02641023f0 2024-03-12T04:17:12.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f0269000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 0x7f02641141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f0269000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 0x7f02641141a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59386/0 (socket says 172.21.15.42:59386) 2024-03-12T04:17:12.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.354+0000 7f0269000640 1 -- 172.21.15.42:0/3632209679 learned_addr learned my addr 172.21.15.42:0/3632209679 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0263fff640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0264102d10 0x7f0264110320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0269801640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f026410b7c0 0x7f0264110a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0269000640 1 -- 172.21.15.42:0/3632209679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0264102d10 msgr2=0x7f0264110320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0269000640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0264102d10 0x7f0264110320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0269000640 1 -- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f026410b7c0 msgr2=0x7f0264110a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0269000640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f026410b7c0 0x7f0264110a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0269000640 1 -- 172.21.15.42:0/3632209679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250009560 con 0x7f02641023f0 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0269801640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f026410b7c0 0x7f0264110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.355+0000 7f0263fff640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0264102d10 0x7f0264110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.356+0000 7f0269000640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 0x7f02641141a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f025800b450 tx=0x7f025800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.356+0000 7f0261ffb640 1 -- 172.21.15.42:0/3632209679 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258011890 con 0x7f02641023f0 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.356+0000 7f0261ffb640 1 -- 172.21.15.42:0/3632209679 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02580103e0 con 0x7f02641023f0 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.356+0000 7f0261ffb640 1 -- 172.21.15.42:0/3632209679 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258011ea0 con 0x7f02641023f0 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.356+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02641b2da0 con 0x7f02641023f0 2024-03-12T04:17:12.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.356+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02641b31d0 con 0x7f02641023f0 2024-03-12T04:17:12.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.358+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0264068b20 con 0x7f02641023f0 2024-03-12T04:17:12.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.359+0000 7f0261ffb640 1 -- 172.21.15.42:0/3632209679 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0258010550 con 0x7f02641023f0 2024-03-12T04:17:12.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.360+0000 7f0261ffb640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02540a1c20 0x7f02540a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.361+0000 7f0263fff640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02540a1c20 0x7f02540a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.361+0000 7f0261ffb640 1 -- 172.21.15.42:0/3632209679 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f02580cd480 con 0x7f02641023f0 2024-03-12T04:17:12.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.361+0000 7f0263fff640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02540a1c20 0x7f02540a40e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0250006010 tx=0x7f0250005e10 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.368+0000 7f0261ffb640 1 -- 172.21.15.42:0/3632209679 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0258096450 con 0x7f02641023f0 2024-03-12T04:17:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.762+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0264067f70 con 0x7f02641023f0 2024-03-12T04:17:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.763+0000 7f0261ffb640 1 -- 172.21.15.42:0/3632209679 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f0258099b10 con 0x7f02641023f0 2024-03-12T04:17:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:17:12.767 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:17:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:17:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02540a1c20 msgr2=0x7f02540a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02540a1c20 0x7f02540a40e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0250006010 tx=0x7f0250005e10 comp rx=0 tx=0).stop 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 msgr2=0x7f02641141a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 0x7f02641141a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f025800b450 tx=0x7f025800b910 comp rx=0 tx=0).stop 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 shutdown_connections 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02540a1c20 0x7f02540a40e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f026410b7c0 0x7f0264110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0264102d10 0x7f0264110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 --2- 172.21.15.42:0/3632209679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02641023f0 0x7f02641141a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.766+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 >> 172.21.15.42:0/3632209679 conn(0x7f0264077180 msgr2=0x7f0264103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.767+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 shutdown_connections 2024-03-12T04:17:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.767+0000 7f026b264640 1 -- 172.21.15.42:0/3632209679 wait complete. 2024-03-12T04:17:12.786 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:12.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:17:12.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.951+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/397694331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810f6a0 msgr2=0x7f0a8810fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.951+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/397694331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810f6a0 0x7f0a8810fa80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009cd0 tx=0x7f0a8002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.951+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/397694331 shutdown_connections 2024-03-12T04:17:12.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.951+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/397694331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a8810dce0 0x7f0a8811c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.951+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/397694331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810d260 0x7f0a8810d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.951+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/397694331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810f6a0 0x7f0a8810fa80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.951+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/397694331 >> 172.21.15.42:0/397694331 conn(0x7f0a8806c030 msgr2=0x7f0a8806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.952+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/397694331 shutdown_connections 2024-03-12T04:17:12.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.952+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/397694331 wait complete. 2024-03-12T04:17:12.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 Processor -- start 2024-03-12T04:17:12.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 -- start start 2024-03-12T04:17:12.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810d260 0x7f0a881a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a8810dce0 0x7f0a881a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810f6a0 0x7f0a881aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881ab160 con 0x7f0a8810f6a0 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881ab2d0 con 0x7f0a8810dce0 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8e2a0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881ab490 con 0x7f0a8810d260 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8da9f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810f6a0 0x7f0a881aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8da9f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810f6a0 0x7f0a881aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41656/0 (socket says 172.21.15.42:41656) 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8da9f640 1 -- 172.21.15.42:0/665602454 learned_addr learned my addr 172.21.15.42:0/665602454 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810d260 0x7f0a881a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a8810dce0 0x7f0a881a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8da9f640 1 -- 172.21.15.42:0/665602454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810d260 msgr2=0x7f0a881a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8da9f640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810d260 0x7f0a881a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8da9f640 1 -- 172.21.15.42:0/665602454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a8810dce0 msgr2=0x7f0a881a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.953+0000 7f0a8da9f640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a8810dce0 0x7f0a881a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.954+0000 7f0a8da9f640 1 -- 172.21.15.42:0/665602454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a78009490 con 0x7f0a8810f6a0 2024-03-12T04:17:12.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.954+0000 7f0a8da9f640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810f6a0 0x7f0a881aab60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400e970 tx=0x7f0a8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.954+0000 7f0a7e7fc640 1 -- 172.21.15.42:0/665602454 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84009720 con 0x7f0a8810f6a0 2024-03-12T04:17:12.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.954+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a80009900 con 0x7f0a8810f6a0 2024-03-12T04:17:12.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.954+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a881b2e10 con 0x7f0a8810f6a0 2024-03-12T04:17:12.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.954+0000 7f0a7e7fc640 1 -- 172.21.15.42:0/665602454 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a840054d0 con 0x7f0a8810f6a0 2024-03-12T04:17:12.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.954+0000 7f0a7e7fc640 1 -- 172.21.15.42:0/665602454 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84010430 con 0x7f0a8810f6a0 2024-03-12T04:17:12.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.958+0000 7f0a921b1740 1 Processor -- start 2024-03-12T04:17:12.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.958+0000 7f0a7e7fc640 1 -- 172.21.15.42:0/665602454 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0a84010590 con 0x7f0a8810f6a0 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a921b1740 1 -- start start 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0c2460 0x5653bb0c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0cb960 0x5653bb0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a921b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb07d960 con 0x5653bb0c2460 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a921b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb031310 con 0x5653bb0c2d80 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a921b1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb02e840 con 0x5653bb0cb960 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a8ca9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0c2460 0x5653bb0c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a8ca9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0c2460 0x5653bb0c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41670/0 (socket says 172.21.15.42:41670) 2024-03-12T04:17:12.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a8ca9d640 1 -- 172.21.15.42:0/546589229 learned_addr learned my addr 172.21.15.42:0/546589229 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.959+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0cb960 0x5653bb0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a8d29e640 1 -- 172.21.15.42:0/546589229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0cb960 msgr2=0x5653bb0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0cb960 0x5653bb0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a8d29e640 1 -- 172.21.15.42:0/546589229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0c2460 msgr2=0x5653bb0c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0c2460 0x5653bb0c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a8d29e640 1 -- 172.21.15.42:0/546589229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653bafec060 con 0x5653bb0c2d80 2024-03-12T04:17:12.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a7e7fc640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0a740a1c20 0x7f0a740a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0cb420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a8002c700 tx=0x7f0a80005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af6c24590be15e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a7e7fc640 1 -- 172.21.15.42:0/665602454 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a84014070 con 0x7f0a8810f6a0 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.960+0000 7f0a737fe640 1 -- 172.21.15.42:0/546589229 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032c50 con 0x5653bb0c2d80 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.961+0000 7f0a737fe640 1 -- 172.21.15.42:0/546589229 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a80032db0 con 0x5653bb0c2d80 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.961+0000 7f0a737fe640 1 -- 172.21.15.42:0/546589229 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032a00 con 0x5653bb0c2d80 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.961+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0a740a1c20 0x7f0a740a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.961+0000 7f0a921b1740 1 -- 172.21.15.42:0/546589229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 msgr2=0x5653bb0cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.961+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0a740a1c20 0x7f0a740a40e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5653bb0bdd20 tx=0x7f0a78015040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.961+0000 7f0a921b1740 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0cb420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a8002c700 tx=0x7f0a80005f70 comp rx=0 tx=0).stop 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 -- 172.21.15.42:0/546589229 shutdown_connections 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0cb960 0x5653bb0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0cb420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 --2- 172.21.15.42:0/546589229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0c2460 0x5653bb0c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 -- 172.21.15.42:0/546589229 >> 172.21.15.42:0/546589229 conn(0x5653bb0bcb90 msgr2=0x5653bb0bcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 -- 172.21.15.42:0/546589229 shutdown_connections 2024-03-12T04:17:12.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 -- 172.21.15.42:0/546589229 wait complete. 2024-03-12T04:17:12.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 -- start start 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.962+0000 7f0a921b1740 1 Processor -- start 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.963+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.963+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.963+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a921b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb02a020 con 0x5653bb0cb960 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a921b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bacfd430 con 0x5653bb0c2460 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a921b1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bacb9e00 con 0x5653bb0c2d80 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a921b1740 10 client.0 _ll_register_callbacks cb 0x5653bafd1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56168/0 (socket says 172.21.15.42:56168) 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 -- 172.21.15.42:0/1570399990 learned_addr learned my addr 172.21.15.42:0/1570399990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8da9f640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 -- 172.21.15.42:0/1570399990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 msgr2=0x5653bb0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 -- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 msgr2=0x5653bb0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8d29e640 1 -- 172.21.15.42:0/1570399990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8811d450 con 0x5653bb0c2460 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.964+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0c8920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000f80 tx=0x7f0a80034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc2cecf6a27bc80 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a70ff9640 1 -- 172.21.15.42:0/1570399990 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032c50 con 0x5653bb0c2460 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a70ff9640 1 -- 172.21.15.42:0/1570399990 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a80034c70 con 0x5653bb0c2460 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653bad297e0 con 0x5653bb0c2460 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a70ff9640 1 -- 172.21.15.42:0/1570399990 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a80034e30 con 0x5653bb0c2460 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a921b1740 10 client.25568 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:12.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5653bb0286a0 con 0x5653bb0c2460 2024-03-12T04:17:12.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.965+0000 7f0a70ff9640 1 -- 172.21.15.42:0/1570399990 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a80003e20 con 0x5653bb0c2460 2024-03-12T04:17:12.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a70ff9640 1 -- 172.21.15.42:0/1570399990 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a80004570 con 0x5653bb0c2460 2024-03-12T04:17:12.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a921b1740 10 client.25568 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:12.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a921b1740 10 client.25568 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a921b1740 20 client.25568 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a921b1740 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb0db9e0 0x5653bb0dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a921b1740 4 client.25568 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653bad297e0 con 0x5653bb0db9e0 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb0db9e0 0x5653bb0dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a70ff9640 10 client.25568 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.966+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb0db9e0 0x5653bb0dde60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6e80bec3d3092bc server_cookie=4bd3d8fb5346bf94 in_seq=0 out_seq=0 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.967+0000 7f0a70ff9640 1 -- 172.21.15.42:0/1570399990 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653bad297e0 con 0x5653bb0db9e0 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.967+0000 7f0a70ff9640 10 client.25568 handle_command_reply: tid=0 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.967+0000 7f0a921b1740 1 client.25568 shutdown 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.967+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 msgr2=0x5653bb0c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.967+0000 7f0a921b1740 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0c8920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000f80 tx=0x7f0a80034670 comp rx=0 tx=0).stop 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 shutdown_connections 2024-03-12T04:17:12.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb0db9e0 0x5653bb0dde60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 --2- 172.21.15.42:0/1570399990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0c8920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a70ff9640 0 client.25568 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 >> 172.21.15.42:0/1570399990 conn(0x5653bb0bcb90 msgr2=0x5653bb0be8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 shutdown_connections 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 1 -- 172.21.15.42:0/1570399990 wait complete. 2024-03-12T04:17:12.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.968+0000 7f0a921b1740 20 client.25568 trim_cache size 0 max 16384 2024-03-12T04:17:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 Processor -- start 2024-03-12T04:17:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 -- start start 2024-03-12T04:17:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bafaf130 con 0x5653bb0cb960 2024-03-12T04:17:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb0cdd50 con 0x5653bb0c2460 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a921b1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb0d7720 con 0x5653bb0c2d80 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a8da9f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a8d29e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a8da9f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41700/0 (socket says 172.21.15.42:41700) 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a8d29e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56170/0 (socket says 172.21.15.42:56170) 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.987+0000 7f0a8da9f640 1 -- 172.21.15.42:0/3475872066 learned_addr learned my addr 172.21.15.42:0/3475872066 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.988+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.988+0000 7f0a8d29e640 1 -- 172.21.15.42:0/3475872066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 msgr2=0x5653bb0c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.988+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.988+0000 7f0a8d29e640 1 -- 172.21.15.42:0/3475872066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 msgr2=0x5653bb0c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.988+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.988+0000 7f0a8d29e640 1 -- 172.21.15.42:0/3475872066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653bafc4c20 con 0x5653bb0c2460 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.988+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0cb050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a80004900 tx=0x7f0a80009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6051248760f816 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.989+0000 7f0a72ffd640 1 -- 172.21.15.42:0/3475872066 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032c50 con 0x5653bb0c2460 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.989+0000 7f0a72ffd640 1 -- 172.21.15.42:0/3475872066 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a800325d0 con 0x5653bb0c2460 2024-03-12T04:17:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.989+0000 7f0a72ffd640 1 -- 172.21.15.42:0/3475872066 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032440 con 0x5653bb0c2460 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.990+0000 7f0a921b1740 1 -- 172.21.15.42:0/3475872066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 msgr2=0x5653bb0cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.990+0000 7f0a921b1740 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0cb050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a80004900 tx=0x7f0a80009540 comp rx=0 tx=0).stop 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.990+0000 7f0a921b1740 1 -- 172.21.15.42:0/3475872066 shutdown_connections 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.990+0000 7f0a921b1740 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.990+0000 7f0a921b1740 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2d80 0x5653bb0c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.990+0000 7f0a921b1740 1 --2- 172.21.15.42:0/3475872066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2460 0x5653bb0cb050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.990+0000 7f0a921b1740 1 -- 172.21.15.42:0/3475872066 >> 172.21.15.42:0/3475872066 conn(0x5653bb0bcb90 msgr2=0x5653bb0bcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.991+0000 7f0a921b1740 1 -- 172.21.15.42:0/3475872066 shutdown_connections 2024-03-12T04:17:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.991+0000 7f0a921b1740 1 -- 172.21.15.42:0/3475872066 wait complete. 2024-03-12T04:17:12.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.991+0000 7f0a921b1740 1 -- start start 2024-03-12T04:17:12.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.992+0000 7f0a921b1740 1 Processor -- start 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.992+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2460 0x5653bb0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.992+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.992+0000 7f0a921b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a921b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb131f50 con 0x5653bb0cb960 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a921b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb132090 con 0x5653bb0c2d80 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a921b1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bb132250 con 0x5653bb0c2460 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2460 0x5653bb0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a921b1740 10 client.0 _ll_register_callbacks cb 0x7f0a78005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2460 0x5653bb0d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59446/0 (socket says 172.21.15.42:59446) 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 -- 172.21.15.42:0/2203985397 learned_addr learned my addr 172.21.15.42:0/2203985397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8da9f640 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:12.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 -- 172.21.15.42:0/2203985397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 msgr2=0x5653bb0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 -- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 msgr2=0x5653bb0d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:12.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:12.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.993+0000 7f0a8d29e640 1 -- 172.21.15.42:0/2203985397 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653bb0286a0 con 0x5653bb0c2460 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.994+0000 7f0a8d29e640 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2460 0x5653bb0d1380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a800049d0 tx=0x7f0a80034880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aff1a4af833b0ad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.994+0000 7f0a70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.994+0000 7f0a70ff9640 1 -- 172.21.15.42:0/2203985397 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032a50 con 0x5653bb0c2460 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.994+0000 7f0a70ff9640 1 -- 172.21.15.42:0/2203985397 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a8003f650 con 0x5653bb0c2460 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.994+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a80032db0 con 0x5653bb0c2460 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.994+0000 7f0a70ff9640 1 -- 172.21.15.42:0/2203985397 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a80032c50 con 0x5653bb0c2460 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.995+0000 7f0a921b1740 10 client.25462 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.995+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5653bafea8c0 con 0x5653bb0c2460 2024-03-12T04:17:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.995+0000 7f0a70ff9640 1 -- 172.21.15.42:0/2203985397 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a80009b20 con 0x5653bb0c2460 2024-03-12T04:17:12.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a70ff9640 1 -- 172.21.15.42:0/2203985397 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a80067020 con 0x5653bb0c2460 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a921b1740 10 client.25462 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a921b1740 10 client.25462 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a921b1740 20 client.25462 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a921b1740 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb133520 0x5653bb135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a921b1740 4 client.25462 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0a80032db0 con 0x5653bb133520 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.996+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb133520 0x5653bb135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a8ca9d640 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb133520 0x5653bb135910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7fe527b6bb91eb55 server_cookie=c2df72df2c62a9d7 in_seq=0 out_seq=0 2024-03-12T04:17:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a70ff9640 10 client.25462 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a70ff9640 1 -- 172.21.15.42:0/2203985397 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0a80032db0 con 0x5653bb133520 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a70ff9640 10 client.25462 handle_command_reply: tid=0 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 client.25462 shutdown 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2460 msgr2=0x5653bb0d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2460 0x5653bb0d1380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a800049d0 tx=0x7f0a80034880 comp rx=0 tx=0).stop 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 shutdown_connections 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5653bb133520 0x5653bb135910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653bb0cb960 0x5653bb0d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653bb0c2d80 0x5653bb0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.997+0000 7f0a921b1740 1 --2- 172.21.15.42:0/2203985397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653bb0c2460 0x5653bb0d1380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.998+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 >> 172.21.15.42:0/2203985397 conn(0x5653bb0bcb90 msgr2=0x5653bb0c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.998+0000 7f0a70ff9640 0 client.25462 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.998+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 shutdown_connections 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.998+0000 7f0a921b1740 1 -- 172.21.15.42:0/2203985397 wait complete. 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.998+0000 7f0a921b1740 20 client.25462 trim_cache size 0 max 16384 2024-03-12T04:17:13.002 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0a740a1c20 msgr2=0x7f0a740a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0a740a1c20 0x7f0a740a40e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5653bb0bdd20 tx=0x7f0a78015040 comp rx=0 tx=0).stop 2024-03-12T04:17:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810f6a0 msgr2=0x7f0a881aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810f6a0 0x7f0a881aab60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400e970 tx=0x7f0a8400ed70 comp rx=0 tx=0).stop 2024-03-12T04:17:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 shutdown_connections 2024-03-12T04:17:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0a740a1c20 0x7f0a740a40e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a8810f6a0 0x7f0a881aab60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a8810dce0 0x7f0a881a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 --2- 172.21.15.42:0/665602454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a8810d260 0x7f0a881a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 >> 172.21.15.42:0/665602454 conn(0x7f0a8806c030 msgr2=0x7f0a88070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 shutdown_connections 2024-03-12T04:17:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:12.999+0000 7f0a8e2a0640 1 -- 172.21.15.42:0/665602454 wait complete. 2024-03-12T04:17:13.018 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:13.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:17:13.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.180+0000 7f431a977640 1 -- 172.21.15.42:0/1892373814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141142d0 msgr2=0x7f43141166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.180+0000 7f431a977640 1 --2- 172.21.15.42:0/1892373814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141142d0 0x7f43141166c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f430c009ca0 tx=0x7f430c02c100 comp rx=0 tx=0).stop 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.181+0000 7f431a977640 1 -- 172.21.15.42:0/1892373814 shutdown_connections 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.181+0000 7f431a977640 1 --2- 172.21.15.42:0/1892373814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141142d0 0x7f43141166c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.181+0000 7f431a977640 1 --2- 172.21.15.42:0/1892373814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4314108c80 0x7f4314113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.181+0000 7f431a977640 1 --2- 172.21.15.42:0/1892373814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4314108360 0x7f4314108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.181+0000 7f431a977640 1 -- 172.21.15.42:0/1892373814 >> 172.21.15.42:0/1892373814 conn(0x7f431406c260 msgr2=0x7f431406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.181+0000 7f431a977640 1 -- 172.21.15.42:0/1892373814 shutdown_connections 2024-03-12T04:17:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.181+0000 7f431a977640 1 -- 172.21.15.42:0/1892373814 wait complete. 2024-03-12T04:17:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 Processor -- start 2024-03-12T04:17:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 -- start start 2024-03-12T04:17:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314108360 0x7f43141a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4314108c80 0x7f43141a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43141142d0 0x7f43141a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141a6f20 con 0x7f43141142d0 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141a7090 con 0x7f4314108360 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f431a977640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141a7250 con 0x7f4314108c80 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f4313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314108360 0x7f43141a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f4318f14640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43141142d0 0x7f43141a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f4313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314108360 0x7f43141a2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56204/0 (socket says 172.21.15.42:56204) 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f4318f14640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43141142d0 0x7f43141a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41722/0 (socket says 172.21.15.42:41722) 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.182+0000 7f4313fff640 1 -- 172.21.15.42:0/4047707451 learned_addr learned my addr 172.21.15.42:0/4047707451 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.183+0000 7f4318f14640 1 -- 172.21.15.42:0/4047707451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4314108c80 msgr2=0x7f43141a25a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.183+0000 7f4318f14640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4314108c80 0x7f43141a25a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.183+0000 7f4318f14640 1 -- 172.21.15.42:0/4047707451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314108360 msgr2=0x7f43141a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.183+0000 7f4318f14640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314108360 0x7f43141a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.183+0000 7f4318f14640 1 -- 172.21.15.42:0/4047707451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4308009490 con 0x7f43141142d0 2024-03-12T04:17:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.183+0000 7f4318f14640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43141142d0 0x7f43141a6920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f430c037040 tx=0x7f430c034460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.184+0000 7f43117fa640 1 -- 172.21.15.42:0/4047707451 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0325f0 con 0x7f43141142d0 2024-03-12T04:17:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.184+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c009900 con 0x7f43141142d0 2024-03-12T04:17:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.184+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43141aebd0 con 0x7f43141142d0 2024-03-12T04:17:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.184+0000 7f43117fa640 1 -- 172.21.15.42:0/4047707451 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f430c003970 con 0x7f43141142d0 2024-03-12T04:17:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.184+0000 7f43117fa640 1 -- 172.21.15.42:0/4047707451 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0045a0 con 0x7f43141142d0 2024-03-12T04:17:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.186+0000 7f43117fa640 1 -- 172.21.15.42:0/4047707451 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f430c03b040 con 0x7f43141142d0 2024-03-12T04:17:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.187+0000 7f431d628740 1 Processor -- start 2024-03-12T04:17:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.187+0000 7f43117fa640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42fc0a1c60 0x7f42fc0a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.187+0000 7f431d628740 1 -- start start 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f43137fe640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42fc0a1c60 0x7f42fc0a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f431d628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cd5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f431d628740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f431d628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cdef60 0x55f612ce1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f431d628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612c88960 con 0x55f612cd5a60 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f431d628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612c3c310 con 0x55f612cdef60 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f431d628740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612c39840 con 0x55f612cd6380 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f43117fa640 1 -- 172.21.15.42:0/4047707451 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f430c0efed0 con 0x7f43141142d0 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cdef60 0x55f612ce1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cdef60 0x55f612ce1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56216/0 (socket says 172.21.15.42:56216) 2024-03-12T04:17:13.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 -- 172.21.15.42:0/499722512 learned_addr learned my addr 172.21.15.42:0/499722512 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f43137fe640 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 -- 172.21.15.42:0/499722512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 msgr2=0x55f612cdea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 -- 172.21.15.42:0/499722512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 msgr2=0x55f612cd5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cd5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.188+0000 7f4318f14640 1 -- 172.21.15.42:0/499722512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f612bf7060 con 0x55f612cdef60 2024-03-12T04:17:13.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.189+0000 7f4318f14640 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cdef60 0x55f612ce1350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c0bd810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc027ac41d8ddef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.189+0000 7f43137fe640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42fc0a1c60 0x7f42fc0a4120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4314116100 tx=0x7f43080128d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.189+0000 7f42fa7fc640 1 -- 172.21.15.42:0/499722512 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0bdc10 con 0x55f612cdef60 2024-03-12T04:17:13.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.189+0000 7f42fa7fc640 1 -- 172.21.15.42:0/499722512 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f430c0c1d60 con 0x55f612cdef60 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.189+0000 7f42fa7fc640 1 -- 172.21.15.42:0/499722512 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c003c30 con 0x55f612cdef60 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.190+0000 7f431d628740 1 -- 172.21.15.42:0/499722512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cdef60 msgr2=0x55f612ce1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.190+0000 7f431d628740 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cdef60 0x55f612ce1350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c0bd810 comp rx=0 tx=0).stop 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.190+0000 7f431d628740 1 -- 172.21.15.42:0/499722512 shutdown_connections 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.190+0000 7f431d628740 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cdef60 0x55f612ce1350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.190+0000 7f431d628740 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.190+0000 7f431d628740 1 --2- 172.21.15.42:0/499722512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cd5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.190+0000 7f431d628740 1 -- 172.21.15.42:0/499722512 >> 172.21.15.42:0/499722512 conn(0x55f612cd0190 msgr2=0x55f612cd0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.191+0000 7f431d628740 1 -- 172.21.15.42:0/499722512 shutdown_connections 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.191+0000 7f431d628740 1 -- 172.21.15.42:0/499722512 wait complete. 2024-03-12T04:17:13.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.191+0000 7f431d628740 1 -- start start 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.191+0000 7f431d628740 1 Processor -- start 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f431d628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612cdbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f431d628740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f431d628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612cdc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f431d628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612c35020 con 0x55f612cdef60 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f431d628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612908430 con 0x55f612cd5a60 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f431d628740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6128c4e00 con 0x55f612cd6380 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f431d628740 10 client.0 _ll_register_callbacks cb 0x55f612bdccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4318f14640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612cdc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612cdbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612cdbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56230/0 (socket says 172.21.15.42:56230) 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4313fff640 1 -- 172.21.15.42:0/568944654 learned_addr learned my addr 172.21.15.42:0/568944654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4318f14640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612cdc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41738/0 (socket says 172.21.15.42:41738) 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f43137fe640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4313fff640 1 -- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 msgr2=0x55f612cdc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4313fff640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4313fff640 1 -- 172.21.15.42:0/568944654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 msgr2=0x55f612cdc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.192+0000 7f4313fff640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612cdc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f4313fff640 1 -- 172.21.15.42:0/568944654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43141aebd0 con 0x55f612cd5a60 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f4318f14640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612cdc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f43137fe640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f4313fff640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612cdbeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4304005fa0 tx=0x7f430400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28d0d52c2973cfe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f42f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f42f37fe640 1 -- 172.21.15.42:0/568944654 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304011850 con 0x55f612cd5a60 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f42f37fe640 1 -- 172.21.15.42:0/568944654 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4304017070 con 0x55f612cd5a60 2024-03-12T04:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.193+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6129347e0 con 0x55f612cd5a60 2024-03-12T04:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.194+0000 7f42f37fe640 1 -- 172.21.15.42:0/568944654 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43040119b0 con 0x55f612cd5a60 2024-03-12T04:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.194+0000 7f431d628740 10 client.25586 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.194+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f612c336a0 con 0x55f612cd5a60 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.194+0000 7f42f37fe640 1 -- 172.21.15.42:0/568944654 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f6129347e0 con 0x55f612cd5a60 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.194+0000 7f42f37fe640 1 -- 172.21.15.42:0/568944654 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f430401bd20 con 0x55f612cd5a60 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f431d628740 10 client.25586 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f431d628740 10 client.25586 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f431d628740 20 client.25586 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f431d628740 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612cef020 0x55f612cf1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f431d628740 4 client.25586 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f612971720 con 0x55f612cef020 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f43137fe640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612cef020 0x55f612cf1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f43137fe640 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612cef020 0x55f612cf1440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f438a8179669f3e server_cookie=d87414a8f823e25e in_seq=0 out_seq=0 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.195+0000 7f42f37fe640 10 client.25586 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f42f37fe640 1 -- 172.21.15.42:0/568944654 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f612971720 con 0x55f612cef020 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f42f37fe640 10 client.25586 handle_command_reply: tid=0 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 client.25586 shutdown 2024-03-12T04:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 msgr2=0x55f612cdbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612cdbeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4304005fa0 tx=0x7f430400bf90 comp rx=0 tx=0).stop 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 shutdown_connections 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612cef020 0x55f612cf1440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612cdc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612cdc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 --2- 172.21.15.42:0/568944654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612cdbeb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 >> 172.21.15.42:0/568944654 conn(0x55f612cd0190 msgr2=0x55f612cd1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f42f37fe640 0 client.25586 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 shutdown_connections 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.196+0000 7f431d628740 1 -- 172.21.15.42:0/568944654 wait complete. 2024-03-12T04:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.197+0000 7f431d628740 20 client.25586 trim_cache size 0 max 16384 2024-03-12T04:17:13.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.217+0000 7f431d628740 1 Processor -- start 2024-03-12T04:17:13.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.217+0000 7f431d628740 1 -- start start 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f431d628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cc6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f431d628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd6380 0x55f612cdae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f431d628740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cdef60 0x55f612cdc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f4313fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cc6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f431d628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612bba130 con 0x55f612cd5a60 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f431d628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612971720 con 0x55f612cd6380 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f431d628740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612cb9a40 con 0x55f612cdef60 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f4313fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cc6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41744/0 (socket says 172.21.15.42:41744) 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f43137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd6380 0x55f612cdae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f4313fff640 1 -- 172.21.15.42:0/3159618584 learned_addr learned my addr 172.21.15.42:0/3159618584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.218+0000 7f4318f14640 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cdef60 0x55f612cdc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.219+0000 7f43137fe640 1 -- 172.21.15.42:0/3159618584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cdef60 msgr2=0x55f612cdc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.219+0000 7f43137fe640 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cdef60 0x55f612cdc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.219+0000 7f43137fe640 1 -- 172.21.15.42:0/3159618584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 msgr2=0x55f612cc6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.219+0000 7f43137fe640 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cc6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.219+0000 7f43137fe640 1 -- 172.21.15.42:0/3159618584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f612bcfc20 con 0x55f612cd6380 2024-03-12T04:17:13.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.219+0000 7f43137fe640 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd6380 0x55f612cdae90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55f612cf1d90 tx=0x7f430801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab1a811885bb9748 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.220+0000 7f42f9ffb640 1 -- 172.21.15.42:0/3159618584 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308028be0 con 0x55f612cd6380 2024-03-12T04:17:13.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.220+0000 7f42f9ffb640 1 -- 172.21.15.42:0/3159618584 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4308018070 con 0x55f612cd6380 2024-03-12T04:17:13.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.220+0000 7f42f9ffb640 1 -- 172.21.15.42:0/3159618584 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308005280 con 0x55f612cd6380 2024-03-12T04:17:13.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.221+0000 7f431d628740 1 -- 172.21.15.42:0/3159618584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd6380 msgr2=0x55f612cdae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.221+0000 7f431d628740 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd6380 0x55f612cdae90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55f612cf1d90 tx=0x7f430801c000 comp rx=0 tx=0).stop 2024-03-12T04:17:13.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.222+0000 7f431d628740 1 -- 172.21.15.42:0/3159618584 shutdown_connections 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.222+0000 7f431d628740 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cdef60 0x55f612cdc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.222+0000 7f431d628740 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd6380 0x55f612cdae90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.222+0000 7f431d628740 1 --2- 172.21.15.42:0/3159618584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cd5a60 0x55f612cc6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.222+0000 7f431d628740 1 -- 172.21.15.42:0/3159618584 >> 172.21.15.42:0/3159618584 conn(0x55f612cd0190 msgr2=0x55f612cda780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.222+0000 7f431d628740 1 -- 172.21.15.42:0/3159618584 shutdown_connections 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.222+0000 7f431d628740 1 -- 172.21.15.42:0/3159618584 wait complete. 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.223+0000 7f431d628740 1 -- start start 2024-03-12T04:17:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.223+0000 7f431d628740 1 Processor -- start 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.223+0000 7f431d628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612ce4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.223+0000 7f431d628740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612ce4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.223+0000 7f431d628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612d45570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f431d628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612ce56b0 con 0x55f612cdef60 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f431d628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612ce5820 con 0x55f612cd5a60 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f431d628740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612d45b00 con 0x55f612cd6380 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612ce4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f431d628740 10 client.0 _ll_register_callbacks cb 0x7f430400bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f43137fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612ce4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4318f14640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612d45570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612ce4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56236/0 (socket says 172.21.15.42:56236) 2024-03-12T04:17:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 -- 172.21.15.42:0/887748978 learned_addr learned my addr 172.21.15.42:0/887748978 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 -- 172.21.15.42:0/887748978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 msgr2=0x55f612ce4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612ce4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 -- 172.21.15.42:0/887748978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 msgr2=0x55f612d45570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612d45570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 -- 172.21.15.42:0/887748978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4308004480 con 0x55f612cd5a60 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f4313fff640 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612ce4990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43040052a0 tx=0x7f430400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193be887734ee934 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.224+0000 7f42f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f42f37fe640 1 -- 172.21.15.42:0/887748978 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430400ba70 con 0x55f612cd5a60 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f42f37fe640 1 -- 172.21.15.42:0/887748978 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4304000e40 con 0x55f612cd5a60 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f430c0c1d60 con 0x55f612cd5a60 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f42f37fe640 1 -- 172.21.15.42:0/887748978 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4304005880 con 0x55f612cd5a60 2024-03-12T04:17:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 10 client.25592 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f612bf58c0 con 0x55f612cd5a60 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f42f37fe640 1 -- 172.21.15.42:0/887748978 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4304015070 con 0x55f612cd5a60 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f42f37fe640 1 -- 172.21.15.42:0/887748978 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f43040035f0 con 0x55f612cd5a60 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 10 client.25592 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 10 client.25592 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 20 client.25592 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612d47340 0x55f612d49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 4 client.25592 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.225+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f430c0c1d60 con 0x55f612d47340 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.226+0000 7f43137fe640 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612d47340 0x55f612d49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.226+0000 7f43137fe640 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612d47340 0x55f612d49760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=387c944c13e67f35 server_cookie=643a193f33b83843 in_seq=0 out_seq=0 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.226+0000 7f42f37fe640 10 client.25592 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.226+0000 7f42f37fe640 1 -- 172.21.15.42:0/887748978 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f430c0c1d60 con 0x55f612d47340 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.226+0000 7f42f37fe640 10 client.25592 handle_command_reply: tid=0 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.226+0000 7f431d628740 1 client.25592 shutdown 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 msgr2=0x55f612ce4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612ce4990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43040052a0 tx=0x7f430400b850 comp rx=0 tx=0).stop 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 shutdown_connections 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f612d47340 0x55f612d49760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f612cdef60 0x55f612d45570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f612cd6380 0x55f612ce4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f42f37fe640 0 client.25592 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 --2- 172.21.15.42:0/887748978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f612cd5a60 0x55f612ce4990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 >> 172.21.15.42:0/887748978 conn(0x55f612cd0190 msgr2=0x55f612cd1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 shutdown_connections 2024-03-12T04:17:13.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 1 -- 172.21.15.42:0/887748978 wait complete. 2024-03-12T04:17:13.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431d628740 20 client.25592 trim_cache size 0 max 16384 2024-03-12T04:17:13.234 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42fc0a1c60 msgr2=0x7f42fc0a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431a977640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42fc0a1c60 0x7f42fc0a4120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4314116100 tx=0x7f43080128d0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43141142d0 msgr2=0x7f43141a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.227+0000 7f431a977640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43141142d0 0x7f43141a6920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f430c037040 tx=0x7f430c034460 comp rx=0 tx=0).stop 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 shutdown_connections 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42fc0a1c60 0x7f42fc0a4120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43141142d0 0x7f43141a6920 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4314108c80 0x7f43141a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 --2- 172.21.15.42:0/4047707451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314108360 0x7f43141a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 >> 172.21.15.42:0/4047707451 conn(0x7f431406c260 msgr2=0x7f431406ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 shutdown_connections 2024-03-12T04:17:13.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.228+0000 7f431a977640 1 -- 172.21.15.42:0/4047707451 wait complete. 2024-03-12T04:17:13.247 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:13.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:17:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:13 smithi042 ceph-mon[32260]: pgmap v63: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.45k op/s 2024-03-12T04:17:13.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:13 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3632209679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 -- 172.21.15.42:0/2757674241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a810b7c0 msgr2=0x7fe0a810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 --2- 172.21.15.42:0/2757674241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a810b7c0 0x7fe0a810dc30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009ca0 tx=0x7fe0a002c180 comp rx=0 tx=0).stop 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 -- 172.21.15.42:0/2757674241 shutdown_connections 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 --2- 172.21.15.42:0/2757674241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a810b7c0 0x7fe0a810dc30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 --2- 172.21.15.42:0/2757674241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a8102d10 0x7fe0a810b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 --2- 172.21.15.42:0/2757674241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a81023f0 0x7fe0a81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 -- 172.21.15.42:0/2757674241 >> 172.21.15.42:0/2757674241 conn(0x7fe0a80771e0 msgr2=0x7fe0a80775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 -- 172.21.15.42:0/2757674241 shutdown_connections 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.391+0000 7fe0afa88640 1 -- 172.21.15.42:0/2757674241 wait complete. 2024-03-12T04:17:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 Processor -- start 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 -- start start 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a81023f0 0x7fe0a81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a8102d10 0x7fe0a8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a810b7c0 0x7fe0a819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a819c5a0 con 0x7fe0a8102d10 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a819c710 con 0x7fe0a810b7c0 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0afa88640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a819c8d0 con 0x7fe0a81023f0 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a8102d10 0x7fe0a8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a8102d10 0x7fe0a8197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41762/0 (socket says 172.21.15.42:41762) 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 -- 172.21.15.42:0/146222517 learned_addr learned my addr 172.21.15.42:0/146222517 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad824640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a81023f0 0x7fe0a81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ae025640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a810b7c0 0x7fe0a819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 -- 172.21.15.42:0/146222517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a81023f0 msgr2=0x7fe0a81976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a81023f0 0x7fe0a81976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 -- 172.21.15.42:0/146222517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a810b7c0 msgr2=0x7fe0a819bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a810b7c0 0x7fe0a819bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 -- 172.21.15.42:0/146222517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0009900 con 0x7fe0a8102d10 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad824640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a81023f0 0x7fe0a81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.392+0000 7fe0ad023640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a8102d10 0x7fe0a8197c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe09800e9f0 tx=0x7fe09800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.393+0000 7fe09effd640 1 -- 172.21.15.42:0/146222517 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098009720 con 0x7fe0a8102d10 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.393+0000 7fe09effd640 1 -- 172.21.15.42:0/146222517 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe09800cd80 con 0x7fe0a8102d10 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.393+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a819cbb0 con 0x7fe0a8102d10 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.393+0000 7fe09effd640 1 -- 172.21.15.42:0/146222517 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098010430 con 0x7fe0a8102d10 2024-03-12T04:17:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.393+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a81a4430 con 0x7fe0a8102d10 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.395+0000 7fe09effd640 1 -- 172.21.15.42:0/146222517 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe098005020 con 0x7fe0a8102d10 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.395+0000 7fe0b2665740 1 Processor -- start 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.395+0000 7fe0b2665740 1 -- start start 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.395+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01c660 0x55cbab01ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab025620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab025b60 0x55cbab027f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0b2665740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaafd3960 con 0x55cbab025b60 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0b2665740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaaf87310 con 0x55cbab01c660 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0b2665740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaaf84840 con 0x55cbab01cf80 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab025b60 0x55cbab027f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab025b60 0x55cbab027f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41766/0 (socket says 172.21.15.42:41766) 2024-03-12T04:17:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 -- 172.21.15.42:0/101893901 learned_addr learned my addr 172.21.15.42:0/101893901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ae025640 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01c660 0x55cbab01ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad824640 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab025620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe09effd640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0940a1c20 0x7fe0940a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 -- 172.21.15.42:0/101893901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 msgr2=0x55cbab025620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab025620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 -- 172.21.15.42:0/101893901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01c660 msgr2=0x55cbab01ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01c660 0x55cbab01ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 -- 172.21.15.42:0/101893901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbaaf42060 con 0x55cbab025b60 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ae025640 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01c660 0x55cbab01ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad824640 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab025620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ae025640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0940a1c20 0x7fe0940a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe09effd640 1 -- 172.21.15.42:0/146222517 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe098014070 con 0x7fe0a8102d10 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe0ad023640 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab025b60 0x55cbab027f50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0980cb240 tx=0x7fe098005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91afeb572249ed21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.396+0000 7fe093fff640 1 -- 172.21.15.42:0/101893901 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098019ec0 con 0x55cbab025b60 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe093fff640 1 -- 172.21.15.42:0/101893901 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe09809a5b0 con 0x55cbab025b60 2024-03-12T04:17:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe093fff640 1 -- 172.21.15.42:0/101893901 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980d30c0 con 0x55cbab025b60 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0ae025640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0940a1c20 0x7fe0940a40e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0004af0 tx=0x7fe0a00095c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 -- 172.21.15.42:0/101893901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab025b60 msgr2=0x55cbab027f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab025b60 0x55cbab027f50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0980cb240 tx=0x7fe098005f10 comp rx=0 tx=0).stop 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 -- 172.21.15.42:0/101893901 shutdown_connections 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab025b60 0x55cbab027f50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab025620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 --2- 172.21.15.42:0/101893901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01c660 0x55cbab01ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 -- 172.21.15.42:0/101893901 >> 172.21.15.42:0/101893901 conn(0x55cbab016d90 msgr2=0x55cbab017170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 -- 172.21.15.42:0/101893901 shutdown_connections 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.397+0000 7fe0b2665740 1 -- 172.21.15.42:0/101893901 wait complete. 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 -- start start 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 Processor -- start 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01c660 0x55cbab022ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01cf80 0x55cbab0230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab023620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaaf80020 con 0x55cbab01cf80 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaac53430 con 0x55cbab025b60 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaac0fe00 con 0x55cbab01c660 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0b2665740 10 client.0 _ll_register_callbacks cb 0x55cbaaf27cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0ad023640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01cf80 0x55cbab0230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0ad023640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01cf80 0x55cbab0230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41768/0 (socket says 172.21.15.42:41768) 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0ad023640 1 -- 172.21.15.42:0/751649202 learned_addr learned my addr 172.21.15.42:0/751649202 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0ae025640 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab023620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.398+0000 7fe0ad824640 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01c660 0x55cbab022ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0ad023640 1 -- 172.21.15.42:0/751649202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01c660 msgr2=0x55cbab022ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0ad023640 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01c660 0x55cbab022ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0ad023640 1 -- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 msgr2=0x55cbab023620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0ad023640 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab023620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0ad023640 1 -- 172.21.15.42:0/751649202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a81a4430 con 0x55cbab01cf80 2024-03-12T04:17:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0ad023640 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01cf80 0x55cbab0230e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe09809a720 tx=0x7fe098019ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13d6f4d2bb95640 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0917fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0917fa640 1 -- 172.21.15.42:0/751649202 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09809aee0 con 0x55cbab01cf80 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0917fa640 1 -- 172.21.15.42:0/751649202 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe0980046c0 con 0x55cbab01cf80 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbaac7f7e0 con 0x55cbab01cf80 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0917fa640 1 -- 172.21.15.42:0/751649202 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe09809a140 con 0x55cbab01cf80 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0b2665740 10 client.16101 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cbaaf7e6a0 con 0x55cbab01cf80 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0917fa640 1 -- 172.21.15.42:0/751649202 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55cbaac7f7e0 con 0x55cbab01cf80 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0917fa640 1 -- 172.21.15.42:0/751649202 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe09801f060 con 0x55cbab01cf80 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0b2665740 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0b2665740 10 client.16101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0b2665740 20 client.16101 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.399+0000 7fe0b2665740 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab035c80 0x55cbab0380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.400+0000 7fe0b2665740 4 client.16101 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:13.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.400+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbaacbc720 con 0x55cbab035c80 2024-03-12T04:17:13.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.400+0000 7fe0ad824640 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab035c80 0x55cbab0380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.401+0000 7fe0ad824640 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab035c80 0x55cbab0380a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b45bc20916d6085 server_cookie=5941bfc6f225a6ad in_seq=0 out_seq=0 2024-03-12T04:17:13.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.401+0000 7fe0917fa640 10 client.16101 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.402+0000 7fe0917fa640 1 -- 172.21.15.42:0/751649202 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbaacbc720 con 0x55cbab035c80 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.402+0000 7fe0917fa640 10 client.16101 handle_command_reply: tid=0 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.402+0000 7fe0b2665740 1 client.16101 shutdown 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.402+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01cf80 msgr2=0x55cbab0230e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.402+0000 7fe0b2665740 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01cf80 0x55cbab0230e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe09809a720 tx=0x7fe098019ec0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.402+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 shutdown_connections 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.402+0000 7fe0b2665740 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab035c80 0x55cbab0380a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0b2665740 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab023620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0b2665740 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01cf80 0x55cbab0230e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0b2665740 1 --2- 172.21.15.42:0/751649202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01c660 0x55cbab022ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 >> 172.21.15.42:0/751649202 conn(0x55cbab016d90 msgr2=0x55cbab018250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0917fa640 0 client.16101 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:13.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 shutdown_connections 2024-03-12T04:17:13.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0b2665740 1 -- 172.21.15.42:0/751649202 wait complete. 2024-03-12T04:17:13.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.403+0000 7fe0b2665740 20 client.16101 trim_cache size 0 max 16384 2024-03-12T04:17:13.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.423+0000 7fe0b2665740 1 Processor -- start 2024-03-12T04:17:13.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.423+0000 7fe0b2665740 1 -- start start 2024-03-12T04:17:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.423+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab023440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.423+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab02c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.423+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab02ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0b2665740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaaf05130 con 0x55cbab01c660 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0b2665740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab0329d0 con 0x55cbab025b60 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0b2665740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab01c0a0 con 0x55cbab01cf80 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab023440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad023640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab02c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab023440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41782/0 (socket says 172.21.15.42:41782) 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ae025640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab02ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 -- 172.21.15.42:0/4072267272 learned_addr learned my addr 172.21.15.42:0/4072267272 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 -- 172.21.15.42:0/4072267272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 msgr2=0x55cbab02c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 --2- 172.21.15.42:0/4072267272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab02c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 -- 172.21.15.42:0/4072267272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 msgr2=0x55cbab02ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 --2- 172.21.15.42:0/4072267272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab02ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 -- 172.21.15.42:0/4072267272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbaaf1ac20 con 0x55cbab01c660 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0ad824640 1 --2- 172.21.15.42:0/4072267272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab023440 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005ca0 tx=0x7fe0a400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6dcacfdc85d062 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.424+0000 7fe0937fe640 1 -- 172.21.15.42:0/4072267272 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4017040 con 0x55cbab01c660 2024-03-12T04:17:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.425+0000 7fe0937fe640 1 -- 172.21.15.42:0/4072267272 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe0a4020070 con 0x55cbab01c660 2024-03-12T04:17:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.425+0000 7fe0937fe640 1 -- 172.21.15.42:0/4072267272 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40047a0 con 0x55cbab01c660 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 -- 172.21.15.42:0/4072267272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 msgr2=0x55cbab023440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 --2- 172.21.15.42:0/4072267272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab023440 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005ca0 tx=0x7fe0a400bdd0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 -- 172.21.15.42:0/4072267272 shutdown_connections 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 --2- 172.21.15.42:0/4072267272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab025b60 0x55cbab02ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 --2- 172.21.15.42:0/4072267272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab01cf80 0x55cbab02c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 --2- 172.21.15.42:0/4072267272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab023440 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 -- 172.21.15.42:0/4072267272 >> 172.21.15.42:0/4072267272 conn(0x55cbab016d90 msgr2=0x55cbab01dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 -- 172.21.15.42:0/4072267272 shutdown_connections 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.426+0000 7fe0b2665740 1 -- 172.21.15.42:0/4072267272 wait complete. 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 -- start start 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 Processor -- start 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab02cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01cf80 0x55cbab02b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab025b60 0x55cbab02b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab02be20 con 0x55cbab01c660 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab02bf90 con 0x55cbab01cf80 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab08c190 con 0x55cbab025b60 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0b2665740 10 client.0 _ll_register_callbacks cb 0x7fe0980d33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0ad824640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab02cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0ad824640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab02cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41792/0 (socket says 172.21.15.42:41792) 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0ad824640 1 -- 172.21.15.42:0/3987948112 learned_addr learned my addr 172.21.15.42:0/3987948112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.427+0000 7fe0ad023640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01cf80 0x55cbab02b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ae025640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab025b60 0x55cbab02b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ad824640 1 -- 172.21.15.42:0/3987948112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab025b60 msgr2=0x55cbab02b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ad824640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab025b60 0x55cbab02b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ad824640 1 -- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01cf80 msgr2=0x55cbab02b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ad824640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01cf80 0x55cbab02b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ad824640 1 -- 172.21.15.42:0/3987948112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a40042c0 con 0x55cbab01c660 2024-03-12T04:17:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ad023640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01cf80 0x55cbab02b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0ad824640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab02cd90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4003ec0 tx=0x7fe0a4003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5dafac37fd81f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0917fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0917fa640 1 -- 172.21.15.42:0/3987948112 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4017050 con 0x55cbab01c660 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0917fa640 1 -- 172.21.15.42:0/3987948112 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe0a401c620 con 0x55cbab01c660 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.428+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe09809a5b0 con 0x55cbab01c660 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0917fa640 1 -- 172.21.15.42:0/3987948112 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0a4003c20 con 0x55cbab01c660 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 10 client.16107 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cbaaf408c0 con 0x55cbab01c660 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0917fa640 1 -- 172.21.15.42:0/3987948112 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe09809a5b0 con 0x55cbab01c660 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0917fa640 1 -- 172.21.15.42:0/3987948112 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe0a402b020 con 0x55cbab01c660 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 10 client.16107 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 10 client.16107 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 20 client.16107 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab08d6f0 0x55cbab08fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 4 client.16107 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:13.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbaad0b890 con 0x55cbab08d6f0 2024-03-12T04:17:13.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.429+0000 7fe0ad023640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab08d6f0 0x55cbab08fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.430+0000 7fe0917fa640 10 client.16107 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.430+0000 7fe0ad023640 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab08d6f0 0x55cbab08fae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd2152c6d901256d server_cookie=91b701bb44cbd3b6 in_seq=0 out_seq=0 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.430+0000 7fe0917fa640 1 -- 172.21.15.42:0/3987948112 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbaad0b890 con 0x55cbab08d6f0 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.430+0000 7fe0917fa640 10 client.16107 handle_command_reply: tid=0 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.430+0000 7fe0b2665740 1 client.16107 shutdown 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 msgr2=0x55cbab02cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab02cd90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4003ec0 tx=0x7fe0a4003ef0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 shutdown_connections 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cbab08d6f0 0x55cbab08fae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cbab025b60 0x55cbab02b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbab01cf80 0x55cbab02b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 --2- 172.21.15.42:0/3987948112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbab01c660 0x55cbab02cd90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0917fa640 0 client.16107 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 >> 172.21.15.42:0/3987948112 conn(0x55cbab016d90 msgr2=0x55cbab0112f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 shutdown_connections 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 1 -- 172.21.15.42:0/3987948112 wait complete. 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0b2665740 20 client.16107 trim_cache size 0 max 16384 2024-03-12T04:17:13.437 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:13.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0940a1c20 msgr2=0x7fe0940a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.431+0000 7fe0afa88640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0940a1c20 0x7fe0940a40e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0004af0 tx=0x7fe0a00095c0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a8102d10 msgr2=0x7fe0a8197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a8102d10 0x7fe0a8197c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe09800e9f0 tx=0x7fe09800edf0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 shutdown_connections 2024-03-12T04:17:13.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0940a1c20 0x7fe0940a40e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a810b7c0 0x7fe0a819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0a8102d10 0x7fe0a8197c20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 --2- 172.21.15.42:0/146222517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a81023f0 0x7fe0a81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 >> 172.21.15.42:0/146222517 conn(0x7fe0a80771e0 msgr2=0x7fe0a80fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 shutdown_connections 2024-03-12T04:17:13.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.432+0000 7fe0afa88640 1 -- 172.21.15.42:0/146222517 wait complete. 2024-03-12T04:17:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:13 smithi119 ceph-mon[36119]: pgmap v63: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.45k op/s 2024-03-12T04:17:13.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:13 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3632209679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:17:13.450 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:13.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:17:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:13 smithi136 ceph-mon[35571]: pgmap v63: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.45k op/s 2024-03-12T04:17:13.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:13 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3632209679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:17:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.606+0000 7f0e95ffb640 1 -- 172.21.15.42:0/3725120067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8802cbf0 con 0x7f0e981008c0 2024-03-12T04:17:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.607+0000 7f0e97fff640 1 -- 172.21.15.42:0/3725120067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e981008c0 msgr2=0x7f0e98104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.607+0000 7f0e97fff640 1 --2- 172.21.15.42:0/3725120067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e981008c0 0x7f0e98104d80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009900 tx=0x7f0e8802bd10 comp rx=0 tx=0).stop 2024-03-12T04:17:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.607+0000 7f0e97fff640 1 -- 172.21.15.42:0/3725120067 shutdown_connections 2024-03-12T04:17:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.607+0000 7f0e97fff640 1 --2- 172.21.15.42:0/3725120067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981053a0 0x7f0e98105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.607+0000 7f0e97fff640 1 --2- 172.21.15.42:0/3725120067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e981008c0 0x7f0e98104d80 secure :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009900 tx=0x7f0e8802bd10 comp rx=0 tx=0).stop 2024-03-12T04:17:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.607+0000 7f0e97fff640 1 --2- 172.21.15.42:0/3725120067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e980fffa0 0x7f0e98100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.607+0000 7f0e97fff640 1 -- 172.21.15.42:0/3725120067 >> 172.21.15.42:0/3725120067 conn(0x7f0e980fb9d0 msgr2=0x7f0e980fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.608+0000 7f0e97fff640 1 -- 172.21.15.42:0/3725120067 shutdown_connections 2024-03-12T04:17:13.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.609+0000 7f0e97fff640 1 -- 172.21.15.42:0/3725120067 wait complete. 2024-03-12T04:17:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.609+0000 7f0e97fff640 1 Processor -- start 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e97fff640 1 -- start start 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e97fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e980fffa0 0x7f0e9819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e97fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e981053a0 0x7f0e9819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e97fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981a2730 0x7f0e981a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e97fff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981a3120 con 0x7f0e980fffa0 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e97fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981a3260 con 0x7f0e981a2730 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e97fff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981a3420 con 0x7f0e981053a0 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981a2730 0x7f0e981a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981a2730 0x7f0e981a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56290/0 (socket says 172.21.15.42:56290) 2024-03-12T04:17:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.610+0000 7f0e977fe640 1 -- 172.21.15.42:0/2520937063 learned_addr learned my addr 172.21.15.42:0/2520937063 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e980fffa0 0x7f0e9819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e967fc640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e981053a0 0x7f0e9819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e977fe640 1 -- 172.21.15.42:0/2520937063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e981053a0 msgr2=0x7f0e9819e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e977fe640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e981053a0 0x7f0e9819e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e977fe640 1 -- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e980fffa0 msgr2=0x7f0e9819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e977fe640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e980fffa0 0x7f0e9819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e977fe640 1 -- 172.21.15.42:0/2520937063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88009560 con 0x7f0e981a2730 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e967fc640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e981053a0 0x7f0e9819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.611+0000 7f0e977fe640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981a2730 0x7f0e981a2be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000d750 tx=0x7f0e8000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.612+0000 7f0e8ffff640 1 -- 172.21.15.42:0/2520937063 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80009bd0 con 0x7f0e981a2730 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.612+0000 7f0e8ffff640 1 -- 172.21.15.42:0/2520937063 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e8000de60 con 0x7f0e981a2730 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.612+0000 7f0e8ffff640 1 -- 172.21.15.42:0/2520937063 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80004e00 con 0x7f0e981a2730 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.612+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e98106350 con 0x7f0e981a2730 2024-03-12T04:17:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.612+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e98106810 con 0x7f0e981a2730 2024-03-12T04:17:13.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.615+0000 7f0e8ffff640 1 -- 172.21.15.42:0/2520937063 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0e80004f60 con 0x7f0e981a2730 2024-03-12T04:17:13.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e8ffff640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e840a1ca0 0x7f0e840a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 Processor -- start 2024-03-12T04:17:13.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 -- start start 2024-03-12T04:17:13.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e840a1ca0 0x7f0e840a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f7fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8f2f960 con 0x555fa8f74440 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8ee3310 con 0x555fa8f74d60 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.616+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8ee0840 con 0x555fa8f7d940 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56304/0 (socket says 172.21.15.42:56304) 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 -- 172.21.15.42:0/3064831631 learned_addr learned my addr 172.21.15.42:0/3064831631 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e8ffff640 1 -- 172.21.15.42:0/2520937063 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0e800cd1a0 con 0x7f0e981a2730 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e977fe640 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f7fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e840a1ca0 0x7f0e840a4160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e90005fd0 tx=0x7f0e90009f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 -- 172.21.15.42:0/3064831631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 msgr2=0x555fa8f7fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f7fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 -- 172.21.15.42:0/3064831631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 msgr2=0x555fa8f74820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f74820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 -- 172.21.15.42:0/3064831631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa8e9e060 con 0x555fa8f74d60 2024-03-12T04:17:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.617+0000 7f0e967fc640 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005ab0 tx=0x7f0e88005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b1090670cbccb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.618+0000 7f0e8d7fa640 1 -- 172.21.15.42:0/3064831631 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88004ac0 con 0x555fa8f74d60 2024-03-12T04:17:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.618+0000 7f0e8d7fa640 1 -- 172.21.15.42:0/3064831631 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e880044f0 con 0x555fa8f74d60 2024-03-12T04:17:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.618+0000 7f0e8d7fa640 1 -- 172.21.15.42:0/3064831631 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88003880 con 0x555fa8f74d60 2024-03-12T04:17:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.619+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3064831631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 msgr2=0x555fa8f7d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.619+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005ab0 tx=0x7f0e88005e80 comp rx=0 tx=0).stop 2024-03-12T04:17:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.620+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3064831631 shutdown_connections 2024-03-12T04:17:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.621+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f7fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.621+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7d400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.621+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3064831631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f74820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.621+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3064831631 >> 172.21.15.42:0/3064831631 conn(0x555fa8f6eb70 msgr2=0x555fa8f6ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.621+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3064831631 shutdown_connections 2024-03-12T04:17:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.621+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3064831631 wait complete. 2024-03-12T04:17:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.622+0000 7f0e9ffc4740 1 -- start start 2024-03-12T04:17:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.622+0000 7f0e9ffc4740 1 Processor -- start 2024-03-12T04:17:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.622+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f7a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.622+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.622+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f7d940 0x555fa8f7b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8edc020 con 0x555fa8f7d940 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8baf430 con 0x555fa8f74d60 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8b6be00 con 0x555fa8f74440 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e9ffc4740 10 client.0 _ll_register_callbacks cb 0x555fa8e83cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f7a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f7a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59596/0 (socket says 172.21.15.42:59596) 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 -- 172.21.15.42:0/3229973345 learned_addr learned my addr 172.21.15.42:0/3229973345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e967fc640 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e977fe640 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f7d940 0x555fa8f7b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 -- 172.21.15.42:0/3229973345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 msgr2=0x555fa8f7aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 -- 172.21.15.42:0/3229973345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f7d940 msgr2=0x555fa8f7b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f7d940 0x555fa8f7b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.623+0000 7f0e96ffd640 1 -- 172.21.15.42:0/3229973345 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e98114d00 con 0x555fa8f74440 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.624+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f7a970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e90013eb0 tx=0x7f0e90004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24f4688440226b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.624+0000 7f0e7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.625+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/3229973345 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90028640 con 0x555fa8f74440 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.625+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/3229973345 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0e90019040 con 0x555fa8f74440 2024-03-12T04:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.625+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fa8bdb7e0 con 0x555fa8f74440 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.625+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/3229973345 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e900132e0 con 0x555fa8f74440 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.625+0000 7f0e9ffc4740 10 client.25504 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.625+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555fa8eda6a0 con 0x555fa8f74440 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/3229973345 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e900118f0 con 0x555fa8f74440 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/3229973345 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e90027700 con 0x555fa8f74440 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e9ffc4740 10 client.25504 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e9ffc4740 10 client.25504 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e9ffc4740 20 client.25504 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8f8dd00 0x555fa8f901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e9ffc4740 4 client.25504 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fa8bdb7e0 con 0x555fa8f8dd00 2024-03-12T04:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.626+0000 7f0e967fc640 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8f8dd00 0x555fa8f901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.627+0000 7f0e967fc640 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8f8dd00 0x555fa8f901c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8dc0cd30a2f7f65 server_cookie=ac6407b104dc8a2f in_seq=0 out_seq=0 2024-03-12T04:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.627+0000 7f0e7e7fc640 10 client.25504 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:13.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.628+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/3229973345 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fa8bdb7e0 con 0x555fa8f8dd00 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.628+0000 7f0e7e7fc640 10 client.25504 handle_command_reply: tid=0 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.628+0000 7f0e9ffc4740 1 client.25504 shutdown 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.628+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 msgr2=0x555fa8f7a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.628+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f7a970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e90013eb0 tx=0x7f0e90004ab0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 shutdown_connections 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8f8dd00 0x555fa8f901c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f7d940 0x555fa8f7b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f7aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/3229973345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f7a970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 >> 172.21.15.42:0/3229973345 conn(0x555fa8f6eb70 msgr2=0x555fa8f708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e7e7fc640 0 client.25504 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 shutdown_connections 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/3229973345 wait complete. 2024-03-12T04:17:13.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.629+0000 7f0e9ffc4740 20 client.25504 trim_cache size 0 max 16384 2024-03-12T04:17:13.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.648+0000 7f0e9ffc4740 1 Processor -- start 2024-03-12T04:17:13.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.648+0000 7f0e9ffc4740 1 -- start start 2024-03-12T04:17:13.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f7d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f79240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f79780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f7d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f7d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41828/0 (socket says 172.21.15.42:41828) 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8e61130 con 0x555fa8f74440 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8f6e6a0 con 0x555fa8f74d60 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8f90380 con 0x555fa8f7d940 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e96ffd640 1 -- 172.21.15.42:0/150654043 learned_addr learned my addr 172.21.15.42:0/150654043 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e967fc640 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f79240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.649+0000 7f0e977fe640 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f79780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e967fc640 1 -- 172.21.15.42:0/150654043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 msgr2=0x555fa8f79780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e967fc640 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f79780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e967fc640 1 -- 172.21.15.42:0/150654043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 msgr2=0x555fa8f7d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e967fc640 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f7d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e967fc640 1 -- 172.21.15.42:0/150654043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa8e76c20 con 0x555fa8f74d60 2024-03-12T04:17:13.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e967fc640 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f79240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009850 tx=0x7f0e8802e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c06adc99cdc9a6bb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e8cff9640 1 -- 172.21.15.42:0/150654043 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8804e070 con 0x555fa8f74d60 2024-03-12T04:17:13.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e8cff9640 1 -- 172.21.15.42:0/150654043 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e8802ce20 con 0x555fa8f74d60 2024-03-12T04:17:13.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.650+0000 7f0e8cff9640 1 -- 172.21.15.42:0/150654043 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8804f710 con 0x555fa8f74d60 2024-03-12T04:17:13.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.652+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/150654043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 msgr2=0x555fa8f79240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.653+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f79240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009850 tx=0x7f0e8802e040 comp rx=0 tx=0).stop 2024-03-12T04:17:13.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.653+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/150654043 shutdown_connections 2024-03-12T04:17:13.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.653+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f7d940 0x555fa8f79780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.653+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f74d60 0x555fa8f79240 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.653+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/150654043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74440 0x555fa8f7d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.653+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/150654043 >> 172.21.15.42:0/150654043 conn(0x555fa8f6eb70 msgr2=0x555fa8f75c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.654+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/150654043 shutdown_connections 2024-03-12T04:17:13.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.654+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/150654043 wait complete. 2024-03-12T04:17:13.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.655+0000 7f0e9ffc4740 1 -- start start 2024-03-12T04:17:13.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.655+0000 7f0e9ffc4740 1 Processor -- start 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f82a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74d60 0x555fa8f82f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e9ffc4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f7d940 0x555fa8fdd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8fdd860 con 0x555fa8f74d60 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8fdd9d0 con 0x555fa8f7d940 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e9ffc4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa8fddb90 con 0x555fa8f74440 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e9ffc4740 10 client.0 _ll_register_callbacks cb 0x7f0e98069340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f82a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f82a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59616/0 (socket says 172.21.15.42:59616) 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e96ffd640 1 -- 172.21.15.42:0/4033253910 learned_addr learned my addr 172.21.15.42:0/4033253910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e967fc640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74d60 0x555fa8f82f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e977fe640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f7d940 0x555fa8fdd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e967fc640 1 -- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 msgr2=0x555fa8f82a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e967fc640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f82a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e967fc640 1 -- 172.21.15.42:0/4033253910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f7d940 msgr2=0x555fa8fdd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e967fc640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f7d940 0x555fa8fdd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e967fc640 1 -- 172.21.15.42:0/4033253910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fa8e9e060 con 0x555fa8f74d60 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f82a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.656+0000 7f0e977fe640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f7d940 0x555fa8fdd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e967fc640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74d60 0x555fa8f82f70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009900 tx=0x7f0e88005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3402f8628fd00e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/4033253910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8804e050 con 0x555fa8f74d60 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/4033253910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0e88000e40 con 0x555fa8f74d60 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e880044f0 con 0x555fa8f74d60 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/4033253910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e8804c070 con 0x555fa8f74d60 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e9ffc4740 10 client.16125 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.657+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555fa8e9c8c0 con 0x555fa8f74d60 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/4033253910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e880044f0 con 0x555fa8f74d60 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/4033253910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e8803c040 con 0x555fa8f74d60 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e9ffc4740 10 client.16125 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e9ffc4740 10 client.16125 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e9ffc4740 20 client.16125 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8fe4ef0 0x555fa8fe72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e9ffc4740 4 client.16125 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fa8c18720 con 0x555fa8fe4ef0 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.658+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8fe4ef0 0x555fa8fe72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.659+0000 7f0e96ffd640 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8fe4ef0 0x555fa8fe72e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88131ab91c36873 server_cookie=3a9d8a020f29aa00 in_seq=0 out_seq=0 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.659+0000 7f0e7e7fc640 10 client.16125 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e7e7fc640 1 -- 172.21.15.42:0/4033253910 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fa8c18720 con 0x555fa8fe4ef0 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e7e7fc640 10 client.16125 handle_command_reply: tid=0 2024-03-12T04:17:13.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 client.16125 shutdown 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74d60 msgr2=0x555fa8f82f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74d60 0x555fa8f82f70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009900 tx=0x7f0e88005be0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 shutdown_connections 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555fa8fe4ef0 0x555fa8fe72e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fa8f7d940 0x555fa8fdd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555fa8f74d60 0x555fa8f82f70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e7e7fc640 0 client.16125 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 --2- 172.21.15.42:0/4033253910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555fa8f74440 0x555fa8f82a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 >> 172.21.15.42:0/4033253910 conn(0x555fa8f6eb70 msgr2=0x555fa8f79200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 shutdown_connections 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 1 -- 172.21.15.42:0/4033253910 wait complete. 2024-03-12T04:17:13.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.660+0000 7f0e9ffc4740 20 client.16125 trim_cache size 0 max 16384 2024-03-12T04:17:13.666 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:13.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.661+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e840a1ca0 msgr2=0x7f0e840a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.661+0000 7f0e97fff640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e840a1ca0 0x7f0e840a4160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e90005fd0 tx=0x7f0e90009f90 comp rx=0 tx=0).stop 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.661+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981a2730 msgr2=0x7f0e981a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.661+0000 7f0e97fff640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981a2730 0x7f0e981a2be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000d750 tx=0x7f0e8000dc10 comp rx=0 tx=0).stop 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 shutdown_connections 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e840a1ca0 0x7f0e840a4160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e981a2730 0x7f0e981a2be0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e981053a0 0x7f0e9819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 --2- 172.21.15.42:0/2520937063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e980fffa0 0x7f0e9819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 >> 172.21.15.42:0/2520937063 conn(0x7f0e980fb9d0 msgr2=0x7f0e980fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 shutdown_connections 2024-03-12T04:17:13.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.662+0000 7f0e97fff640 1 -- 172.21.15.42:0/2520937063 wait complete. 2024-03-12T04:17:13.682 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:13.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:17:13.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.846+0000 7f01f92c0640 1 -- 172.21.15.42:0/3441473100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f41006a0 msgr2=0x7f01f4104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.846+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3441473100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f41006a0 0x7f01f4104440 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009930 tx=0x7f01e802ce50 comp rx=0 tx=0).stop 2024-03-12T04:17:13.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.846+0000 7f01f92c0640 1 -- 172.21.15.42:0/3441473100 shutdown_connections 2024-03-12T04:17:13.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.846+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3441473100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f4104b70 0x7f01f410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.846+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3441473100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f41006a0 0x7f01f4104440 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.846+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3441473100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f40ffcf0 0x7f01f41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.846+0000 7f01f92c0640 1 -- 172.21.15.42:0/3441473100 >> 172.21.15.42:0/3441473100 conn(0x7f01f40752e0 msgr2=0x7f01f40756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.847+0000 7f01f92c0640 1 -- 172.21.15.42:0/3441473100 shutdown_connections 2024-03-12T04:17:13.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.847+0000 7f01f92c0640 1 -- 172.21.15.42:0/3441473100 wait complete. 2024-03-12T04:17:13.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.847+0000 7f01f92c0640 1 Processor -- start 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f92c0640 1 -- start start 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f92c0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40ffcf0 0x7f01f4197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f92c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4104b70 0x7f01f4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f92c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f419be00 0x7f01f419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f92c0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f419c7f0 con 0x7f01f419be00 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f92c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f419c960 con 0x7f01f4104b70 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f92c0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f419cad0 con 0x7f01f40ffcf0 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40ffcf0 0x7f01f4197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40ffcf0 0x7f01f4197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59646/0 (socket says 172.21.15.42:59646) 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f2ffd640 1 -- 172.21.15.42:0/3557419755 learned_addr learned my addr 172.21.15.42:0/3557419755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f37fe640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f419be00 0x7f01f419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.848+0000 7f01f27fc640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4104b70 0x7f01f4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01f2ffd640 1 -- 172.21.15.42:0/3557419755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4104b70 msgr2=0x7f01f4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4104b70 0x7f01f4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01f2ffd640 1 -- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f419be00 msgr2=0x7f01f419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f419be00 0x7f01f419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01f2ffd640 1 -- 172.21.15.42:0/3557419755 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8009560 con 0x7f01f40ffcf0 2024-03-12T04:17:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40ffcf0 0x7f01f4197540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f01e400d950 tx=0x7f01e400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01dbfff640 1 -- 172.21.15.42:0/3557419755 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4009c30 con 0x7f01f40ffcf0 2024-03-12T04:17:13.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01dbfff640 1 -- 172.21.15.42:0/3557419755 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01e4009d90 con 0x7f01f40ffcf0 2024-03-12T04:17:13.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.849+0000 7f01dbfff640 1 -- 172.21.15.42:0/3557419755 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e400f7e0 con 0x7f01f40ffcf0 2024-03-12T04:17:13.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.850+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4101380 con 0x7f01f40ffcf0 2024-03-12T04:17:13.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.850+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f41017b0 con 0x7f01f40ffcf0 2024-03-12T04:17:13.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.852+0000 7f01dbfff640 1 -- 172.21.15.42:0/3557419755 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f01e4010460 con 0x7f01f40ffcf0 2024-03-12T04:17:13.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 Processor -- start 2024-03-12T04:17:13.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01dbfff640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f01dc0a1c20 0x7f01dc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 -- start start 2024-03-12T04:17:13.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01dbfff640 1 -- 172.21.15.42:0/3557419755 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f01e40cd350 con 0x7f01f40ffcf0 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721edabb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721ed5b960 con 0x55721eda0b70 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721ed0f310 con 0x55721eda0250 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01fbf6c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721ed0c840 con 0x55721eda9750 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01f37fe640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f01dc0a1c20 0x7f01dc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01f27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56368/0 (socket says 172.21.15.42:56368) 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.853+0000 7f01f27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41868/0 (socket says 172.21.15.42:41868) 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 -- 172.21.15.42:0/1350836804 learned_addr learned my addr 172.21.15.42:0/1350836804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f37fe640 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721edabb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 -- 172.21.15.42:0/1350836804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 msgr2=0x55721edabb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721edabb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 -- 172.21.15.42:0/1350836804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 msgr2=0x55721eda9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 -- 172.21.15.42:0/1350836804 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55721ecca060 con 0x55721eda0250 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda0630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01e4096430 tx=0x7f01e409a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20897ee543ae81de server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01d97fa640 1 -- 172.21.15.42:0/1350836804 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e409e120 con 0x55721eda0250 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.854+0000 7f01f37fe640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f01dc0a1c20 0x7f01dc0a40e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005fd0 tx=0x7f01ec013ae0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.855+0000 7f01d97fa640 1 -- 172.21.15.42:0/1350836804 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01e40a2770 con 0x55721eda0250 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.855+0000 7f01d97fa640 1 -- 172.21.15.42:0/1350836804 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40a22e0 con 0x55721eda0250 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.855+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1350836804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 msgr2=0x55721eda0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.855+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda0630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01e4096430 tx=0x7f01e409a6d0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.856+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1350836804 shutdown_connections 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.856+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721edabb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.856+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.856+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1350836804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda0630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.856+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1350836804 >> 172.21.15.42:0/1350836804 conn(0x55721ed9a980 msgr2=0x55721ed9ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.856+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1350836804 shutdown_connections 2024-03-12T04:17:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.856+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1350836804 wait complete. 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.857+0000 7f01fbf6c740 1 -- start start 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.857+0000 7f01fbf6c740 1 Processor -- start 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.857+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.857+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01fbf6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721ed08020 con 0x55721eda0b70 2024-03-12T04:17:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01fbf6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721e9db430 con 0x55721eda0250 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56376/0 (socket says 172.21.15.42:56376) 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 -- 172.21.15.42:0/17433928 learned_addr learned my addr 172.21.15.42:0/17433928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f27fc640 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f37fe640 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721e997e00 con 0x55721eda9750 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01fbf6c740 10 client.0 _ll_register_callbacks cb 0x55721ecafcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 -- 172.21.15.42:0/17433928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 msgr2=0x55721eda7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 -- 172.21.15.42:0/17433928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 msgr2=0x55721eda6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.858+0000 7f01f2ffd640 1 -- 172.21.15.42:0/17433928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f41017b0 con 0x55721eda0250 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda6680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01e4099f60 tx=0x7f01e409e1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29fed18008ddbd60 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01d6ff5640 1 -- 172.21.15.42:0/17433928 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b82f0 con 0x55721eda0250 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01d6ff5640 1 -- 172.21.15.42:0/17433928 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f01e40b8450 con 0x55721eda0250 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55721ea077e0 con 0x55721eda0250 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01d6ff5640 1 -- 172.21.15.42:0/17433928 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e40a22e0 con 0x55721eda0250 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01fbf6c740 10 client.25637 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.859+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55721ed066a0 con 0x55721eda0250 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01d6ff5640 1 -- 172.21.15.42:0/17433928 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01e40a2440 con 0x55721eda0250 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01d6ff5640 1 -- 172.21.15.42:0/17433928 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f01e40cd070 con 0x55721eda0250 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01fbf6c740 10 client.25637 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01fbf6c740 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01fbf6c740 20 client.25637 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721edc1e80 0x55721edc42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01fbf6c740 4 client.25637 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55721ea077e0 con 0x55721edc1e80 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.860+0000 7f01f27fc640 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721edc1e80 0x55721edc42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.861+0000 7f01f27fc640 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721edc1e80 0x55721edc42f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e1d40876ca0b4b server_cookie=13faae6e0db90bf1 in_seq=0 out_seq=0 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.861+0000 7f01d6ff5640 10 client.25637 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01d6ff5640 1 -- 172.21.15.42:0/17433928 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55721ea077e0 con 0x55721edc1e80 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01d6ff5640 10 client.25637 handle_command_reply: tid=0 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 client.25637 shutdown 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 msgr2=0x55721eda6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda6680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01e4099f60 tx=0x7f01e409e1a0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 shutdown_connections 2024-03-12T04:17:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721edc1e80 0x55721edc42f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/17433928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda6680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01d6ff5640 0 client.25637 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 >> 172.21.15.42:0/17433928 conn(0x55721ed9a980 msgr2=0x55721ed9c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 shutdown_connections 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.862+0000 7f01fbf6c740 1 -- 172.21.15.42:0/17433928 wait complete. 2024-03-12T04:17:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.863+0000 7f01fbf6c740 20 client.25637 trim_cache size 0 max 16384 2024-03-12T04:17:13.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.884+0000 7f01fbf6c740 1 Processor -- start 2024-03-12T04:17:13.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01fbf6c740 1 -- start start 2024-03-12T04:17:13.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01fbf6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721ec8d130 con 0x55721eda0b70 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01fbf6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721edb55a0 con 0x55721eda0250 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56378/0 (socket says 172.21.15.42:56378) 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01f2ffd640 1 -- 172.21.15.42:0/60042976 learned_addr learned my addr 172.21.15.42:0/60042976 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01f27fc640 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01fbf6c740 1 -- 172.21.15.42:0/60042976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721ed965f0 con 0x55721eda9750 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.885+0000 7f01f37fe640 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.886+0000 7f01f37fe640 1 -- 172.21.15.42:0/60042976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 msgr2=0x55721eda8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.886+0000 7f01f37fe640 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.886+0000 7f01f37fe640 1 -- 172.21.15.42:0/60042976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 msgr2=0x55721eda3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.886+0000 7f01f37fe640 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.886+0000 7f01f37fe640 1 -- 172.21.15.42:0/60042976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55721eca2c20 con 0x55721eda9750 2024-03-12T04:17:13.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.886+0000 7f01f37fe640 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda3e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00f3c0 tx=0x7f01ec009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa37fe5b7b5c2bbf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.887+0000 7f01d8ff9640 1 -- 172.21.15.42:0/60042976 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00fe90 con 0x55721eda9750 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.887+0000 7f01d8ff9640 1 -- 172.21.15.42:0/60042976 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01ec004050 con 0x55721eda9750 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.887+0000 7f01d8ff9640 1 -- 172.21.15.42:0/60042976 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec005280 con 0x55721eda9750 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.887+0000 7f01fbf6c740 1 -- 172.21.15.42:0/60042976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 msgr2=0x55721eda3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.887+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda3e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00f3c0 tx=0x7f01ec009500 comp rx=0 tx=0).stop 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.887+0000 7f01fbf6c740 1 -- 172.21.15.42:0/60042976 shutdown_connections 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda9750 0x55721eda3e40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0b70 0x55721eda3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/60042976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda0250 0x55721eda8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 -- 172.21.15.42:0/60042976 >> 172.21.15.42:0/60042976 conn(0x55721ed9a980 msgr2=0x55721edc8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 -- 172.21.15.42:0/60042976 shutdown_connections 2024-03-12T04:17:13.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 -- 172.21.15.42:0/60042976 wait complete. 2024-03-12T04:17:13.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 -- start start 2024-03-12T04:17:13.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.888+0000 7f01fbf6c740 1 Processor -- start 2024-03-12T04:17:13.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.889+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0250 0x55721edb0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.889+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda0b70 0x55721edaeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.889+0000 7f01fbf6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda9750 0x55721edaf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.889+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0250 0x55721edb0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.889+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0250 0x55721edb0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41888/0 (socket says 172.21.15.42:41888) 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.889+0000 7f01f27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda0b70 0x55721edaeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01f2ffd640 1 -- 172.21.15.42:0/1010226409 learned_addr learned my addr 172.21.15.42:0/1010226409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721edaf9e0 con 0x55721eda0250 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01f37fe640 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda9750 0x55721edaf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721edafb50 con 0x55721eda9750 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721ee183b0 con 0x55721eda0b70 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01fbf6c740 10 client.0 _ll_register_callbacks cb 0x7f01e40962f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01f27fc640 1 -- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda9750 msgr2=0x55721edaf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01f27fc640 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda9750 0x55721edaf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01f27fc640 1 -- 172.21.15.42:0/1010226409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0250 msgr2=0x55721edb0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01f27fc640 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0250 0x55721edb0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.890+0000 7f01f27fc640 1 -- 172.21.15.42:0/1010226409 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec0121d0 con 0x55721eda0b70 2024-03-12T04:17:13.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01f27fc640 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda0b70 0x55721edaeda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f01f4073870 tx=0x7f01e8004b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3444a27e3b2feb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:13.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:17:13.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01d6ff5640 1 -- 172.21.15.42:0/1010226409 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8046040 con 0x55721eda0b70 2024-03-12T04:17:13.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01d6ff5640 1 -- 172.21.15.42:0/1010226409 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f01e8048d70 con 0x55721eda0b70 2024-03-12T04:17:13.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01e40a2770 con 0x55721eda0b70 2024-03-12T04:17:13.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01d6ff5640 1 -- 172.21.15.42:0/1010226409 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e8004040 con 0x55721eda0b70 2024-03-12T04:17:13.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01fbf6c740 10 client.25528 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:13.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.891+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55721ecc88c0 con 0x55721eda0b70 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01d6ff5640 1 -- 172.21.15.42:0/1010226409 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01e40a2770 con 0x55721eda0b70 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01d6ff5640 1 -- 172.21.15.42:0/1010226409 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f01e803d040 con 0x55721eda0b70 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01fbf6c740 10 client.25528 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01fbf6c740 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01fbf6c740 20 client.25528 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721ee19b60 0x55721ee1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01fbf6c740 4 client.25528 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.892+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55721ea44720 con 0x55721ee19b60 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.893+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721ee19b60 0x55721ee1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.893+0000 7f01f2ffd640 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721ee19b60 0x55721ee1bf80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde1f2a0827c1a73 server_cookie=4cc647db28172ba7 in_seq=0 out_seq=0 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.894+0000 7f01d6ff5640 10 client.25528 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.894+0000 7f01d6ff5640 1 -- 172.21.15.42:0/1010226409 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55721ea44720 con 0x55721ee19b60 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.894+0000 7f01d6ff5640 10 client.25528 handle_command_reply: tid=0 2024-03-12T04:17:13.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.894+0000 7f01fbf6c740 1 client.25528 shutdown 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.894+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda0b70 msgr2=0x55721edaeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.894+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda0b70 0x55721edaeda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f01f4073870 tx=0x7f01e8004b70 comp rx=0 tx=0).stop 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 shutdown_connections 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55721ee19b60 0x55721ee1bf80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55721eda9750 0x55721edaf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01d6ff5640 0 client.25528 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55721eda0b70 0x55721edaeda0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 --2- 172.21.15.42:0/1010226409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55721eda0250 0x55721edb0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 >> 172.21.15.42:0/1010226409 conn(0x55721ed9a980 msgr2=0x55721eda4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 shutdown_connections 2024-03-12T04:17:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 1 -- 172.21.15.42:0/1010226409 wait complete. 2024-03-12T04:17:13.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.895+0000 7f01fbf6c740 20 client.25528 trim_cache size 0 max 16384 2024-03-12T04:17:13.900 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f01dc0a1c20 msgr2=0x7f01dc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f01dc0a1c20 0x7f01dc0a40e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005fd0 tx=0x7f01ec013ae0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40ffcf0 msgr2=0x7f01f4197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40ffcf0 0x7f01f4197540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f01e400d950 tx=0x7f01e400de10 comp rx=0 tx=0).stop 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 shutdown_connections 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f01dc0a1c20 0x7f01dc0a40e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01f419be00 0x7f01f419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4104b70 0x7f01f4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 --2- 172.21.15.42:0/3557419755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40ffcf0 0x7f01f4197540 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f01e400d950 tx=0x7f01e400de10 comp rx=0 tx=0).stop 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.896+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 >> 172.21.15.42:0/3557419755 conn(0x7f01f40752e0 msgr2=0x7f01f4103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.897+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 shutdown_connections 2024-03-12T04:17:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:13.897+0000 7f01f92c0640 1 -- 172.21.15.42:0/3557419755 wait complete. 2024-03-12T04:17:14.916 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:17:14.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:17:14.916 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:17:15.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.067+0000 7f7638664640 1 -- 172.21.15.42:0/805789960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7630068fa0 msgr2=0x7f7630105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.067+0000 7f7638664640 1 --2- 172.21.15.42:0/805789960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7630068fa0 0x7f7630105de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7620009930 tx=0x7f762002bc30 comp rx=0 tx=0).stop 2024-03-12T04:17:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.067+0000 7f7638664640 1 -- 172.21.15.42:0/805789960 shutdown_connections 2024-03-12T04:17:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.067+0000 7f7638664640 1 --2- 172.21.15.42:0/805789960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76301064c0 0x7f763010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.067+0000 7f7638664640 1 --2- 172.21.15.42:0/805789960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7630068fa0 0x7f7630105de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.067+0000 7f7638664640 1 --2- 172.21.15.42:0/805789960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76300685f0 0x7f76300689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.067+0000 7f7638664640 1 -- 172.21.15.42:0/805789960 >> 172.21.15.42:0/805789960 conn(0x7f7630077180 msgr2=0x7f7630077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.068+0000 7f7638664640 1 -- 172.21.15.42:0/805789960 shutdown_connections 2024-03-12T04:17:15.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.068+0000 7f7638664640 1 -- 172.21.15.42:0/805789960 wait complete. 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.068+0000 7f7638664640 1 Processor -- start 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.068+0000 7f7638664640 1 -- start start 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7638664640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76300685f0 0x7f763010db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7638664640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76301064c0 0x7f7630109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7638664640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763010a150 0x7f763010a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7638664640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763010e330 con 0x7f76300685f0 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7638664640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763010e470 con 0x7f76301064c0 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7638664640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763006ac20 con 0x7f763010a150 2024-03-12T04:17:15.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76300685f0 0x7f763010db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76300685f0 0x7f763010db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41902/0 (socket says 172.21.15.42:41902) 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 -- 172.21.15.42:0/3321888859 learned_addr learned my addr 172.21.15.42:0/3321888859 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7635bff640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76301064c0 0x7f7630109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636c01640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763010a150 0x7f763010a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 -- 172.21.15.42:0/3321888859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763010a150 msgr2=0x7f763010a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763010a150 0x7f763010a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 -- 172.21.15.42:0/3321888859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76301064c0 msgr2=0x7f7630109c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76301064c0 0x7f7630109c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.069+0000 7f7636400640 1 -- 172.21.15.42:0/3321888859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620009560 con 0x7f76300685f0 2024-03-12T04:17:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.070+0000 7f7636400640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76300685f0 0x7f763010db80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f762c00e9f0 tx=0x7f762c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.070+0000 7f76277fe640 1 -- 172.21.15.42:0/3321888859 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c009720 con 0x7f76300685f0 2024-03-12T04:17:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.070+0000 7f76277fe640 1 -- 172.21.15.42:0/3321888859 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f762c00cd80 con 0x7f76300685f0 2024-03-12T04:17:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.070+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763006af00 con 0x7f76300685f0 2024-03-12T04:17:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.070+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763006b3e0 con 0x7f76300685f0 2024-03-12T04:17:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.070+0000 7f76277fe640 1 -- 172.21.15.42:0/3321888859 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c010430 con 0x7f76300685f0 2024-03-12T04:17:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 Processor -- start 2024-03-12T04:17:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f76277fe640 1 -- 172.21.15.42:0/3321888859 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f762c010690 con 0x7f76300685f0 2024-03-12T04:17:15.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 -- start start 2024-03-12T04:17:15.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d23a0 0x55f0b09d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d2d50 0x55f0b09db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0940f90 con 0x55f0b09d23a0 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b093e4c0 con 0x55f0b09db8d0 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.075+0000 7f763b234740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0939ca0 con 0x55f0b09d2d50 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7636c01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d23a0 0x55f0b09d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09ddd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56426/0 (socket says 172.21.15.42:56426) 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 -- 172.21.15.42:0/2537708032 learned_addr learned my addr 172.21.15.42:0/2537708032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7636400640 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d2d50 0x55f0b09db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 -- 172.21.15.42:0/2537708032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d2d50 msgr2=0x55f0b09db300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d2d50 0x55f0b09db300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 -- 172.21.15.42:0/2537708032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d23a0 msgr2=0x55f0b09d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d23a0 0x55f0b09d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 -- 172.21.15.42:0/2537708032 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0b08fc0b0 con 0x55f0b09db8d0 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f76277fe640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f761c0a1cf0 0x7f761c0a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7636c01640 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d23a0 0x55f0b09d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f76277fe640 1 -- 172.21.15.42:0/3321888859 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f762c014070 con 0x7f76300685f0 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7636c01640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f761c0a1cf0 0x7f761c0a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.076+0000 7f7635bff640 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09ddd20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7630102290 tx=0x7f7620000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1180713fc914d01b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.077+0000 7f7624bfe640 1 -- 172.21.15.42:0/2537708032 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620005220 con 0x55f0b09db8d0 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.077+0000 7f7636c01640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f761c0a1cf0 0x7f761c0a41b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7628005fd0 tx=0x7f7628009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.077+0000 7f7624bfe640 1 -- 172.21.15.42:0/2537708032 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7620004dc0 con 0x55f0b09db8d0 2024-03-12T04:17:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.077+0000 7f7624bfe640 1 -- 172.21.15.42:0/2537708032 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620004930 con 0x55f0b09db8d0 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.078+0000 7f763b234740 1 -- 172.21.15.42:0/2537708032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 msgr2=0x55f0b09ddd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.078+0000 7f763b234740 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09ddd20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7630102290 tx=0x7f7620000bc0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.078+0000 7f763b234740 1 -- 172.21.15.42:0/2537708032 shutdown_connections 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.078+0000 7f763b234740 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09ddd20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.078+0000 7f763b234740 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d2d50 0x55f0b09db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.078+0000 7f763b234740 1 --2- 172.21.15.42:0/2537708032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d23a0 0x55f0b09d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.078+0000 7f763b234740 1 -- 172.21.15.42:0/2537708032 >> 172.21.15.42:0/2537708032 conn(0x55f0b09cc860 msgr2=0x55f0b09ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.079+0000 7f763b234740 1 -- 172.21.15.42:0/2537708032 shutdown_connections 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.079+0000 7f763b234740 1 -- 172.21.15.42:0/2537708032 wait complete. 2024-03-12T04:17:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.079+0000 7f763b234740 1 -- start start 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.079+0000 7f763b234740 1 Processor -- start 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.079+0000 7f763b234740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d23a0 0x55f0b09d8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.079+0000 7f763b234740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.079+0000 7f763b234740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b09d92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f763b234740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b08def40 con 0x55f0b09d2d50 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f763b234740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b060d4b0 con 0x55f0b09d23a0 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f763b234740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b08bf670 con 0x55f0b09db8d0 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f763b234740 10 client.0 _ll_register_callbacks cb 0x55f0b092cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7636400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d23a0 0x55f0b09d8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09d8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41924/0 (socket says 172.21.15.42:41924) 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 -- 172.21.15.42:0/1385283833 learned_addr learned my addr 172.21.15.42:0/1385283833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7636c01640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b09d92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 -- 172.21.15.42:0/1385283833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 msgr2=0x55f0b09d92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b09d92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 -- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d23a0 msgr2=0x55f0b09d8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d23a0 0x55f0b09d8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7635bff640 1 -- 172.21.15.42:0/1385283833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763010e880 con 0x55f0b09d2d50 2024-03-12T04:17:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7636400640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d23a0 0x55f0b09d8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.080+0000 7f7636c01640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b09d92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f7635bff640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09d8d60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7620009900 tx=0x7f762002dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4621701762077a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f7619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f7619ffb640 1 -- 172.21.15.42:0/1385283833 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620004450 con 0x55f0b09d2d50 2024-03-12T04:17:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f7619ffb640 1 -- 172.21.15.42:0/1385283833 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7620004dc0 con 0x55f0b09d2d50 2024-03-12T04:17:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0b0913f40 con 0x55f0b09d2d50 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f7619ffb640 1 -- 172.21.15.42:0/1385283833 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7620005020 con 0x55f0b09d2d50 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f763b234740 10 client.16155 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0b0938320 con 0x55f0b09d2d50 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.081+0000 7f7619ffb640 1 -- 172.21.15.42:0/1385283833 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f0b0913f40 con 0x55f0b09d2d50 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f7619ffb640 1 -- 172.21.15.42:0/1385283833 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f762004e030 con 0x55f0b09d2d50 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f763b234740 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f763b234740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f763b234740 20 client.16155 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f763b234740 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b09ebaf0 0x55f0b09edf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f763b234740 4 client.16155 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0b0930310 con 0x55f0b09ebaf0 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.082+0000 7f7636400640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b09ebaf0 0x55f0b09edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f7636400640 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b09ebaf0 0x55f0b09edf10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c111a4cfd68b85a server_cookie=8559475e3b560f7f in_seq=0 out_seq=0 2024-03-12T04:17:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f7619ffb640 10 client.16155 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f7619ffb640 1 -- 172.21.15.42:0/1385283833 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0b0930310 con 0x55f0b09ebaf0 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f7619ffb640 10 client.16155 handle_command_reply: tid=0 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 client.16155 shutdown 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 msgr2=0x55f0b09d8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09d8d60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7620009900 tx=0x7f762002dcd0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 shutdown_connections 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b09ebaf0 0x55f0b09edf10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b09d92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f7619ffb640 0 client.16155 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09d8d60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 --2- 172.21.15.42:0/1385283833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d23a0 0x55f0b09d8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.083+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 >> 172.21.15.42:0/1385283833 conn(0x55f0b09cc860 msgr2=0x55f0b09ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.084+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 shutdown_connections 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.084+0000 7f763b234740 1 -- 172.21.15.42:0/1385283833 wait complete. 2024-03-12T04:17:15.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.084+0000 7f763b234740 20 client.16155 trim_cache size 0 max 16384 2024-03-12T04:17:15.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.113+0000 7f763b234740 1 Processor -- start 2024-03-12T04:17:15.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.113+0000 7f763b234740 1 -- start start 2024-03-12T04:17:15.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.113+0000 7f763b234740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d23a0 0x55f0b09d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.113+0000 7f763b234740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.113+0000 7f763b234740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.113+0000 7f763b234740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b09cc3c0 con 0x55f0b09d2d50 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f763b234740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b09c8170 con 0x55f0b09db8d0 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f763b234740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b09f3170 con 0x55f0b09d23a0 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d23a0 0x55f0b09d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d23a0 0x55f0b09d90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59732/0 (socket says 172.21.15.42:59732) 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 -- 172.21.15.42:0/2978691023 learned_addr learned my addr 172.21.15.42:0/2978691023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636c01640 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7635bff640 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 -- 172.21.15.42:0/2978691023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 msgr2=0x55f0b09e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 -- 172.21.15.42:0/2978691023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 msgr2=0x55f0b09cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 -- 172.21.15.42:0/2978691023 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0b08d4c00 con 0x55f0b09d23a0 2024-03-12T04:17:15.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.114+0000 7f7636400640 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d23a0 0x55f0b09d90c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f762c097b20 tx=0x7f762c0d2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5f4314dede865e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.115+0000 7f761bfff640 1 -- 172.21.15.42:0/2978691023 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c09b6c0 con 0x55f0b09d23a0 2024-03-12T04:17:15.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.115+0000 7f761bfff640 1 -- 172.21.15.42:0/2978691023 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f762c09f810 con 0x55f0b09d23a0 2024-03-12T04:17:15.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.115+0000 7f761bfff640 1 -- 172.21.15.42:0/2978691023 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c0ba0b0 con 0x55f0b09d23a0 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 -- 172.21.15.42:0/2978691023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d23a0 msgr2=0x55f0b09d90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d23a0 0x55f0b09d90c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f762c097b20 tx=0x7f762c0d2040 comp rx=0 tx=0).stop 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 -- 172.21.15.42:0/2978691023 shutdown_connections 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09db8d0 0x55f0b09e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09d2d50 0x55f0b09cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 --2- 172.21.15.42:0/2978691023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09d23a0 0x55f0b09d90c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 -- 172.21.15.42:0/2978691023 >> 172.21.15.42:0/2978691023 conn(0x55f0b09cc860 msgr2=0x55f0b09cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 -- 172.21.15.42:0/2978691023 shutdown_connections 2024-03-12T04:17:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.116+0000 7f763b234740 1 -- 172.21.15.42:0/2978691023 wait complete. 2024-03-12T04:17:15.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 -- start start 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 Processor -- start 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d2d50 0x55f0b0a3b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b0a3b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09e1a10 0x55f0b09e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b09e2590 con 0x55f0b09e1a10 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b09e2700 con 0x55f0b09d2d50 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b09e28c0 con 0x55f0b09db8d0 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f763b234740 10 client.0 _ll_register_callbacks cb 0x7f762c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f7636400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d2d50 0x55f0b0a3b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f7636c01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09e1a10 0x55f0b09e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f7636400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d2d50 0x55f0b0a3b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56452/0 (socket says 172.21.15.42:56452) 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f7636400640 1 -- 172.21.15.42:0/1807557576 learned_addr learned my addr 172.21.15.42:0/1807557576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f7635bff640 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b0a3b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f7636400640 1 -- 172.21.15.42:0/1807557576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 msgr2=0x55f0b0a3b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.117+0000 7f7636400640 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b0a3b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7636400640 1 -- 172.21.15.42:0/1807557576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09e1a10 msgr2=0x55f0b09e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7636400640 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09e1a10 0x55f0b09e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7636400640 1 -- 172.21.15.42:0/1807557576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763006b3e0 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7636400640 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d2d50 0x55f0b0a3b390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f762c09b3c0 tx=0x7f762c0cdc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be7e760dc703560 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7619ffb640 1 -- 172.21.15.42:0/1807557576 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c0a3e60 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7619ffb640 1 -- 172.21.15.42:0/1807557576 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f762c09b6c0 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0b095f900 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7619ffb640 1 -- 172.21.15.42:0/1807557576 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f762c0046c0 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f763b234740 10 client.25661 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0b08fa910 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.118+0000 7f7619ffb640 1 -- 172.21.15.42:0/1807557576 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f762c0af080 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f7619ffb640 1 -- 172.21.15.42:0/1807557576 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f762c01f060 con 0x55f0b09d2d50 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f763b234740 10 client.25661 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f763b234740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f763b234740 20 client.25661 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f763b234740 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b0a44170 0x55f0b0a46560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f763b234740 4 client.25661 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:17:15.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f0b0639860 con 0x55f0b0a44170 2024-03-12T04:17:15.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f7635bff640 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b0a44170 0x55f0b0a46560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f7635bff640 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b0a44170 0x55f0b0a46560 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f85ef7bc64461e4 server_cookie=4017d2e337ce66f5 in_seq=0 out_seq=0 2024-03-12T04:17:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.119+0000 7f7619ffb640 10 client.25661 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f7619ffb640 1 -- 172.21.15.42:0/1807557576 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f0b0639860 con 0x55f0b0a44170 2024-03-12T04:17:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f7619ffb640 10 client.25661 handle_command_reply: tid=0 2024-03-12T04:17:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 client.25661 shutdown 2024-03-12T04:17:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d2d50 msgr2=0x55f0b0a3b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d2d50 0x55f0b0a3b390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f762c09b3c0 tx=0x7f762c0cdc80 comp rx=0 tx=0).stop 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 shutdown_connections 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f0b0a44170 0x55f0b0a46560 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b09e1a10 0x55f0b09e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f7619ffb640 0 client.25661 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0b09db8d0 0x55f0b0a3b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 --2- 172.21.15.42:0/1807557576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f0b09d2d50 0x55f0b0a3b390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.121+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 >> 172.21.15.42:0/1807557576 conn(0x55f0b09cc860 msgr2=0x55f0b09cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.122+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 shutdown_connections 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.122+0000 7f763b234740 1 -- 172.21.15.42:0/1807557576 wait complete. 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.122+0000 7f763b234740 20 client.25661 trim_cache size 0 max 16384 2024-03-12T04:17:15.126 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:17:15.127 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:17:15.127 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "61a7bffa-77e8-4338-98a8-4515e94538c7", 2024-03-12T04:17:15.127 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:17:15.127 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.122+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f761c0a1cf0 msgr2=0x7f761c0a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.122+0000 7f7638664640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f761c0a1cf0 0x7f761c0a41b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7628005fd0 tx=0x7f7628009500 comp rx=0 tx=0).stop 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.122+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76300685f0 msgr2=0x7f763010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.122+0000 7f7638664640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76300685f0 0x7f763010db80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f762c00e9f0 tx=0x7f762c00edf0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.123+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 shutdown_connections 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.123+0000 7f7638664640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f761c0a1cf0 0x7f761c0a41b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.123+0000 7f7638664640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763010a150 0x7f763010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.123+0000 7f7638664640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76301064c0 0x7f7630109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.123+0000 7f7638664640 1 --2- 172.21.15.42:0/3321888859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76300685f0 0x7f763010db80 secure :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f762c00e9f0 tx=0x7f762c00edf0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.123+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 >> 172.21.15.42:0/3321888859 conn(0x7f7630077180 msgr2=0x7f76300fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.125+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 shutdown_connections 2024-03-12T04:17:15.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.125+0000 7f7638664640 1 -- 172.21.15.42:0/3321888859 wait complete. 2024-03-12T04:17:15.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:17:15.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 -- 172.21.15.42:0/1481317029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 msgr2=0x7f6038104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1481317029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 0x7f6038104f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6020009900 tx=0x7f602002bcc0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 -- 172.21.15.42:0/1481317029 shutdown_connections 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1481317029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60381054b0 0x7f6038105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1481317029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 0x7f6038104f70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1481317029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 0x7f6038100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 -- 172.21.15.42:0/1481317029 >> 172.21.15.42:0/1481317029 conn(0x7f6038077230 msgr2=0x7f6038077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 -- 172.21.15.42:0/1481317029 shutdown_connections 2024-03-12T04:17:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.301+0000 7f603cf4b640 1 -- 172.21.15.42:0/1481317029 wait complete. 2024-03-12T04:17:15.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 Processor -- start 2024-03-12T04:17:15.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 -- start start 2024-03-12T04:17:15.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 0x7f60381a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 0x7f60381a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60381054b0 0x7f603819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819e090 con 0x7f6038100190 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819e200 con 0x7f60381054b0 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603cf4b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819e3c0 con 0x7f6038100ab0 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.302+0000 7f603659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 0x7f60381a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f603659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 0x7f60381a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41962/0 (socket says 172.21.15.42:41962) 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6036d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60381054b0 0x7f603819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f603659c640 1 -- 172.21.15.42:0/1366775739 learned_addr learned my addr 172.21.15.42:0/1366775739 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6035d9b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 0x7f60381a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6036d9d640 1 -- 172.21.15.42:0/1366775739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 msgr2=0x7f60381a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6036d9d640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 0x7f60381a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6036d9d640 1 -- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 msgr2=0x7f60381a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6036d9d640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 0x7f60381a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6036d9d640 1 -- 172.21.15.42:0/1366775739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009560 con 0x7f60381054b0 2024-03-12T04:17:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f603659c640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 0x7f60381a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.303+0000 7f6036d9d640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60381054b0 0x7f603819da40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f603000e970 tx=0x7f603000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.304+0000 7f602f7fe640 1 -- 172.21.15.42:0/1366775739 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030009620 con 0x7f60381054b0 2024-03-12T04:17:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.304+0000 7f602f7fe640 1 -- 172.21.15.42:0/1366775739 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60300044d0 con 0x7f60381054b0 2024-03-12T04:17:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.304+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603819e6a0 con 0x7f60381054b0 2024-03-12T04:17:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.304+0000 7f602f7fe640 1 -- 172.21.15.42:0/1366775739 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030004800 con 0x7f60381054b0 2024-03-12T04:17:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.304+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60381aa930 con 0x7f60381054b0 2024-03-12T04:17:15.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.308+0000 7f602f7fe640 1 -- 172.21.15.42:0/1366775739 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f60300049a0 con 0x7f60381054b0 2024-03-12T04:17:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 Processor -- start 2024-03-12T04:17:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 -- start start 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f602f7fe640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60240a1ca0 0x7f60240a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a271f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2728270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f6035d9b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60240a1ca0 0x7f60240a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a272aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a268e280 con 0x55a3a27287b0 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a268b7b0 con 0x55a3a271fbd0 2024-03-12T04:17:15.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.309+0000 7f603fbf5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a2686f90 con 0x55a3a271f2b0 2024-03-12T04:17:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a271f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a271f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59770/0 (socket says 172.21.15.42:59770) 2024-03-12T04:17:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 -- 172.21.15.42:0/3121046807 learned_addr learned my addr 172.21.15.42:0/3121046807 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f602f7fe640 1 -- 172.21.15.42:0/1366775739 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6030014070 con 0x7f60381054b0 2024-03-12T04:17:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f6035d9b640 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2728270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 -- 172.21.15.42:0/3121046807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 msgr2=0x55a3a2728270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2728270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 -- 172.21.15.42:0/3121046807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 msgr2=0x55a3a272aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a272aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 -- 172.21.15.42:0/3121046807 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a2645570 con 0x55a3a271f2b0 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f6035d9b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60240a1ca0 0x7f60240a4160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6020005ab0 tx=0x7f6020004120 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.310+0000 7f603659c640 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a271f690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f602800f7d0 tx=0x7f602800fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=652c29a9581735df server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.311+0000 7f602cff9640 1 -- 172.21.15.42:0/3121046807 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800fee0 con 0x55a3a271f2b0 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.311+0000 7f602cff9640 1 -- 172.21.15.42:0/3121046807 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6028004d20 con 0x55a3a271f2b0 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.311+0000 7f602cff9640 1 -- 172.21.15.42:0/3121046807 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028005980 con 0x55a3a271f2b0 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 -- 172.21.15.42:0/3121046807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 msgr2=0x55a3a271f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a271f690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f602800f7d0 tx=0x7f602800fc90 comp rx=0 tx=0).stop 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 -- 172.21.15.42:0/3121046807 shutdown_connections 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a272aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2728270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 --2- 172.21.15.42:0/3121046807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a271f690 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 -- 172.21.15.42:0/3121046807 >> 172.21.15.42:0/3121046807 conn(0x55a3a2719990 msgr2=0x55a3a2719d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 -- 172.21.15.42:0/3121046807 shutdown_connections 2024-03-12T04:17:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 -- 172.21.15.42:0/3121046807 wait complete. 2024-03-12T04:17:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 -- start start 2024-03-12T04:17:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.312+0000 7f603fbf5740 1 Processor -- start 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a271f2b0 0x55a3a2725650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a2725b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a27287b0 0x55a3a27260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603fbf5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a235a430 con 0x55a3a271f2b0 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603fbf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a2316e00 con 0x55a3a27287b0 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603fbf5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a260bfd0 con 0x55a3a271fbd0 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603fbf5740 10 client.0 _ll_register_callbacks cb 0x55a3a2661fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a271f2b0 0x55a3a2725650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a271f2b0 0x55a3a2725650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41972/0 (socket says 172.21.15.42:41972) 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f603659c640 1 -- 172.21.15.42:0/309607256 learned_addr learned my addr 172.21.15.42:0/309607256 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f6035d9b640 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a2725b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.313+0000 7f6036d9d640 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a27287b0 0x55a3a27260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.318 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:15 smithi042 ceph-mon[32260]: pgmap v64: 97 pgs: 97 active+clean; 395 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.14k op/s 2024-03-12T04:17:15.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f6035d9b640 1 -- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a27287b0 msgr2=0x55a3a27260d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f6035d9b640 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a27287b0 0x55a3a27260d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f6035d9b640 1 -- 172.21.15.42:0/309607256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a271f2b0 msgr2=0x55a3a2725650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f6035d9b640 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a271f2b0 0x55a3a2725650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f6035d9b640 1 -- 172.21.15.42:0/309607256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602800f400 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f6035d9b640 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a2725b90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f602002c200 tx=0x7f602002bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e36df1b10095a29 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f601dffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f601dffb640 1 -- 172.21.15.42:0/309607256 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602002ce50 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f601dffb640 1 -- 172.21.15.42:0/309607256 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6020004fd0 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.314+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3a23867e0 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f601dffb640 1 -- 172.21.15.42:0/309607256 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6020005190 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 10 client.25558 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a3a2685610 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f601dffb640 1 -- 172.21.15.42:0/309607256 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a3a23867e0 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f601dffb640 1 -- 172.21.15.42:0/309607256 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f602003c030 con 0x55a3a271fbd0 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 10 client.25558 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 20 client.25558 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a27387d0 0x55a3a273abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 4 client.25558 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.315+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3a23c3720 con 0x55a3a27387d0 2024-03-12T04:17:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.316+0000 7f603659c640 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a27387d0 0x55a3a273abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.317+0000 7f603659c640 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a27387d0 0x55a3a273abf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f828ad118c55cc3e server_cookie=a80476e05a21b648 in_seq=0 out_seq=0 2024-03-12T04:17:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.317+0000 7f601dffb640 10 client.25558 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.317+0000 7f601dffb640 1 -- 172.21.15.42:0/309607256 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3a23c3720 con 0x55a3a27387d0 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f601dffb640 10 client.25558 handle_command_reply: tid=0 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 client.25558 shutdown 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 msgr2=0x55a3a2725b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a2725b90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f602002c200 tx=0x7f602002bcc0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 shutdown_connections 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a27387d0 0x55a3a273abf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a27287b0 0x55a3a27260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f601dffb640 0 client.25558 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a2725b90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 --2- 172.21.15.42:0/309607256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a271f2b0 0x55a3a2725650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 >> 172.21.15.42:0/309607256 conn(0x55a3a2719990 msgr2=0x55a3a271b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 shutdown_connections 2024-03-12T04:17:15.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 1 -- 172.21.15.42:0/309607256 wait complete. 2024-03-12T04:17:15.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.318+0000 7f603fbf5740 20 client.25558 trim_cache size 0 max 16384 2024-03-12T04:17:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.346+0000 7f603fbf5740 1 Processor -- start 2024-03-12T04:17:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.346+0000 7f603fbf5740 1 -- start start 2024-03-12T04:17:15.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.346+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a2727d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2724630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a27228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f6035d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2724630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f6035d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2724630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56504/0 (socket says 172.21.15.42:56504) 2024-03-12T04:17:15.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f6035d9b640 1 -- 172.21.15.42:0/1290489393 learned_addr learned my addr 172.21.15.42:0/1290489393 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a27150f0 con 0x55a3a27287b0 2024-03-12T04:17:15.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a271ecf0 con 0x55a3a271fbd0 2024-03-12T04:17:15.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.347+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a271a790 con 0x55a3a271f2b0 2024-03-12T04:17:15.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.348+0000 7f6036d9d640 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a27228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.348+0000 7f603659c640 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a2727d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.349+0000 7f6035d9b640 1 -- 172.21.15.42:0/1290489393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 msgr2=0x55a3a2727d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.349+0000 7f6035d9b640 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a2727d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.349+0000 7f6035d9b640 1 -- 172.21.15.42:0/1290489393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 msgr2=0x55a3a27228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.349+0000 7f6035d9b640 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a27228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.349+0000 7f6035d9b640 1 -- 172.21.15.42:0/1290489393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a2621ac0 con 0x55a3a271fbd0 2024-03-12T04:17:15.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.349+0000 7f6035d9b640 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2724630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6020005df0 tx=0x7f6020004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ce23b0fdda904b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.349+0000 7f601ffff640 1 -- 172.21.15.42:0/1290489393 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200053e0 con 0x55a3a271fbd0 2024-03-12T04:17:15.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f601ffff640 1 -- 172.21.15.42:0/1290489393 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f602004d520 con 0x55a3a271fbd0 2024-03-12T04:17:15.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f601ffff640 1 -- 172.21.15.42:0/1290489393 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602004d780 con 0x55a3a271fbd0 2024-03-12T04:17:15.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 msgr2=0x55a3a2724630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2724630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6020005df0 tx=0x7f6020004ab0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 shutdown_connections 2024-03-12T04:17:15.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a27228b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271fbd0 0x55a3a2724630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 --2- 172.21.15.42:0/1290489393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271f2b0 0x55a3a2727d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 >> 172.21.15.42:0/1290489393 conn(0x55a3a2719990 msgr2=0x55a3a2723f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 shutdown_connections 2024-03-12T04:17:15.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.350+0000 7f603fbf5740 1 -- 172.21.15.42:0/1290489393 wait complete. 2024-03-12T04:17:15.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 -- start start 2024-03-12T04:17:15.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 Processor -- start 2024-03-12T04:17:15.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271f2b0 0x55a3a272fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a272df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a272e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a272ed10 con 0x55a3a27287b0 2024-03-12T04:17:15.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a278ed20 con 0x55a3a271f2b0 2024-03-12T04:17:15.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a278ee60 con 0x55a3a271fbd0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.351+0000 7f603fbf5740 10 client.0 _ll_register_callbacks cb 0x7f6028007e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271f2b0 0x55a3a272fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271f2b0 0x55a3a272fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:56516/0 (socket says 172.21.15.42:56516) 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 -- 172.21.15.42:0/2919666865 learned_addr learned my addr 172.21.15.42:0/2919666865 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f6036d9d640 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a272e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f6035d9b640 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a272df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 -- 172.21.15.42:0/2919666865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 msgr2=0x55a3a272df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a272df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 -- 172.21.15.42:0/2919666865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 msgr2=0x55a3a272e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a272e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 -- 172.21.15.42:0/2919666865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3a2685610 con 0x55a3a271f2b0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603659c640 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271f2b0 0x55a3a272fb80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f602800f7d0 tx=0x7f60280050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0cffe222b4f887 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f601dffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f601dffb640 1 -- 172.21.15.42:0/2919666865 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602801d040 con 0x55a3a271f2b0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6028004d20 con 0x55a3a271f2b0 2024-03-12T04:17:15.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.352+0000 7f601dffb640 1 -- 172.21.15.42:0/2919666865 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f60280054f0 con 0x55a3a271f2b0 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f601dffb640 1 -- 172.21.15.42:0/2919666865 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f602800fee0 con 0x55a3a271f2b0 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 10 client.25679 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a3a26476a0 con 0x55a3a271f2b0 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f601dffb640 1 -- 172.21.15.42:0/2919666865 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6028004d20 con 0x55a3a271f2b0 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f601dffb640 1 -- 172.21.15.42:0/2919666865 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60280265a0 con 0x55a3a271f2b0 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 10 client.25679 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 20 client.25679 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a2790640 0x55a3a2792a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 4 client.25679 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a3a2412890 con 0x55a3a2790640 2024-03-12T04:17:15.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.353+0000 7f6035d9b640 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a2790640 0x55a3a2792a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.354+0000 7f6035d9b640 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a2790640 0x55a3a2792a60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a54b09032d5ba67 server_cookie=e79a69ad76ba145b in_seq=0 out_seq=0 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.354+0000 7f601dffb640 10 client.25679 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.354+0000 7f601dffb640 1 -- 172.21.15.42:0/2919666865 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a3a2412890 con 0x55a3a2790640 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.354+0000 7f601dffb640 10 client.25679 handle_command_reply: tid=0 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 client.25679 shutdown 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271f2b0 msgr2=0x55a3a272fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271f2b0 0x55a3a272fb80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f602800f7d0 tx=0x7f60280050c0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 shutdown_connections 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55a3a2790640 0x55a3a2792a60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3a27287b0 0x55a3a272e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3a271fbd0 0x55a3a272df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 --2- 172.21.15.42:0/2919666865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3a271f2b0 0x55a3a272fb80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 >> 172.21.15.42:0/2919666865 conn(0x55a3a2719990 msgr2=0x55a3a27247b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f601dffb640 0 client.25679 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 shutdown_connections 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 1 -- 172.21.15.42:0/2919666865 wait complete. 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.355+0000 7f603fbf5740 20 client.25679 trim_cache size 0 max 16384 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: "61a7bffa-77e8-4338-98a8-4515e94538c7": { 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "61a7bffa-77e8-4338-98a8-4515e94538c7", 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:17:15.361 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:17:15.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60240a1ca0 msgr2=0x7f60240a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60240a1ca0 0x7f60240a4160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6020005ab0 tx=0x7f6020004120 comp rx=0 tx=0).stop 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60381054b0 msgr2=0x7f603819da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60381054b0 0x7f603819da40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f603000e970 tx=0x7f603000ee30 comp rx=0 tx=0).stop 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 shutdown_connections 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f60240a1ca0 0x7f60240a4160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60381054b0 0x7f603819da40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6038100ab0 0x7f60381a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 --2- 172.21.15.42:0/1366775739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6038100190 0x7f60381a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.356+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 >> 172.21.15.42:0/1366775739 conn(0x7f6038077230 msgr2=0x7f6038102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.357+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 shutdown_connections 2024-03-12T04:17:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:15.357+0000 7f603cf4b640 1 -- 172.21.15.42:0/1366775739 wait complete. 2024-03-12T04:17:15.375 INFO:tasks.cephfs.filesystem:scrub status for tag:61a7bffa-77e8-4338-98a8-4515e94538c7 - {'path': '/', 'tag': '61a7bffa-77e8-4338-98a8-4515e94538c7', 'options': 'recursive,force'} 2024-03-12T04:17:15.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:15 smithi119 ceph-mon[36119]: pgmap v64: 97 pgs: 97 active+clean; 395 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.14k op/s 2024-03-12T04:17:15.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:15 smithi136 ceph-mon[35571]: pgmap v64: 97 pgs: 97 active+clean; 395 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.14k op/s 2024-03-12T04:17:16.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:16 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:17:16.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:16 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:16.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:16 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:17:16.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:16 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:16 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:17:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:16 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:16 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:17:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:16 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:16.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:16 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:17:16.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:16 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:16.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:16 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:17:16.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:16 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:17.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:17 smithi119 ceph-mon[36119]: pgmap v65: 97 pgs: 97 active+clean; 395 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.40k op/s 2024-03-12T04:17:17.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:17 smithi136 ceph-mon[35571]: pgmap v65: 97 pgs: 97 active+clean; 395 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.40k op/s 2024-03-12T04:17:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:17 smithi042 ceph-mon[32260]: pgmap v65: 97 pgs: 97 active+clean; 395 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.40k op/s 2024-03-12T04:17:18.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:18 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:17:18.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:18 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:18.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:18 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:18.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:18 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:17:18.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:18 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:18.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:18 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:18.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:18 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:17:18.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:18 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:18.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:18 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:19.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:19 smithi119 ceph-mon[36119]: pgmap v66: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.04k op/s 2024-03-12T04:17:19.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:19 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:17:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:19 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:17:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:19 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:19 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:17:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:19 smithi136 ceph-mon[35571]: pgmap v66: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.04k op/s 2024-03-12T04:17:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:19 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:17:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:19 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:17:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:19 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:19 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:17:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:19 smithi042 ceph-mon[32260]: pgmap v66: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.04k op/s 2024-03-12T04:17:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:19 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:17:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:19 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:17:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:19 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:17:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:19 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:17:20.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:20 smithi119 ceph-mon[36119]: pgmap v67: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.33k op/s 2024-03-12T04:17:20.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:20 smithi136 ceph-mon[35571]: pgmap v67: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.33k op/s 2024-03-12T04:17:20.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:20 smithi042 ceph-mon[32260]: pgmap v67: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.33k op/s 2024-03-12T04:17:22.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:22 smithi042 ceph-mon[32260]: pgmap v68: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.33k op/s 2024-03-12T04:17:22.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:22 smithi119 ceph-mon[36119]: pgmap v68: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.33k op/s 2024-03-12T04:17:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:22 smithi136 ceph-mon[35571]: pgmap v68: 97 pgs: 97 active+clean; 428 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.33k op/s 2024-03-12T04:17:24.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:24 smithi042 ceph-mon[32260]: pgmap v69: 97 pgs: 97 active+clean; 453 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.00k op/s 2024-03-12T04:17:24.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:24 smithi119 ceph-mon[36119]: pgmap v69: 97 pgs: 97 active+clean; 453 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.00k op/s 2024-03-12T04:17:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:24 smithi136 ceph-mon[35571]: pgmap v69: 97 pgs: 97 active+clean; 453 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.00k op/s 2024-03-12T04:17:26.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:26 smithi042 ceph-mon[32260]: pgmap v70: 97 pgs: 97 active+clean; 453 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.31k op/s 2024-03-12T04:17:26.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:26 smithi119 ceph-mon[36119]: pgmap v70: 97 pgs: 97 active+clean; 453 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.31k op/s 2024-03-12T04:17:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:26 smithi136 ceph-mon[35571]: pgmap v70: 97 pgs: 97 active+clean; 453 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.31k op/s 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:setting up random write file 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:done setting up random write file 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:starting fsync run 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0046s fsync time: 0.0062s 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0031s fsync time: 0.0197s 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0028s fsync time: 0.0114s 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0023s fsync time: 0.0198s 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0023s fsync time: 0.0118s 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0023s fsync time: 0.0108s 2024-03-12T04:17:28.195 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0031s fsync time: 0.0102s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0021s fsync time: 0.0067s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0023s fsync time: 0.0130s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0027s fsync time: 0.0148s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0027s fsync time: 0.0099s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0022s fsync time: 0.0088s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0036s fsync time: 0.0156s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0019s fsync time: 0.0252s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0034s fsync time: 0.0139s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0030s fsync time: 0.0188s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0015s fsync time: 0.0133s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0032s fsync time: 0.0154s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0022s fsync time: 0.0183s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0021s fsync time: 0.0096s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0027s fsync time: 0.0159s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0024s fsync time: 0.0114s 2024-03-12T04:17:28.196 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0025s fsync time: 0.0117s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0029s fsync time: 0.0204s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0020s fsync time: 0.0071s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0024s fsync time: 0.0083s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0023s fsync time: 0.0082s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0023s fsync time: 0.0171s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0024s fsync time: 0.0085s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0020s fsync time: 0.0091s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0021s fsync time: 0.0137s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0021s fsync time: 0.0131s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0025s fsync time: 0.0151s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0027s fsync time: 0.5876s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0051s fsync time: 0.0123s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0088s fsync time: 0.0147s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0023s fsync time: 0.0145s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0022s fsync time: 0.0156s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0026s fsync time: 0.0166s 2024-03-12T04:17:28.197 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0029s fsync time: 0.0151s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0028s fsync time: 0.0178s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0042s fsync time: 0.0101s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0024s fsync time: 0.0151s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0029s fsync time: 0.0096s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0012s fsync time: 0.0058s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0024s fsync time: 0.0429s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0026s fsync time: 0.0135s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0028s fsync time: 0.0145s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0037s fsync time: 0.0078s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0034s fsync time: 0.0115s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0027s fsync time: 0.0112s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0025s fsync time: 0.0171s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0028s fsync time: 0.0190s 2024-03-12T04:17:28.198 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0021s fsync time: 0.0093s 2024-03-12T04:17:28.199 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0021s fsync time: 0.0116s 2024-03-12T04:17:28.199 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0022s fsync time: 0.0120s 2024-03-12T04:17:28.199 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0021s fsync time: 0.0066s 2024-03-12T04:17:28.199 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0019s fsync time: 0.0124s 2024-03-12T04:17:28.199 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0027s fsync time: 0.0134s 2024-03-12T04:17:28.199 INFO:tasks.workunit.client.0.smithi042.stdout:write time: 0.0026s fsync time: 0.0101s 2024-03-12T04:17:28.199 INFO:tasks.workunit.client.0.smithi042.stdout:run done 60 fsyncs total, killing random writer 2024-03-12T04:17:28.209 INFO:tasks.workunit.client.0.smithi042.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-12T04:17:28.209 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-12T04:17:28.209 INFO:tasks.workunit.client.0.smithi042.stderr:+ whereis lsof 2024-03-12T04:17:28.245 INFO:tasks.workunit.client.0.smithi042.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-03-12T04:17:28.245 INFO:tasks.workunit.client.0.smithi042.stderr:+ lsof 2024-03-12T04:17:28.262 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/7a7b6618f707fda70007416825d3afd6dd89e16bd9cd4df0c9ef57898378eb35/merged 2024-03-12T04:17:28.262 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.262 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/3f50e22543facfefb6b374844324601da5909153d9e0a2f421ba2b05a359c6c5/merged 2024-03-12T04:17:28.263 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d9bb4bf25c6836f532201c4d00400a3f7c3d7c260928f5d2c283656902b791e9/merged 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/cef5d7aea60bc7010169bedd82aa3793aac04a095c8b2cd101b752ff48ce0552/merged 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/4f5db9aa364fee3a2cd9308a305c86e324d8b1a7fa849e4a5c5cfba9ec2b766a/merged 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/ede19f5b36a049244a2483d18fd0f409f5d496109f48b9d75052d6a7f4e21c57/merged 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fa67d47d8bcd6440e521bc3c58d42c8f4ec37221fe9a63a60e02a4fdade523c3/merged 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/9aa6785dc874cd7038109ee3b7c406f57ed632f61df396684ba898e5b0efca8f/merged 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.273 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/3bcbd2975c62824aca712b30a555ea1496ecda68c3dabc6c6380a4c3856e5210/merged 2024-03-12T04:17:28.274 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.274 INFO:tasks.workunit.client.0.smithi042.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0f9546f982ff1b6d6f12726c237ce555c686f3fdd8ece912859819672613209b/merged 2024-03-12T04:17:28.274 INFO:tasks.workunit.client.0.smithi042.stderr: Output information may be incomplete. 2024-03-12T04:17:28.333 INFO:tasks.workunit.client.0.smithi042.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-03-12T04:17:28.333 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-03-12T04:17:28.333 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-03-12T04:17:28.333 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-03-12T04:17:28.333 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-03-12T04:17:28.334 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-03-12T04:17:28.335 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-03-12T04:17:28.336 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-03-12T04:17:28.337 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-03-12T04:17:28.338 INFO:tasks.workunit.client.0.smithi042.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-03-12T04:17:28.339 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-03-12T04:17:28.340 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-03-12T04:17:28.341 INFO:tasks.workunit.client.0.smithi042.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-03-12T04:17:28.342 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-03-12T04:17:28.343 INFO:tasks.workunit.client.0.smithi042.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-03-12T04:17:28.344 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-03-12T04:17:28.345 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-03-12T04:17:28.346 INFO:tasks.workunit.client.0.smithi042.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-03-12T04:17:28.347 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-03-12T04:17:28.348 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-03-12T04:17:28.349 INFO:tasks.workunit.client.0.smithi042.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-03-12T04:17:28.350 INFO:tasks.workunit.client.0.smithi042.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-03-12T04:17:28.350 INFO:tasks.workunit.client.0.smithi042.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-03-12T04:17:28.350 INFO:tasks.workunit.client.0.smithi042.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-03-12T04:17:28.350 INFO:tasks.workunit.client.0.smithi042.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-03-12T04:17:28.350 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-03-12T04:17:28.351 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-03-12T04:17:28.352 INFO:tasks.workunit.client.0.smithi042.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-03-12T04:17:28.353 INFO:tasks.workunit.client.0.smithi042.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:khugepage 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:khugepage 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:khugepage 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:khugepage 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-03-12T04:17:28.354 INFO:tasks.workunit.client.0.smithi042.stdout:cryptd 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:cryptd 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:cryptd 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:cryptd 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kintegrit 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kintegrit 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kintegrit 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kintegrit 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kblockd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kblockd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kblockd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:kblockd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:blkcg_pun 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:blkcg_pun 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:blkcg_pun 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-03-12T04:17:28.355 INFO:tasks.workunit.client.0.smithi042.stdout:blkcg_pun 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-03-12T04:17:28.356 INFO:tasks.workunit.client.0.smithi042.stdout:tpm_dev_w 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:tpm_dev_w 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:tpm_dev_w 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:md 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:md 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:md 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:md 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:md_bitmap 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:md_bitmap 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-03-12T04:17:28.358 INFO:tasks.workunit.client.0.smithi042.stdout:md_bitmap 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:md_bitmap 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:edac-poll 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:edac-poll 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:edac-poll 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:edac-poll 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:watchdogd 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:watchdogd 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:watchdogd 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:watchdogd 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:kswapd0 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:kswapd0 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-03-12T04:17:28.359 INFO:tasks.workunit.client.0.smithi042.stdout:kswapd0 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kswapd0 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kthrotld 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kthrotld 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kthrotld 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kthrotld 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 107 root cwd unknown /proc/107/cwd (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 107 root rtd unknown /proc/107/root (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 107 root txt unknown /proc/107/exe (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 107 root NOFD /proc/107/fd (opendir: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:acpi_ther 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:acpi_ther 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:acpi_ther 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-03-12T04:17:28.360 INFO:tasks.workunit.client.0.smithi042.stdout:acpi_ther 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kmpath_rd 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kmpath_rd 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kmpath_rd 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kmpath_rd 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kaluad 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kaluad 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kaluad 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kaluad 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:mld 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:mld 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:mld 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:mld 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-03-12T04:17:28.361 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:ipv6_addr 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:ipv6_addr 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:ipv6_addr 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:ipv6_addr 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:kstrp 122 root cwd unknown /proc/122/cwd (readlink: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:kstrp 122 root rtd unknown /proc/122/root (readlink: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:kstrp 122 root txt unknown /proc/122/exe (readlink: Permission denied) 2024-03-12T04:17:28.362 INFO:tasks.workunit.client.0.smithi042.stdout:kstrp 122 root NOFD /proc/122/fd (opendir: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:zswap-shr 129 root cwd unknown /proc/129/cwd (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:zswap-shr 129 root rtd unknown /proc/129/root (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:zswap-shr 129 root txt unknown /proc/129/exe (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:zswap-shr 129 root NOFD /proc/129/fd (opendir: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 238 root cwd unknown /proc/238/cwd (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 238 root rtd unknown /proc/238/root (readlink: Permission denied) 2024-03-12T04:17:28.363 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 238 root txt unknown /proc/238/exe (readlink: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 238 root NOFD /proc/238/fd (opendir: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 278 root cwd unknown /proc/278/cwd (readlink: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 278 root rtd unknown /proc/278/root (readlink: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 278 root txt unknown /proc/278/exe (readlink: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 278 root NOFD /proc/278/fd (opendir: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 280 root cwd unknown /proc/280/cwd (readlink: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 280 root rtd unknown /proc/280/root (readlink: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 280 root txt unknown /proc/280/exe (readlink: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 280 root NOFD /proc/280/fd (opendir: Permission denied) 2024-03-12T04:17:28.364 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 281 root cwd unknown /proc/281/cwd (readlink: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 281 root rtd unknown /proc/281/root (readlink: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 281 root txt unknown /proc/281/exe (readlink: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 281 root NOFD /proc/281/fd (opendir: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 287 root cwd unknown /proc/287/cwd (readlink: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 287 root rtd unknown /proc/287/root (readlink: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 287 root txt unknown /proc/287/exe (readlink: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 287 root NOFD /proc/287/fd (opendir: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 288 root cwd unknown /proc/288/cwd (readlink: Permission denied) 2024-03-12T04:17:28.365 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 288 root rtd unknown /proc/288/root (readlink: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 288 root txt unknown /proc/288/exe (readlink: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 288 root NOFD /proc/288/fd (opendir: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 299 root cwd unknown /proc/299/cwd (readlink: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 299 root rtd unknown /proc/299/root (readlink: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 299 root txt unknown /proc/299/exe (readlink: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 299 root NOFD /proc/299/fd (opendir: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:ata_sff 424 root cwd unknown /proc/424/cwd (readlink: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:ata_sff 424 root rtd unknown /proc/424/root (readlink: Permission denied) 2024-03-12T04:17:28.366 INFO:tasks.workunit.client.0.smithi042.stdout:ata_sff 424 root txt unknown /proc/424/exe (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:ata_sff 424 root NOFD /proc/424/fd (opendir: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-wq 426 root cwd unknown /proc/426/cwd (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-wq 426 root rtd unknown /proc/426/root (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-wq 426 root txt unknown /proc/426/exe (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-wq 426 root NOFD /proc/426/fd (opendir: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-rese 428 root cwd unknown /proc/428/cwd (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-rese 428 root rtd unknown /proc/428/root (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-rese 428 root txt unknown /proc/428/exe (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-rese 428 root NOFD /proc/428/fd (opendir: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-dele 429 root cwd unknown /proc/429/cwd (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-dele 429 root rtd unknown /proc/429/root (readlink: Permission denied) 2024-03-12T04:17:28.367 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-dele 429 root txt unknown /proc/429/exe (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-dele 429 root NOFD /proc/429/fd (opendir: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-auth 430 root cwd unknown /proc/430/cwd (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-auth 430 root rtd unknown /proc/430/root (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-auth 430 root txt unknown /proc/430/exe (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:nvme-auth 430 root NOFD /proc/430/fd (opendir: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:ixgbe 436 root cwd unknown /proc/436/cwd (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:ixgbe 436 root rtd unknown /proc/436/root (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:ixgbe 436 root txt unknown /proc/436/exe (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:ixgbe 436 root NOFD /proc/436/fd (opendir: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_0 443 root cwd unknown /proc/443/cwd (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_0 443 root rtd unknown /proc/443/root (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_0 443 root txt unknown /proc/443/exe (readlink: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_0 443 root NOFD /proc/443/fd (opendir: Permission denied) 2024-03-12T04:17:28.368 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 444 root cwd unknown /proc/444/cwd (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 444 root rtd unknown /proc/444/root (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 444 root txt unknown /proc/444/exe (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 444 root NOFD /proc/444/fd (opendir: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_1 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_1 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_1 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_1 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-03-12T04:17:28.369 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_2 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_2 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_2 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_2 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_3 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_3 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_3 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_3 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-03-12T04:17:28.370 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_4 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_4 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_4 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_4 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_5 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-03-12T04:17:28.371 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_5 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_5 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_5 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_6 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_6 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_6 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_6 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-03-12T04:17:28.372 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_7 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_7 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_7 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_7 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_8 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_8 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_8 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_8 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-03-12T04:17:28.373 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_9 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_9 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_9 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_eh_9 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:scsi_tmf_ 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-03-12T04:17:28.374 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 495 root cwd unknown /proc/495/cwd (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 495 root rtd unknown /proc/495/root (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 495 root txt unknown /proc/495/exe (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 495 root NOFD /proc/495/fd (opendir: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:jbd2/sda1 501 root cwd unknown /proc/501/cwd (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:jbd2/sda1 501 root rtd unknown /proc/501/root (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:jbd2/sda1 501 root txt unknown /proc/501/exe (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:jbd2/sda1 501 root NOFD /proc/501/fd (opendir: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:ext4-rsv- 502 root cwd unknown /proc/502/cwd (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:ext4-rsv- 502 root rtd unknown /proc/502/root (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:ext4-rsv- 502 root txt unknown /proc/502/exe (readlink: Permission denied) 2024-03-12T04:17:28.375 INFO:tasks.workunit.client.0.smithi042.stdout:ext4-rsv- 502 root NOFD /proc/502/fd (opendir: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-j 578 root cwd unknown /proc/578/cwd (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-j 578 root rtd unknown /proc/578/root (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-j 578 root txt unknown /proc/578/exe (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-j 578 root NOFD /proc/578/fd (opendir: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:ext4lazyi 584 root cwd unknown /proc/584/cwd (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:ext4lazyi 584 root rtd unknown /proc/584/root (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:ext4lazyi 584 root txt unknown /proc/584/exe (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:ext4lazyi 584 root NOFD /proc/584/fd (opendir: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-u 593 root cwd unknown /proc/593/cwd (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-u 593 root rtd unknown /proc/593/root (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-u 593 root txt unknown /proc/593/exe (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-u 593 root NOFD /proc/593/fd (opendir: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:ipmi-msgh 636 root cwd unknown /proc/636/cwd (readlink: Permission denied) 2024-03-12T04:17:28.376 INFO:tasks.workunit.client.0.smithi042.stdout:ipmi-msgh 636 root rtd unknown /proc/636/root (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:ipmi-msgh 636 root txt unknown /proc/636/exe (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:ipmi-msgh 636 root NOFD /proc/636/fd (opendir: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:kipmi0 645 root cwd unknown /proc/645/cwd (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:kipmi0 645 root rtd unknown /proc/645/root (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:kipmi0 645 root txt unknown /proc/645/exe (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:kipmi0 645 root NOFD /proc/645/fd (opendir: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpcbind 660 rpc cwd unknown /proc/660/cwd (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpcbind 660 rpc rtd unknown /proc/660/root (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpcbind 660 rpc txt unknown /proc/660/exe (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpcbind 660 rpc NOFD /proc/660/fd (opendir: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpciod 668 root cwd unknown /proc/668/cwd (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpciod 668 root rtd unknown /proc/668/root (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpciod 668 root txt unknown /proc/668/exe (readlink: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:rpciod 668 root NOFD /proc/668/fd (opendir: Permission denied) 2024-03-12T04:17:28.377 INFO:tasks.workunit.client.0.smithi042.stdout:xprtiod 669 root cwd unknown /proc/669/cwd (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:xprtiod 669 root rtd unknown /proc/669/root (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:xprtiod 669 root txt unknown /proc/669/exe (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:xprtiod 669 root NOFD /proc/669/fd (opendir: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 root cwd unknown /proc/672/cwd (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 root rtd unknown /proc/672/root (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 root txt unknown /proc/672/exe (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 root NOFD /proc/672/fd (opendir: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 673 auditd root cwd unknown /proc/672/task/673/cwd (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 673 auditd root rtd unknown /proc/672/task/673/root (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 673 auditd root txt unknown /proc/672/task/673/exe (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 673 auditd root NOFD /proc/672/task/673/fd (opendir: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 678 auditd root cwd unknown /proc/672/task/678/cwd (readlink: Permission denied) 2024-03-12T04:17:28.378 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 678 auditd root rtd unknown /proc/672/task/678/root (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 678 auditd root txt unknown /proc/672/task/678/exe (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:auditd 672 678 auditd root NOFD /proc/672/task/678/fd (opendir: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:sedispatc 677 root cwd unknown /proc/677/cwd (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:sedispatc 677 root rtd unknown /proc/677/root (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:sedispatc 677 root txt unknown /proc/677/exe (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:sedispatc 677 root NOFD /proc/677/fd (opendir: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 714 dbus cwd unknown /proc/714/cwd (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 714 dbus rtd unknown /proc/714/root (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 714 dbus txt unknown /proc/714/exe (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 714 dbus NOFD /proc/714/fd (opendir: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 715 dbus cwd unknown /proc/715/cwd (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 715 dbus rtd unknown /proc/715/root (readlink: Permission denied) 2024-03-12T04:17:28.379 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 715 dbus txt unknown /proc/715/exe (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:dbus-brok 715 dbus NOFD /proc/715/fd (opendir: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 root cwd unknown /proc/716/cwd (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 root rtd unknown /proc/716/root (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 root txt unknown /proc/716/exe (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 root NOFD /proc/716/fd (opendir: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 744 gmain root cwd unknown /proc/716/task/744/cwd (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 744 gmain root rtd unknown /proc/716/task/744/root (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 744 gmain root txt unknown /proc/716/task/744/exe (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 744 gmain root NOFD /proc/716/task/744/fd (opendir: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 745 gdbus root cwd unknown /proc/716/task/745/cwd (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 745 gdbus root rtd unknown /proc/716/task/745/root (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 745 gdbus root txt unknown /proc/716/task/745/exe (readlink: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:NetworkMa 716 745 gdbus root NOFD /proc/716/task/745/fd (opendir: Permission denied) 2024-03-12T04:17:28.380 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 root cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 root rtd unknown /proc/719/root (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 root txt unknown /proc/719/exe (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 root NOFD /proc/719/fd (opendir: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 726 gmain root cwd unknown /proc/719/task/726/cwd (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 726 gmain root rtd unknown /proc/719/task/726/root (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 726 gmain root txt unknown /proc/719/task/726/exe (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:irqbalanc 719 726 gmain root NOFD /proc/719/task/726/fd (opendir: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:lsmd 720 libstoragemgmt cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:lsmd 720 libstoragemgmt rtd unknown /proc/720/root (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:lsmd 720 libstoragemgmt txt unknown /proc/720/exe (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:lsmd 720 libstoragemgmt NOFD /proc/720/fd (opendir: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:mcelog 721 root cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:mcelog 721 root rtd unknown /proc/721/root (readlink: Permission denied) 2024-03-12T04:17:28.381 INFO:tasks.workunit.client.0.smithi042.stdout:mcelog 721 root txt unknown /proc/721/exe (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:mcelog 721 root NOFD /proc/721/fd (opendir: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 polkitd cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 polkitd rtd unknown /proc/722/root (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 polkitd txt unknown /proc/722/exe (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 polkitd NOFD /proc/722/fd (opendir: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 734 gmain polkitd cwd unknown /proc/722/task/734/cwd (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 734 gmain polkitd rtd unknown /proc/722/task/734/root (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 734 gmain polkitd txt unknown /proc/722/task/734/exe (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 734 gmain polkitd NOFD /proc/722/task/734/fd (opendir: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 735 gdbus polkitd cwd unknown /proc/722/task/735/cwd (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 735 gdbus polkitd rtd unknown /proc/722/task/735/root (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 735 gdbus polkitd txt unknown /proc/722/task/735/exe (readlink: Permission denied) 2024-03-12T04:17:28.382 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 735 gdbus polkitd NOFD /proc/722/task/735/fd (opendir: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 736 JS\x20Hel polkitd cwd unknown /proc/722/task/736/cwd (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 736 JS\x20Hel polkitd rtd unknown /proc/722/task/736/root (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 736 JS\x20Hel polkitd txt unknown /proc/722/task/736/exe (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 736 JS\x20Hel polkitd NOFD /proc/722/task/736/fd (opendir: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 737 JS\x20Hel polkitd cwd unknown /proc/722/task/737/cwd (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 737 JS\x20Hel polkitd rtd unknown /proc/722/task/737/root (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 737 JS\x20Hel polkitd txt unknown /proc/722/task/737/exe (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 737 JS\x20Hel polkitd NOFD /proc/722/task/737/fd (opendir: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 738 JS\x20Hel polkitd cwd unknown /proc/722/task/738/cwd (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 738 JS\x20Hel polkitd rtd unknown /proc/722/task/738/root (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 738 JS\x20Hel polkitd txt unknown /proc/722/task/738/exe (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 738 JS\x20Hel polkitd NOFD /proc/722/task/738/fd (opendir: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 739 JS\x20Hel polkitd cwd unknown /proc/722/task/739/cwd (readlink: Permission denied) 2024-03-12T04:17:28.383 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 739 JS\x20Hel polkitd rtd unknown /proc/722/task/739/root (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 739 JS\x20Hel polkitd txt unknown /proc/722/task/739/exe (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 739 JS\x20Hel polkitd NOFD /proc/722/task/739/fd (opendir: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 740 JS\x20Hel polkitd cwd unknown /proc/722/task/740/cwd (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 740 JS\x20Hel polkitd rtd unknown /proc/722/task/740/root (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 740 JS\x20Hel polkitd txt unknown /proc/722/task/740/exe (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 740 JS\x20Hel polkitd NOFD /proc/722/task/740/fd (opendir: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 741 JS\x20Hel polkitd cwd unknown /proc/722/task/741/cwd (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 741 JS\x20Hel polkitd rtd unknown /proc/722/task/741/root (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 741 JS\x20Hel polkitd txt unknown /proc/722/task/741/exe (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 741 JS\x20Hel polkitd NOFD /proc/722/task/741/fd (opendir: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 742 JS\x20Hel polkitd cwd unknown /proc/722/task/742/cwd (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 742 JS\x20Hel polkitd rtd unknown /proc/722/task/742/root (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 742 JS\x20Hel polkitd txt unknown /proc/722/task/742/exe (readlink: Permission denied) 2024-03-12T04:17:28.384 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 742 JS\x20Hel polkitd NOFD /proc/722/task/742/fd (opendir: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 743 JS\x20Hel polkitd cwd unknown /proc/722/task/743/cwd (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 743 JS\x20Hel polkitd rtd unknown /proc/722/task/743/root (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 743 JS\x20Hel polkitd txt unknown /proc/722/task/743/exe (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 743 JS\x20Hel polkitd NOFD /proc/722/task/743/fd (opendir: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 747 polkitd polkitd cwd unknown /proc/722/task/747/cwd (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 747 polkitd polkitd rtd unknown /proc/722/task/747/root (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 747 polkitd polkitd txt unknown /proc/722/task/747/exe (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:polkitd 722 747 polkitd polkitd NOFD /proc/722/task/747/fd (opendir: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-l 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-l 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-l 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:systemd-l 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 774 root cwd unknown /proc/774/cwd (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 774 root rtd unknown /proc/774/root (readlink: Permission denied) 2024-03-12T04:17:28.385 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 774 root txt unknown /proc/774/exe (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 774 root NOFD /proc/774/fd (opendir: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 root cwd unknown /proc/775/cwd (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 root rtd unknown /proc/775/root (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 root txt unknown /proc/775/exe (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 root NOFD /proc/775/fd (opendir: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 864 tuned root cwd unknown /proc/775/task/864/cwd (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 864 tuned root rtd unknown /proc/775/task/864/root (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 864 tuned root txt unknown /proc/775/task/864/exe (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:tuned 775 864 tuned root NOFD /proc/775/task/864/fd (opendir: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 root cwd unknown /proc/783/cwd (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 root rtd unknown /proc/783/root (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 root txt unknown /proc/783/exe (readlink: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 root NOFD /proc/783/fd (opendir: Permission denied) 2024-03-12T04:17:28.386 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 784 gssproxy root cwd unknown /proc/783/task/784/cwd (readlink: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 784 gssproxy root rtd unknown /proc/783/task/784/root (readlink: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 784 gssproxy root txt unknown /proc/783/task/784/exe (readlink: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 784 gssproxy root NOFD /proc/783/task/784/fd (opendir: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 785 gssproxy root cwd unknown /proc/783/task/785/cwd (readlink: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 785 gssproxy root rtd unknown /proc/783/task/785/root (readlink: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 785 gssproxy root txt unknown /proc/783/task/785/exe (readlink: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 785 gssproxy root NOFD /proc/783/task/785/fd (opendir: Permission denied) 2024-03-12T04:17:28.387 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 786 gssproxy root cwd unknown /proc/783/task/786/cwd (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 786 gssproxy root rtd unknown /proc/783/task/786/root (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 786 gssproxy root txt unknown /proc/783/task/786/exe (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 786 gssproxy root NOFD /proc/783/task/786/fd (opendir: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 787 gssproxy root cwd unknown /proc/783/task/787/cwd (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 787 gssproxy root rtd unknown /proc/783/task/787/root (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 787 gssproxy root txt unknown /proc/783/task/787/exe (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 787 gssproxy root NOFD /proc/783/task/787/fd (opendir: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 788 gssproxy root cwd unknown /proc/783/task/788/cwd (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 788 gssproxy root rtd unknown /proc/783/task/788/root (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 788 gssproxy root txt unknown /proc/783/task/788/exe (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:gssproxy 783 788 gssproxy root NOFD /proc/783/task/788/fd (opendir: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:atd 790 root cwd unknown /proc/790/cwd (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:atd 790 root rtd unknown /proc/790/root (readlink: Permission denied) 2024-03-12T04:17:28.388 INFO:tasks.workunit.client.0.smithi042.stdout:atd 790 root txt unknown /proc/790/exe (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:atd 790 root NOFD /proc/790/fd (opendir: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:crond 791 root cwd unknown /proc/791/cwd (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:crond 791 root rtd unknown /proc/791/root (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:crond 791 root txt unknown /proc/791/exe (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:crond 791 root NOFD /proc/791/fd (opendir: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:nrpe 886 nrpe cwd unknown /proc/886/cwd (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:nrpe 886 nrpe rtd unknown /proc/886/root (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:nrpe 886 nrpe txt unknown /proc/886/exe (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:nrpe 886 nrpe NOFD /proc/886/fd (opendir: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1315 root cwd unknown /proc/1315/cwd (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1315 root rtd unknown /proc/1315/root (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1315 root txt unknown /proc/1315/exe (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1315 root NOFD /proc/1315/fd (opendir: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1321 root cwd unknown /proc/1321/cwd (readlink: Permission denied) 2024-03-12T04:17:28.389 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1321 root rtd unknown /proc/1321/root (readlink: Permission denied) 2024-03-12T04:17:28.390 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1321 root txt unknown /proc/1321/exe (readlink: Permission denied) 2024-03-12T04:17:28.390 INFO:tasks.workunit.client.0.smithi042.stdout:agetty 1321 root NOFD /proc/1321/fd (opendir: Permission denied) 2024-03-12T04:17:28.390 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu cwd DIR 8,1 4096 2 / 2024-03-12T04:17:28.390 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu rtd DIR 8,1 4096 2 / 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu DEL REG 8,1 1167 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu DEL REG 8,1 10007 /usr/lib64/libcrypto.so.3.0.7 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu DEL REG 8,1 1174 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2024-03-12T04:17:28.391 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu DEL REG 8,1 10126 /usr/lib64/libblkid.so.1.1.0 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2024-03-12T04:17:28.392 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu DEL REG 8,1 10156 /usr/lib64/libmount.so.1.1.0 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 1u unix 0x0000000000000000 0t0 24719 type=STREAM (CONNECTED) 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 2u unix 0x0000000000000000 0t0 24719 type=STREAM (CONNECTED) 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 3u unix 0x0000000000000000 0t0 24747 type=DGRAM (CONNECTED) 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 4u a_inode 0,14 0 2078 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 5u a_inode 0,14 0 2078 [signalfd] 2024-03-12T04:17:28.393 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 6r a_inode 0,14 0 2078 inotify 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 7r DIR 0,25 0 3393 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 8u a_inode 0,14 0 2078 [timerfd] 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 9u a_inode 0,14 0 2078 [eventpoll:10,13] 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 10r REG 0,20 0 24736 /proc/4838/mountinfo 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 11r a_inode 0,14 0 2078 inotify 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 12u unix 0x0000000000000000 0t0 24769 /run/user/1000/bus type=STREAM (LISTEN) 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 13r a_inode 0,14 0 2078 inotify 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 15u netlink 0t0 24751 KOBJECT_UEVENT 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 16u unix 0x0000000000000000 0t0 24756 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 17u unix 0x0000000000000000 0t0 24757 type=DGRAM (CONNECTED) 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 18u unix 0x0000000000000000 0t0 24758 type=DGRAM (CONNECTED) 2024-03-12T04:17:28.394 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 19u unix 0x0000000000000000 0t0 24759 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 20u unix 0x0000000000000000 0t0 24761 type=STREAM (CONNECTED) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 4838 ubuntu 21u a_inode 0,14 0 2078 [timerfd] 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 4840 ubuntu cwd unknown /proc/4840/cwd (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 4840 ubuntu rtd unknown /proc/4840/root (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 4840 ubuntu txt unknown /proc/4840/exe (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 4840 ubuntu NOFD /proc/4840/fd (opendir: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 root cwd unknown /proc/5103/cwd (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 root rtd unknown /proc/5103/root (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 root txt unknown /proc/5103/exe (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 root NOFD /proc/5103/fd (opendir: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5105 in:imjour root cwd unknown /proc/5103/task/5105/cwd (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5105 in:imjour root rtd unknown /proc/5103/task/5105/root (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5105 in:imjour root txt unknown /proc/5103/task/5105/exe (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5105 in:imjour root NOFD /proc/5103/task/5105/fd (opendir: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5106 rs:main root cwd unknown /proc/5103/task/5106/cwd (readlink: Permission denied) 2024-03-12T04:17:28.395 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5106 rs:main root rtd unknown /proc/5103/task/5106/root (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5106 rs:main root txt unknown /proc/5103/task/5106/exe (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:rsyslogd 5103 5106 rs:main root NOFD /proc/5103/task/5106/fd (opendir: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:tls-strp 15794 root cwd unknown /proc/15794/cwd (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:tls-strp 15794 root rtd unknown /proc/15794/root (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:tls-strp 15794 root txt unknown /proc/15794/exe (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:tls-strp 15794 root NOFD /proc/15794/fd (opendir: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:smartd 15843 root cwd unknown /proc/15843/cwd (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:smartd 15843 root rtd unknown /proc/15843/root (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:smartd 15843 root txt unknown /proc/15843/exe (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:smartd 15843 root NOFD /proc/15843/fd (opendir: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21191 root cwd unknown /proc/21191/cwd (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21191 root rtd unknown /proc/21191/root (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21191 root txt unknown /proc/21191/exe (readlink: Permission denied) 2024-03-12T04:17:28.396 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21191 root NOFD /proc/21191/fd (opendir: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21220 root cwd unknown /proc/21220/cwd (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21220 root rtd unknown /proc/21220/root (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21220 root txt unknown /proc/21220/exe (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21220 root NOFD /proc/21220/fd (opendir: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21249 root cwd unknown /proc/21249/cwd (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21249 root rtd unknown /proc/21249/root (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21249 root txt unknown /proc/21249/exe (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21249 root NOFD /proc/21249/fd (opendir: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21278 root cwd unknown /proc/21278/cwd (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21278 root rtd unknown /proc/21278/root (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21278 root txt unknown /proc/21278/exe (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21278 root NOFD /proc/21278/fd (opendir: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21307 root cwd unknown /proc/21307/cwd (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21307 root rtd unknown /proc/21307/root (readlink: Permission denied) 2024-03-12T04:17:28.397 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21307 root txt unknown /proc/21307/exe (readlink: Permission denied) 2024-03-12T04:17:28.398 INFO:tasks.workunit.client.0.smithi042.stdout:kdmflush/ 21307 root NOFD /proc/21307/fd (opendir: Permission denied) 2024-03-12T04:17:28.398 INFO:tasks.workunit.client.0.smithi042.stdout:xfsalloc 21471 root cwd unknown /proc/21471/cwd (readlink: Permission denied) 2024-03-12T04:17:28.398 INFO:tasks.workunit.client.0.smithi042.stdout:xfsalloc 21471 root rtd unknown /proc/21471/root (readlink: Permission denied) 2024-03-12T04:17:28.398 INFO:tasks.workunit.client.0.smithi042.stdout:xfsalloc 21471 root txt unknown /proc/21471/exe (readlink: Permission denied) 2024-03-12T04:17:28.398 INFO:tasks.workunit.client.0.smithi042.stdout:xfsalloc 21471 root NOFD /proc/21471/fd (opendir: Permission denied) 2024-03-12T04:17:28.398 INFO:tasks.workunit.client.0.smithi042.stdout:xfs_mru_c 21472 root cwd unknown /proc/21472/cwd (readlink: Permission denied) 2024-03-12T04:17:28.398 INFO:tasks.workunit.client.0.smithi042.stdout:xfs_mru_c 21472 root rtd unknown /proc/21472/root (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs_mru_c 21472 root txt unknown /proc/21472/exe (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs_mru_c 21472 root NOFD /proc/21472/fd (opendir: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-buf/d 21473 root cwd unknown /proc/21473/cwd (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-buf/d 21473 root rtd unknown /proc/21473/root (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-buf/d 21473 root txt unknown /proc/21473/exe (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-buf/d 21473 root NOFD /proc/21473/fd (opendir: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-conv/ 21474 root cwd unknown /proc/21474/cwd (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-conv/ 21474 root rtd unknown /proc/21474/root (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-conv/ 21474 root txt unknown /proc/21474/exe (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-conv/ 21474 root NOFD /proc/21474/fd (opendir: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-recla 21475 root cwd unknown /proc/21475/cwd (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-recla 21475 root rtd unknown /proc/21475/root (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-recla 21475 root txt unknown /proc/21475/exe (readlink: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-recla 21475 root NOFD /proc/21475/fd (opendir: Permission denied) 2024-03-12T04:17:28.399 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-block 21476 root cwd unknown /proc/21476/cwd (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-block 21476 root rtd unknown /proc/21476/root (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-block 21476 root txt unknown /proc/21476/exe (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-block 21476 root NOFD /proc/21476/fd (opendir: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-inode 21477 root cwd unknown /proc/21477/cwd (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-inode 21477 root rtd unknown /proc/21477/root (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-inode 21477 root txt unknown /proc/21477/exe (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-inode 21477 root NOFD /proc/21477/fd (opendir: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-log/d 21478 root cwd unknown /proc/21478/cwd (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-log/d 21478 root rtd unknown /proc/21478/root (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-log/d 21478 root txt unknown /proc/21478/exe (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-log/d 21478 root NOFD /proc/21478/fd (opendir: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-cil/d 21479 root cwd unknown /proc/21479/cwd (readlink: Permission denied) 2024-03-12T04:17:28.400 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-cil/d 21479 root rtd unknown /proc/21479/root (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-cil/d 21479 root txt unknown /proc/21479/exe (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:xfs-cil/d 21479 root NOFD /proc/21479/fd (opendir: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:xfsaild/d 21480 root cwd unknown /proc/21480/cwd (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:xfsaild/d 21480 root rtd unknown /proc/21480/root (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:xfsaild/d 21480 root txt unknown /proc/21480/exe (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:xfsaild/d 21480 root NOFD /proc/21480/fd (opendir: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23430 root cwd unknown /proc/23430/cwd (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23430 root rtd unknown /proc/23430/root (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23430 root txt unknown /proc/23430/exe (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23430 root NOFD /proc/23430/fd (opendir: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23433 ubuntu cwd unknown /proc/23433/cwd (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23433 ubuntu rtd unknown /proc/23433/root (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23433 ubuntu txt unknown /proc/23433/exe (readlink: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 23433 ubuntu NOFD /proc/23433/fd (opendir: Permission denied) 2024-03-12T04:17:28.401 INFO:tasks.workunit.client.0.smithi042.stdout:chronyd 23493 chrony cwd unknown /proc/23493/cwd (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:chronyd 23493 chrony rtd unknown /proc/23493/root (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:chronyd 23493 chrony txt unknown /proc/23493/exe (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:chronyd 23493 chrony NOFD /proc/23493/fd (opendir: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 23661 root cwd unknown /proc/23661/cwd (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 23661 root rtd unknown /proc/23661/root (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 23661 root txt unknown /proc/23661/exe (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 23661 root NOFD /proc/23661/fd (opendir: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:anacron 23686 root cwd unknown /proc/23686/cwd (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:anacron 23686 root rtd unknown /proc/23686/root (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:anacron 23686 root txt unknown /proc/23686/exe (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:anacron 23686 root NOFD /proc/23686/fd (opendir: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-cras 23809 ceph cwd unknown /proc/23809/cwd (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-cras 23809 ceph rtd unknown /proc/23809/root (readlink: Permission denied) 2024-03-12T04:17:28.402 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-cras 23809 ceph txt unknown /proc/23809/exe (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-cras 23809 ceph NOFD /proc/23809/fd (opendir: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 25564 root cwd unknown /proc/25564/cwd (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 25564 root rtd unknown /proc/25564/root (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 25564 root txt unknown /proc/25564/exe (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 25564 root NOFD /proc/25564/fd (opendir: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 25811 root cwd unknown /proc/25811/cwd (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 25811 root rtd unknown /proc/25811/root (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 25811 root txt unknown /proc/25811/exe (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 25811 root NOFD /proc/25811/fd (opendir: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 28159 root cwd unknown /proc/28159/cwd (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 28159 root rtd unknown /proc/28159/root (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 28159 root txt unknown /proc/28159/exe (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 28159 root NOFD /proc/28159/fd (opendir: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 28231 root cwd unknown /proc/28231/cwd (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 28231 root rtd unknown /proc/28231/root (readlink: Permission denied) 2024-03-12T04:17:28.403 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 28231 root txt unknown /proc/28231/exe (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 28231 root NOFD /proc/28231/fd (opendir: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 31176 root cwd unknown /proc/31176/cwd (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 31176 root rtd unknown /proc/31176/root (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 31176 root txt unknown /proc/31176/exe (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 31176 root NOFD /proc/31176/fd (opendir: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31233 root cwd unknown /proc/31233/cwd (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31233 root rtd unknown /proc/31233/root (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31233 root txt unknown /proc/31233/exe (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31233 root NOFD /proc/31233/fd (opendir: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31250 root cwd unknown /proc/31250/cwd (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31250 root rtd unknown /proc/31250/root (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31250 root txt unknown /proc/31250/exe (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31250 root NOFD /proc/31250/fd (opendir: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31251 root cwd unknown /proc/31251/cwd (readlink: Permission denied) 2024-03-12T04:17:28.404 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31251 root rtd unknown /proc/31251/root (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31251 root txt unknown /proc/31251/exe (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 31251 root NOFD /proc/31251/fd (opendir: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31268 root cwd unknown /proc/31268/cwd (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31268 root rtd unknown /proc/31268/root (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31268 root txt unknown /proc/31268/exe (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 31268 root NOFD /proc/31268/fd (opendir: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 32206 root cwd unknown /proc/32206/cwd (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 32206 root rtd unknown /proc/32206/root (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 32206 root txt unknown /proc/32206/exe (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 32206 root NOFD /proc/32206/fd (opendir: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32256 root cwd unknown /proc/32256/cwd (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32256 root rtd unknown /proc/32256/root (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32256 root txt unknown /proc/32256/exe (readlink: Permission denied) 2024-03-12T04:17:28.405 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32256 root NOFD /proc/32256/fd (opendir: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32258 root cwd unknown /proc/32258/cwd (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32258 root rtd unknown /proc/32258/root (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32258 root txt unknown /proc/32258/exe (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32258 root NOFD /proc/32258/fd (opendir: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 ceph cwd unknown /proc/32260/cwd (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 ceph rtd unknown /proc/32260/root (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 ceph txt unknown /proc/32260/exe (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 ceph NOFD /proc/32260/fd (opendir: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32261 log ceph cwd unknown /proc/32260/task/32261/cwd (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32261 log ceph rtd unknown /proc/32260/task/32261/root (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32261 log ceph txt unknown /proc/32260/task/32261/exe (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32261 log ceph NOFD /proc/32260/task/32261/fd (opendir: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32262 service ceph cwd unknown /proc/32260/task/32262/cwd (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32262 service ceph rtd unknown /proc/32260/task/32262/root (readlink: Permission denied) 2024-03-12T04:17:28.406 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32262 service ceph txt unknown /proc/32260/task/32262/exe (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32262 service ceph NOFD /proc/32260/task/32262/fd (opendir: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32263 admin_soc ceph cwd unknown /proc/32260/task/32263/cwd (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32263 admin_soc ceph rtd unknown /proc/32260/task/32263/root (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32263 admin_soc ceph txt unknown /proc/32260/task/32263/exe (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32263 admin_soc ceph NOFD /proc/32260/task/32263/fd (opendir: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32264 signal_ha ceph cwd unknown /proc/32260/task/32264/cwd (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32264 signal_ha ceph rtd unknown /proc/32260/task/32264/root (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32264 signal_ha ceph txt unknown /proc/32260/task/32264/exe (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32264 signal_ha ceph NOFD /proc/32260/task/32264/fd (opendir: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32265 rocksdb:l ceph cwd unknown /proc/32260/task/32265/cwd (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32265 rocksdb:l ceph rtd unknown /proc/32260/task/32265/root (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32265 rocksdb:l ceph txt unknown /proc/32260/task/32265/exe (readlink: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32265 rocksdb:l ceph NOFD /proc/32260/task/32265/fd (opendir: Permission denied) 2024-03-12T04:17:28.407 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32266 rocksdb:h ceph cwd unknown /proc/32260/task/32266/cwd (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32266 rocksdb:h ceph rtd unknown /proc/32260/task/32266/root (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32266 rocksdb:h ceph txt unknown /proc/32260/task/32266/exe (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32266 rocksdb:h ceph NOFD /proc/32260/task/32266/fd (opendir: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32282 ceph-mon ceph cwd unknown /proc/32260/task/32282/cwd (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32282 ceph-mon ceph rtd unknown /proc/32260/task/32282/root (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32282 ceph-mon ceph txt unknown /proc/32260/task/32282/exe (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32282 ceph-mon ceph NOFD /proc/32260/task/32282/fd (opendir: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32283 fn_monsto ceph cwd unknown /proc/32260/task/32283/cwd (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32283 fn_monsto ceph rtd unknown /proc/32260/task/32283/root (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32283 fn_monsto ceph txt unknown /proc/32260/task/32283/exe (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32283 fn_monsto ceph NOFD /proc/32260/task/32283/fd (opendir: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32284 msgr-work ceph cwd unknown /proc/32260/task/32284/cwd (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32284 msgr-work ceph rtd unknown /proc/32260/task/32284/root (readlink: Permission denied) 2024-03-12T04:17:28.408 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32284 msgr-work ceph txt unknown /proc/32260/task/32284/exe (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32284 msgr-work ceph NOFD /proc/32260/task/32284/fd (opendir: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32285 msgr-work ceph cwd unknown /proc/32260/task/32285/cwd (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32285 msgr-work ceph rtd unknown /proc/32260/task/32285/root (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32285 msgr-work ceph txt unknown /proc/32260/task/32285/exe (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32285 msgr-work ceph NOFD /proc/32260/task/32285/fd (opendir: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32286 msgr-work ceph cwd unknown /proc/32260/task/32286/cwd (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32286 msgr-work ceph rtd unknown /proc/32260/task/32286/root (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32286 msgr-work ceph txt unknown /proc/32260/task/32286/exe (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32286 msgr-work ceph NOFD /proc/32260/task/32286/fd (opendir: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32287 OpHistory ceph cwd unknown /proc/32260/task/32287/cwd (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32287 OpHistory ceph rtd unknown /proc/32260/task/32287/root (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32287 OpHistory ceph txt unknown /proc/32260/task/32287/exe (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32287 OpHistory ceph NOFD /proc/32260/task/32287/fd (opendir: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32288 fin ceph cwd unknown /proc/32260/task/32288/cwd (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32288 fin ceph rtd unknown /proc/32260/task/32288/root (readlink: Permission denied) 2024-03-12T04:17:28.409 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32288 fin ceph txt unknown /proc/32260/task/32288/exe (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32288 fin ceph NOFD /proc/32260/task/32288/fd (opendir: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32289 safe_time ceph cwd unknown /proc/32260/task/32289/cwd (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32289 safe_time ceph rtd unknown /proc/32260/task/32289/root (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32289 safe_time ceph txt unknown /proc/32260/task/32289/exe (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32289 safe_time ceph NOFD /proc/32260/task/32289/fd (opendir: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32290 cpu_tp ceph cwd unknown /proc/32260/task/32290/cwd (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32290 cpu_tp ceph rtd unknown /proc/32260/task/32290/root (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32290 cpu_tp ceph txt unknown /proc/32260/task/32290/exe (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32290 cpu_tp ceph NOFD /proc/32260/task/32290/fd (opendir: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32291 cpu_tp ceph cwd unknown /proc/32260/task/32291/cwd (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32291 cpu_tp ceph rtd unknown /proc/32260/task/32291/root (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32291 cpu_tp ceph txt unknown /proc/32260/task/32291/exe (readlink: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32291 cpu_tp ceph NOFD /proc/32260/task/32291/fd (opendir: Permission denied) 2024-03-12T04:17:28.410 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32292 cpu_tp ceph cwd unknown /proc/32260/task/32292/cwd (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32292 cpu_tp ceph rtd unknown /proc/32260/task/32292/root (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32292 cpu_tp ceph txt unknown /proc/32260/task/32292/exe (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32292 cpu_tp ceph NOFD /proc/32260/task/32292/fd (opendir: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32293 cpu_tp ceph cwd unknown /proc/32260/task/32293/cwd (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32293 cpu_tp ceph rtd unknown /proc/32260/task/32293/root (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32293 cpu_tp ceph txt unknown /proc/32260/task/32293/exe (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32293 cpu_tp ceph NOFD /proc/32260/task/32293/fd (opendir: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32294 ms_dispat ceph cwd unknown /proc/32260/task/32294/cwd (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32294 ms_dispat ceph rtd unknown /proc/32260/task/32294/root (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32294 ms_dispat ceph txt unknown /proc/32260/task/32294/exe (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32294 ms_dispat ceph NOFD /proc/32260/task/32294/fd (opendir: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32295 ms_local ceph cwd unknown /proc/32260/task/32295/cwd (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32295 ms_local ceph rtd unknown /proc/32260/task/32295/root (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32295 ms_local ceph txt unknown /proc/32260/task/32295/exe (readlink: Permission denied) 2024-03-12T04:17:28.411 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32295 ms_local ceph NOFD /proc/32260/task/32295/fd (opendir: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32296 safe_time ceph cwd unknown /proc/32260/task/32296/cwd (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32296 safe_time ceph rtd unknown /proc/32260/task/32296/root (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32296 safe_time ceph txt unknown /proc/32260/task/32296/exe (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32296 safe_time ceph NOFD /proc/32260/task/32296/fd (opendir: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32297 ms_dispat ceph cwd unknown /proc/32260/task/32297/cwd (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32297 ms_dispat ceph rtd unknown /proc/32260/task/32297/root (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32297 ms_dispat ceph txt unknown /proc/32260/task/32297/exe (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32297 ms_dispat ceph NOFD /proc/32260/task/32297/fd (opendir: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32298 ms_local ceph cwd unknown /proc/32260/task/32298/cwd (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32298 ms_local ceph rtd unknown /proc/32260/task/32298/root (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32298 ms_local ceph txt unknown /proc/32260/task/32298/exe (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 32298 ms_local ceph NOFD /proc/32260/task/32298/fd (opendir: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 68450 rstore_co ceph cwd unknown /proc/32260/task/68450/cwd (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 68450 rstore_co ceph rtd unknown /proc/32260/task/68450/root (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 68450 rstore_co ceph txt unknown /proc/32260/task/68450/exe (readlink: Permission denied) 2024-03-12T04:17:28.412 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mon 32260 68450 rstore_co ceph NOFD /proc/32260/task/68450/fd (opendir: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32493 root cwd unknown /proc/32493/cwd (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32493 root rtd unknown /proc/32493/root (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32493 root txt unknown /proc/32493/exe (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 32493 root NOFD /proc/32493/fd (opendir: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32495 root cwd unknown /proc/32495/cwd (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32495 root rtd unknown /proc/32495/root (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32495 root txt unknown /proc/32495/exe (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 32495 root NOFD /proc/32495/fd (opendir: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 ceph cwd unknown /proc/32497/cwd (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 ceph rtd unknown /proc/32497/root (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 ceph txt unknown /proc/32497/exe (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 ceph NOFD /proc/32497/fd (opendir: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82782 log ceph cwd unknown /proc/32497/task/82782/cwd (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82782 log ceph rtd unknown /proc/32497/task/82782/root (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82782 log ceph txt unknown /proc/32497/task/82782/exe (readlink: Permission denied) 2024-03-12T04:17:28.413 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82782 log ceph NOFD /proc/32497/task/82782/fd (opendir: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82784 msgr-work ceph cwd unknown /proc/32497/task/82784/cwd (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82784 msgr-work ceph rtd unknown /proc/32497/task/82784/root (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82784 msgr-work ceph txt unknown /proc/32497/task/82784/exe (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82784 msgr-work ceph NOFD /proc/32497/task/82784/fd (opendir: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82785 msgr-work ceph cwd unknown /proc/32497/task/82785/cwd (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82785 msgr-work ceph rtd unknown /proc/32497/task/82785/root (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82785 msgr-work ceph txt unknown /proc/32497/task/82785/exe (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82785 msgr-work ceph NOFD /proc/32497/task/82785/fd (opendir: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82786 msgr-work ceph cwd unknown /proc/32497/task/82786/cwd (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82786 msgr-work ceph rtd unknown /proc/32497/task/82786/root (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82786 msgr-work ceph txt unknown /proc/32497/task/82786/exe (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82786 msgr-work ceph NOFD /proc/32497/task/82786/fd (opendir: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82790 service ceph cwd unknown /proc/32497/task/82790/cwd (readlink: Permission denied) 2024-03-12T04:17:28.414 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82790 service ceph rtd unknown /proc/32497/task/82790/root (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82790 service ceph txt unknown /proc/32497/task/82790/exe (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82790 service ceph NOFD /proc/32497/task/82790/fd (opendir: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82791 admin_soc ceph cwd unknown /proc/32497/task/82791/cwd (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82791 admin_soc ceph rtd unknown /proc/32497/task/82791/root (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82791 admin_soc ceph txt unknown /proc/32497/task/82791/exe (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82791 admin_soc ceph NOFD /proc/32497/task/82791/fd (opendir: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82792 ceph_time ceph cwd unknown /proc/32497/task/82792/cwd (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82792 ceph_time ceph rtd unknown /proc/32497/task/82792/root (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82792 ceph_time ceph txt unknown /proc/32497/task/82792/exe (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82792 ceph_time ceph NOFD /proc/32497/task/82792/fd (opendir: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82793 fn_anonym ceph cwd unknown /proc/32497/task/82793/cwd (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82793 fn_anonym ceph rtd unknown /proc/32497/task/82793/root (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82793 fn_anonym ceph txt unknown /proc/32497/task/82793/exe (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82793 fn_anonym ceph NOFD /proc/32497/task/82793/fd (opendir: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82794 signal_ha ceph cwd unknown /proc/32497/task/82794/cwd (readlink: Permission denied) 2024-03-12T04:17:28.415 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82794 signal_ha ceph rtd unknown /proc/32497/task/82794/root (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82794 signal_ha ceph txt unknown /proc/32497/task/82794/exe (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82794 signal_ha ceph NOFD /proc/32497/task/82794/fd (opendir: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82795 mgrsb-fin ceph cwd unknown /proc/32497/task/82795/cwd (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82795 mgrsb-fin ceph rtd unknown /proc/32497/task/82795/root (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82795 mgrsb-fin ceph txt unknown /proc/32497/task/82795/exe (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82795 mgrsb-fin ceph NOFD /proc/32497/task/82795/fd (opendir: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82796 ms_dispat ceph cwd unknown /proc/32497/task/82796/cwd (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82796 ms_dispat ceph rtd unknown /proc/32497/task/82796/root (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82796 ms_dispat ceph txt unknown /proc/32497/task/82796/exe (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82796 ms_dispat ceph NOFD /proc/32497/task/82796/fd (opendir: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82797 ms_local ceph cwd unknown /proc/32497/task/82797/cwd (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82797 ms_local ceph rtd unknown /proc/32497/task/82797/root (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82797 ms_local ceph txt unknown /proc/32497/task/82797/exe (readlink: Permission denied) 2024-03-12T04:17:28.416 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82797 ms_local ceph NOFD /proc/32497/task/82797/fd (opendir: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82798 io_contex ceph cwd unknown /proc/32497/task/82798/cwd (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82798 io_contex ceph rtd unknown /proc/32497/task/82798/root (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82798 io_contex ceph txt unknown /proc/32497/task/82798/exe (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82798 io_contex ceph NOFD /proc/32497/task/82798/fd (opendir: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82799 io_contex ceph cwd unknown /proc/32497/task/82799/cwd (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82799 io_contex ceph rtd unknown /proc/32497/task/82799/root (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82799 io_contex ceph txt unknown /proc/32497/task/82799/exe (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82799 io_contex ceph NOFD /proc/32497/task/82799/fd (opendir: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82800 safe_time ceph cwd unknown /proc/32497/task/82800/cwd (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82800 safe_time ceph rtd unknown /proc/32497/task/82800/root (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82800 safe_time ceph txt unknown /proc/32497/task/82800/exe (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82800 safe_time ceph NOFD /proc/32497/task/82800/fd (opendir: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82801 safe_time ceph cwd unknown /proc/32497/task/82801/cwd (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82801 safe_time ceph rtd unknown /proc/32497/task/82801/root (readlink: Permission denied) 2024-03-12T04:17:28.417 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82801 safe_time ceph txt unknown /proc/32497/task/82801/exe (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82801 safe_time ceph NOFD /proc/32497/task/82801/fd (opendir: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82802 safe_time ceph cwd unknown /proc/32497/task/82802/cwd (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82802 safe_time ceph rtd unknown /proc/32497/task/82802/root (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82802 safe_time ceph txt unknown /proc/32497/task/82802/exe (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82802 safe_time ceph NOFD /proc/32497/task/82802/fd (opendir: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82803 fn_anonym ceph cwd unknown /proc/32497/task/82803/cwd (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82803 fn_anonym ceph rtd unknown /proc/32497/task/82803/root (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82803 fn_anonym ceph txt unknown /proc/32497/task/82803/exe (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82803 fn_anonym ceph NOFD /proc/32497/task/82803/fd (opendir: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82804 flusher ceph cwd unknown /proc/32497/task/82804/cwd (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82804 flusher ceph rtd unknown /proc/32497/task/82804/root (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82804 flusher ceph txt unknown /proc/32497/task/82804/exe (readlink: Permission denied) 2024-03-12T04:17:28.418 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82804 flusher ceph NOFD /proc/32497/task/82804/fd (opendir: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82805 safe_time ceph cwd unknown /proc/32497/task/82805/cwd (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82805 safe_time ceph rtd unknown /proc/32497/task/82805/root (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82805 safe_time ceph txt unknown /proc/32497/task/82805/exe (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82805 safe_time ceph NOFD /proc/32497/task/82805/fd (opendir: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82977 mgr-fin ceph cwd unknown /proc/32497/task/82977/cwd (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82977 mgr-fin ceph rtd unknown /proc/32497/task/82977/root (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82977 mgr-fin ceph txt unknown /proc/32497/task/82977/exe (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82977 mgr-fin ceph NOFD /proc/32497/task/82977/fd (opendir: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82978 ms_dispat ceph cwd unknown /proc/32497/task/82978/cwd (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82978 ms_dispat ceph rtd unknown /proc/32497/task/82978/root (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82978 ms_dispat ceph txt unknown /proc/32497/task/82978/exe (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82978 ms_dispat ceph NOFD /proc/32497/task/82978/fd (opendir: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82979 ms_local ceph cwd unknown /proc/32497/task/82979/cwd (readlink: Permission denied) 2024-03-12T04:17:28.419 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82979 ms_local ceph rtd unknown /proc/32497/task/82979/root (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82979 ms_local ceph txt unknown /proc/32497/task/82979/exe (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82979 ms_local ceph NOFD /proc/32497/task/82979/fd (opendir: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82980 safe_time ceph cwd unknown /proc/32497/task/82980/cwd (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82980 safe_time ceph rtd unknown /proc/32497/task/82980/root (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82980 safe_time ceph txt unknown /proc/32497/task/82980/exe (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82980 safe_time ceph NOFD /proc/32497/task/82980/fd (opendir: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82983 cmdfin ceph cwd unknown /proc/32497/task/82983/cwd (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82983 cmdfin ceph rtd unknown /proc/32497/task/82983/root (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82983 cmdfin ceph txt unknown /proc/32497/task/82983/exe (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82983 cmdfin ceph NOFD /proc/32497/task/82983/fd (opendir: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82987 io_contex ceph cwd unknown /proc/32497/task/82987/cwd (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82987 io_contex ceph rtd unknown /proc/32497/task/82987/root (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82987 io_contex ceph txt unknown /proc/32497/task/82987/exe (readlink: Permission denied) 2024-03-12T04:17:28.420 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82987 io_contex ceph NOFD /proc/32497/task/82987/fd (opendir: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82988 io_contex ceph cwd unknown /proc/32497/task/82988/cwd (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82988 io_contex ceph rtd unknown /proc/32497/task/82988/root (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82988 io_contex ceph txt unknown /proc/32497/task/82988/exe (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82988 io_contex ceph NOFD /proc/32497/task/82988/fd (opendir: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82989 ceph_time ceph cwd unknown /proc/32497/task/82989/cwd (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82989 ceph_time ceph rtd unknown /proc/32497/task/82989/root (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82989 ceph_time ceph txt unknown /proc/32497/task/82989/exe (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82989 ceph_time ceph NOFD /proc/32497/task/82989/fd (opendir: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82990 ms_dispat ceph cwd unknown /proc/32497/task/82990/cwd (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82990 ms_dispat ceph rtd unknown /proc/32497/task/82990/root (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82990 ms_dispat ceph txt unknown /proc/32497/task/82990/exe (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82990 ms_dispat ceph NOFD /proc/32497/task/82990/fd (opendir: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82991 ms_local ceph cwd unknown /proc/32497/task/82991/cwd (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82991 ms_local ceph rtd unknown /proc/32497/task/82991/root (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82991 ms_local ceph txt unknown /proc/32497/task/82991/exe (readlink: Permission denied) 2024-03-12T04:17:28.421 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82991 ms_local ceph NOFD /proc/32497/task/82991/fd (opendir: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82992 safe_time ceph cwd unknown /proc/32497/task/82992/cwd (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82992 safe_time ceph rtd unknown /proc/32497/task/82992/root (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82992 safe_time ceph txt unknown /proc/32497/task/82992/exe (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82992 safe_time ceph NOFD /proc/32497/task/82992/fd (opendir: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82993 safe_time ceph cwd unknown /proc/32497/task/82993/cwd (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82993 safe_time ceph rtd unknown /proc/32497/task/82993/root (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82993 safe_time ceph txt unknown /proc/32497/task/82993/exe (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82993 safe_time ceph NOFD /proc/32497/task/82993/fd (opendir: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82994 balancer ceph cwd unknown /proc/32497/task/82994/cwd (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82994 balancer ceph rtd unknown /proc/32497/task/82994/root (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82994 balancer ceph txt unknown /proc/32497/task/82994/exe (readlink: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82994 balancer ceph NOFD /proc/32497/task/82994/fd (opendir: Permission denied) 2024-03-12T04:17:28.422 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82995 m-fin-bal ceph cwd unknown /proc/32497/task/82995/cwd (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82995 m-fin-bal ceph rtd unknown /proc/32497/task/82995/root (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82995 m-fin-bal ceph txt unknown /proc/32497/task/82995/exe (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82995 m-fin-bal ceph NOFD /proc/32497/task/82995/fd (opendir: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82996 mgr-fin ceph cwd unknown /proc/32497/task/82996/cwd (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82996 mgr-fin ceph rtd unknown /proc/32497/task/82996/root (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82996 mgr-fin ceph txt unknown /proc/32497/task/82996/exe (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82996 mgr-fin ceph NOFD /proc/32497/task/82996/fd (opendir: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82997 mgr-fin ceph cwd unknown /proc/32497/task/82997/cwd (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82997 mgr-fin ceph rtd unknown /proc/32497/task/82997/root (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82997 mgr-fin ceph txt unknown /proc/32497/task/82997/exe (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82997 mgr-fin ceph NOFD /proc/32497/task/82997/fd (opendir: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82998 mgr-fin ceph cwd unknown /proc/32497/task/82998/cwd (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82998 mgr-fin ceph rtd unknown /proc/32497/task/82998/root (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82998 mgr-fin ceph txt unknown /proc/32497/task/82998/exe (readlink: Permission denied) 2024-03-12T04:17:28.423 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82998 mgr-fin ceph NOFD /proc/32497/task/82998/fd (opendir: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82999 mgr-fin ceph cwd unknown /proc/32497/task/82999/cwd (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82999 mgr-fin ceph rtd unknown /proc/32497/task/82999/root (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82999 mgr-fin ceph txt unknown /proc/32497/task/82999/exe (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 82999 mgr-fin ceph NOFD /proc/32497/task/82999/fd (opendir: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83000 mgr-fin ceph cwd unknown /proc/32497/task/83000/cwd (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83000 mgr-fin ceph rtd unknown /proc/32497/task/83000/root (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83000 mgr-fin ceph txt unknown /proc/32497/task/83000/exe (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83000 mgr-fin ceph NOFD /proc/32497/task/83000/fd (opendir: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83001 mgr-fin ceph cwd unknown /proc/32497/task/83001/cwd (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83001 mgr-fin ceph rtd unknown /proc/32497/task/83001/root (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83001 mgr-fin ceph txt unknown /proc/32497/task/83001/exe (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83001 mgr-fin ceph NOFD /proc/32497/task/83001/fd (opendir: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83002 mgr-fin ceph cwd unknown /proc/32497/task/83002/cwd (readlink: Permission denied) 2024-03-12T04:17:28.424 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83002 mgr-fin ceph rtd unknown /proc/32497/task/83002/root (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83002 mgr-fin ceph txt unknown /proc/32497/task/83002/exe (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83002 mgr-fin ceph NOFD /proc/32497/task/83002/fd (opendir: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83003 mgr-fin ceph cwd unknown /proc/32497/task/83003/cwd (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83003 mgr-fin ceph rtd unknown /proc/32497/task/83003/root (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83003 mgr-fin ceph txt unknown /proc/32497/task/83003/exe (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83003 mgr-fin ceph NOFD /proc/32497/task/83003/fd (opendir: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83004 mgr-fin ceph cwd unknown /proc/32497/task/83004/cwd (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83004 mgr-fin ceph rtd unknown /proc/32497/task/83004/root (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83004 mgr-fin ceph txt unknown /proc/32497/task/83004/exe (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83004 mgr-fin ceph NOFD /proc/32497/task/83004/fd (opendir: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83005 mgr-fin ceph cwd unknown /proc/32497/task/83005/cwd (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83005 mgr-fin ceph rtd unknown /proc/32497/task/83005/root (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83005 mgr-fin ceph txt unknown /proc/32497/task/83005/exe (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83005 mgr-fin ceph NOFD /proc/32497/task/83005/fd (opendir: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83006 mgr-fin ceph cwd unknown /proc/32497/task/83006/cwd (readlink: Permission denied) 2024-03-12T04:17:28.425 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83006 mgr-fin ceph rtd unknown /proc/32497/task/83006/root (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83006 mgr-fin ceph txt unknown /proc/32497/task/83006/exe (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83006 mgr-fin ceph NOFD /proc/32497/task/83006/fd (opendir: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83007 mgr-fin ceph cwd unknown /proc/32497/task/83007/cwd (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83007 mgr-fin ceph rtd unknown /proc/32497/task/83007/root (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83007 mgr-fin ceph txt unknown /proc/32497/task/83007/exe (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83007 mgr-fin ceph NOFD /proc/32497/task/83007/fd (opendir: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83008 mgr-fin ceph cwd unknown /proc/32497/task/83008/cwd (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83008 mgr-fin ceph rtd unknown /proc/32497/task/83008/root (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83008 mgr-fin ceph txt unknown /proc/32497/task/83008/exe (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83008 mgr-fin ceph NOFD /proc/32497/task/83008/fd (opendir: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83009 mgr-fin ceph cwd unknown /proc/32497/task/83009/cwd (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83009 mgr-fin ceph rtd unknown /proc/32497/task/83009/root (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83009 mgr-fin ceph txt unknown /proc/32497/task/83009/exe (readlink: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83009 mgr-fin ceph NOFD /proc/32497/task/83009/fd (opendir: Permission denied) 2024-03-12T04:17:28.426 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83010 mgr-fin ceph cwd unknown /proc/32497/task/83010/cwd (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83010 mgr-fin ceph rtd unknown /proc/32497/task/83010/root (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83010 mgr-fin ceph txt unknown /proc/32497/task/83010/exe (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83010 mgr-fin ceph NOFD /proc/32497/task/83010/fd (opendir: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83011 mgr-fin ceph cwd unknown /proc/32497/task/83011/cwd (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83011 mgr-fin ceph rtd unknown /proc/32497/task/83011/root (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83011 mgr-fin ceph txt unknown /proc/32497/task/83011/exe (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83011 mgr-fin ceph NOFD /proc/32497/task/83011/fd (opendir: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83012 cephadm ceph cwd unknown /proc/32497/task/83012/cwd (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83012 cephadm ceph rtd unknown /proc/32497/task/83012/root (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83012 cephadm ceph txt unknown /proc/32497/task/83012/exe (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83012 cephadm ceph NOFD /proc/32497/task/83012/fd (opendir: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83013 m-fin-cep ceph cwd unknown /proc/32497/task/83013/cwd (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83013 m-fin-cep ceph rtd unknown /proc/32497/task/83013/root (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83013 m-fin-cep ceph txt unknown /proc/32497/task/83013/exe (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83013 m-fin-cep ceph NOFD /proc/32497/task/83013/fd (opendir: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83014 mgr-fin ceph cwd unknown /proc/32497/task/83014/cwd (readlink: Permission denied) 2024-03-12T04:17:28.427 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83014 mgr-fin ceph rtd unknown /proc/32497/task/83014/root (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83014 mgr-fin ceph txt unknown /proc/32497/task/83014/exe (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83014 mgr-fin ceph NOFD /proc/32497/task/83014/fd (opendir: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83015 crash ceph cwd unknown /proc/32497/task/83015/cwd (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83015 crash ceph rtd unknown /proc/32497/task/83015/root (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83015 crash ceph txt unknown /proc/32497/task/83015/exe (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83015 crash ceph NOFD /proc/32497/task/83015/fd (opendir: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83016 m-fin-cra ceph cwd unknown /proc/32497/task/83016/cwd (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83016 m-fin-cra ceph rtd unknown /proc/32497/task/83016/root (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83016 m-fin-cra ceph txt unknown /proc/32497/task/83016/exe (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83016 m-fin-cra ceph NOFD /proc/32497/task/83016/fd (opendir: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83017 mgr-fin ceph cwd unknown /proc/32497/task/83017/cwd (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83017 mgr-fin ceph rtd unknown /proc/32497/task/83017/root (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83017 mgr-fin ceph txt unknown /proc/32497/task/83017/exe (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83017 mgr-fin ceph NOFD /proc/32497/task/83017/fd (opendir: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83018 cephadm ceph cwd unknown /proc/32497/task/83018/cwd (readlink: Permission denied) 2024-03-12T04:17:28.428 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83018 cephadm ceph rtd unknown /proc/32497/task/83018/root (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83018 cephadm ceph txt unknown /proc/32497/task/83018/exe (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83018 cephadm ceph NOFD /proc/32497/task/83018/fd (opendir: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83019 mgr-fin ceph cwd unknown /proc/32497/task/83019/cwd (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83019 mgr-fin ceph rtd unknown /proc/32497/task/83019/root (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83019 mgr-fin ceph txt unknown /proc/32497/task/83019/exe (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83019 mgr-fin ceph NOFD /proc/32497/task/83019/fd (opendir: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83020 mgr-fin ceph cwd unknown /proc/32497/task/83020/cwd (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83020 mgr-fin ceph rtd unknown /proc/32497/task/83020/root (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83020 mgr-fin ceph txt unknown /proc/32497/task/83020/exe (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83020 mgr-fin ceph NOFD /proc/32497/task/83020/fd (opendir: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83021 dashboard ceph cwd unknown /proc/32497/task/83021/cwd (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83021 dashboard ceph rtd unknown /proc/32497/task/83021/root (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83021 dashboard ceph txt unknown /proc/32497/task/83021/exe (readlink: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83021 dashboard ceph NOFD /proc/32497/task/83021/fd (opendir: Permission denied) 2024-03-12T04:17:28.429 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83022 mgr-fin ceph cwd unknown /proc/32497/task/83022/cwd (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83022 mgr-fin ceph rtd unknown /proc/32497/task/83022/root (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83022 mgr-fin ceph txt unknown /proc/32497/task/83022/exe (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83022 mgr-fin ceph NOFD /proc/32497/task/83022/fd (opendir: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83023 m-fin-das ceph cwd unknown /proc/32497/task/83023/cwd (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83023 m-fin-das ceph rtd unknown /proc/32497/task/83023/root (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83023 m-fin-das ceph txt unknown /proc/32497/task/83023/exe (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83023 m-fin-das ceph NOFD /proc/32497/task/83023/fd (opendir: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83024 devicehea ceph cwd unknown /proc/32497/task/83024/cwd (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83024 devicehea ceph rtd unknown /proc/32497/task/83024/root (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83024 devicehea ceph txt unknown /proc/32497/task/83024/exe (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83024 devicehea ceph NOFD /proc/32497/task/83024/fd (opendir: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83025 m-fin-dev ceph cwd unknown /proc/32497/task/83025/cwd (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83025 m-fin-dev ceph rtd unknown /proc/32497/task/83025/root (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83025 m-fin-dev ceph txt unknown /proc/32497/task/83025/exe (readlink: Permission denied) 2024-03-12T04:17:28.430 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83025 m-fin-dev ceph NOFD /proc/32497/task/83025/fd (opendir: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83026 mgr-fin ceph cwd unknown /proc/32497/task/83026/cwd (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83026 mgr-fin ceph rtd unknown /proc/32497/task/83026/root (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83026 mgr-fin ceph txt unknown /proc/32497/task/83026/exe (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83026 mgr-fin ceph NOFD /proc/32497/task/83026/fd (opendir: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83028 m-fin-ios ceph cwd unknown /proc/32497/task/83028/cwd (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83028 m-fin-ios ceph rtd unknown /proc/32497/task/83028/root (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83028 m-fin-ios ceph txt unknown /proc/32497/task/83028/exe (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83028 m-fin-ios ceph NOFD /proc/32497/task/83028/fd (opendir: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83029 mgr-fin ceph cwd unknown /proc/32497/task/83029/cwd (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83029 mgr-fin ceph rtd unknown /proc/32497/task/83029/root (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83029 mgr-fin ceph txt unknown /proc/32497/task/83029/exe (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83029 mgr-fin ceph NOFD /proc/32497/task/83029/fd (opendir: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83031 m-fin-nfs ceph cwd unknown /proc/32497/task/83031/cwd (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83031 m-fin-nfs ceph rtd unknown /proc/32497/task/83031/root (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83031 m-fin-nfs ceph txt unknown /proc/32497/task/83031/exe (readlink: Permission denied) 2024-03-12T04:17:28.431 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83031 m-fin-nfs ceph NOFD /proc/32497/task/83031/fd (opendir: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83032 mgr-fin ceph cwd unknown /proc/32497/task/83032/cwd (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83032 mgr-fin ceph rtd unknown /proc/32497/task/83032/root (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83032 mgr-fin ceph txt unknown /proc/32497/task/83032/exe (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83032 mgr-fin ceph NOFD /proc/32497/task/83032/fd (opendir: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83034 m-fin-orc ceph cwd unknown /proc/32497/task/83034/cwd (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83034 m-fin-orc ceph rtd unknown /proc/32497/task/83034/root (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83034 m-fin-orc ceph txt unknown /proc/32497/task/83034/exe (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83034 m-fin-orc ceph NOFD /proc/32497/task/83034/fd (opendir: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83035 pg_autosc ceph cwd unknown /proc/32497/task/83035/cwd (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83035 pg_autosc ceph rtd unknown /proc/32497/task/83035/root (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83035 pg_autosc ceph txt unknown /proc/32497/task/83035/exe (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83035 pg_autosc ceph NOFD /proc/32497/task/83035/fd (opendir: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83036 m-fin-pg_ ceph cwd unknown /proc/32497/task/83036/cwd (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83036 m-fin-pg_ ceph rtd unknown /proc/32497/task/83036/root (readlink: Permission denied) 2024-03-12T04:17:28.432 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83036 m-fin-pg_ ceph txt unknown /proc/32497/task/83036/exe (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83036 m-fin-pg_ ceph NOFD /proc/32497/task/83036/fd (opendir: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83037 mgr-fin ceph cwd unknown /proc/32497/task/83037/cwd (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83037 mgr-fin ceph rtd unknown /proc/32497/task/83037/root (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83037 mgr-fin ceph txt unknown /proc/32497/task/83037/exe (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83037 mgr-fin ceph NOFD /proc/32497/task/83037/fd (opendir: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83038 progress ceph cwd unknown /proc/32497/task/83038/cwd (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83038 progress ceph rtd unknown /proc/32497/task/83038/root (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83038 progress ceph txt unknown /proc/32497/task/83038/exe (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83038 progress ceph NOFD /proc/32497/task/83038/fd (opendir: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83039 m-fin-pro ceph cwd unknown /proc/32497/task/83039/cwd (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83039 m-fin-pro ceph rtd unknown /proc/32497/task/83039/root (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83039 m-fin-pro ceph txt unknown /proc/32497/task/83039/exe (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83039 m-fin-pro ceph NOFD /proc/32497/task/83039/fd (opendir: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83040 mgr-fin ceph cwd unknown /proc/32497/task/83040/cwd (readlink: Permission denied) 2024-03-12T04:17:28.433 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83040 mgr-fin ceph rtd unknown /proc/32497/task/83040/root (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83040 mgr-fin ceph txt unknown /proc/32497/task/83040/exe (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83040 mgr-fin ceph NOFD /proc/32497/task/83040/fd (opendir: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83041 mgr-fin ceph cwd unknown /proc/32497/task/83041/cwd (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83041 mgr-fin ceph rtd unknown /proc/32497/task/83041/root (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83041 mgr-fin ceph txt unknown /proc/32497/task/83041/exe (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83041 mgr-fin ceph NOFD /proc/32497/task/83041/fd (opendir: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83042 mgr-fin ceph cwd unknown /proc/32497/task/83042/cwd (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83042 mgr-fin ceph rtd unknown /proc/32497/task/83042/root (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83042 mgr-fin ceph txt unknown /proc/32497/task/83042/exe (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83042 mgr-fin ceph NOFD /proc/32497/task/83042/fd (opendir: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83046 io_contex ceph cwd unknown /proc/32497/task/83046/cwd (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83046 io_contex ceph rtd unknown /proc/32497/task/83046/root (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83046 io_contex ceph txt unknown /proc/32497/task/83046/exe (readlink: Permission denied) 2024-03-12T04:17:28.434 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83046 io_contex ceph NOFD /proc/32497/task/83046/fd (opendir: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83047 io_contex ceph cwd unknown /proc/32497/task/83047/cwd (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83047 io_contex ceph rtd unknown /proc/32497/task/83047/root (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83047 io_contex ceph txt unknown /proc/32497/task/83047/exe (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83047 io_contex ceph NOFD /proc/32497/task/83047/fd (opendir: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83048 ceph_time ceph cwd unknown /proc/32497/task/83048/cwd (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83048 ceph_time ceph rtd unknown /proc/32497/task/83048/root (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83048 ceph_time ceph txt unknown /proc/32497/task/83048/exe (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83048 ceph_time ceph NOFD /proc/32497/task/83048/fd (opendir: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83049 ms_dispat ceph cwd unknown /proc/32497/task/83049/cwd (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83049 ms_dispat ceph rtd unknown /proc/32497/task/83049/root (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83049 ms_dispat ceph txt unknown /proc/32497/task/83049/exe (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83049 ms_dispat ceph NOFD /proc/32497/task/83049/fd (opendir: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83050 ms_local ceph cwd unknown /proc/32497/task/83050/cwd (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83050 ms_local ceph rtd unknown /proc/32497/task/83050/root (readlink: Permission denied) 2024-03-12T04:17:28.435 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83050 ms_local ceph txt unknown /proc/32497/task/83050/exe (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83050 ms_local ceph NOFD /proc/32497/task/83050/fd (opendir: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83051 safe_time ceph cwd unknown /proc/32497/task/83051/cwd (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83051 safe_time ceph rtd unknown /proc/32497/task/83051/root (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83051 safe_time ceph txt unknown /proc/32497/task/83051/exe (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83051 safe_time ceph NOFD /proc/32497/task/83051/fd (opendir: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83052 safe_time ceph cwd unknown /proc/32497/task/83052/cwd (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83052 safe_time ceph rtd unknown /proc/32497/task/83052/root (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83052 safe_time ceph txt unknown /proc/32497/task/83052/exe (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83052 safe_time ceph NOFD /proc/32497/task/83052/fd (opendir: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83053 mgr-fin ceph cwd unknown /proc/32497/task/83053/cwd (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83053 mgr-fin ceph rtd unknown /proc/32497/task/83053/root (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83053 mgr-fin ceph txt unknown /proc/32497/task/83053/exe (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83053 mgr-fin ceph NOFD /proc/32497/task/83053/fd (opendir: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83055 m-fin-rbd ceph cwd unknown /proc/32497/task/83055/cwd (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83055 m-fin-rbd ceph rtd unknown /proc/32497/task/83055/root (readlink: Permission denied) 2024-03-12T04:17:28.436 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83055 m-fin-rbd ceph txt unknown /proc/32497/task/83055/exe (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83055 m-fin-rbd ceph NOFD /proc/32497/task/83055/fd (opendir: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83056 restful ceph cwd unknown /proc/32497/task/83056/cwd (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83056 restful ceph rtd unknown /proc/32497/task/83056/root (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83056 restful ceph txt unknown /proc/32497/task/83056/exe (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83056 restful ceph NOFD /proc/32497/task/83056/fd (opendir: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83057 m-fin-res ceph cwd unknown /proc/32497/task/83057/cwd (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83057 m-fin-res ceph rtd unknown /proc/32497/task/83057/root (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83057 m-fin-res ceph txt unknown /proc/32497/task/83057/exe (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83057 m-fin-res ceph NOFD /proc/32497/task/83057/fd (opendir: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83058 mgr-fin ceph cwd unknown /proc/32497/task/83058/cwd (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83058 mgr-fin ceph rtd unknown /proc/32497/task/83058/root (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83058 mgr-fin ceph txt unknown /proc/32497/task/83058/exe (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83058 mgr-fin ceph NOFD /proc/32497/task/83058/fd (opendir: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83059 mgr-fin ceph cwd unknown /proc/32497/task/83059/cwd (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83059 mgr-fin ceph rtd unknown /proc/32497/task/83059/root (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83059 mgr-fin ceph txt unknown /proc/32497/task/83059/exe (readlink: Permission denied) 2024-03-12T04:17:28.437 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83059 mgr-fin ceph NOFD /proc/32497/task/83059/fd (opendir: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83060 mgr-fin ceph cwd unknown /proc/32497/task/83060/cwd (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83060 mgr-fin ceph rtd unknown /proc/32497/task/83060/root (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83060 mgr-fin ceph txt unknown /proc/32497/task/83060/exe (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83060 mgr-fin ceph NOFD /proc/32497/task/83060/fd (opendir: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83061 mgr-fin ceph cwd unknown /proc/32497/task/83061/cwd (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83061 mgr-fin ceph rtd unknown /proc/32497/task/83061/root (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83061 mgr-fin ceph txt unknown /proc/32497/task/83061/exe (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83061 mgr-fin ceph NOFD /proc/32497/task/83061/fd (opendir: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83062 mgr-fin ceph cwd unknown /proc/32497/task/83062/cwd (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83062 mgr-fin ceph rtd unknown /proc/32497/task/83062/root (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83062 mgr-fin ceph txt unknown /proc/32497/task/83062/exe (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83062 mgr-fin ceph NOFD /proc/32497/task/83062/fd (opendir: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83063 mgr-fin ceph cwd unknown /proc/32497/task/83063/cwd (readlink: Permission denied) 2024-03-12T04:17:28.438 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83063 mgr-fin ceph rtd unknown /proc/32497/task/83063/root (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83063 mgr-fin ceph txt unknown /proc/32497/task/83063/exe (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83063 mgr-fin ceph NOFD /proc/32497/task/83063/fd (opendir: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83067 io_contex ceph cwd unknown /proc/32497/task/83067/cwd (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83067 io_contex ceph rtd unknown /proc/32497/task/83067/root (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83067 io_contex ceph txt unknown /proc/32497/task/83067/exe (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83067 io_contex ceph NOFD /proc/32497/task/83067/fd (opendir: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83068 io_contex ceph cwd unknown /proc/32497/task/83068/cwd (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83068 io_contex ceph rtd unknown /proc/32497/task/83068/root (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83068 io_contex ceph txt unknown /proc/32497/task/83068/exe (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83068 io_contex ceph NOFD /proc/32497/task/83068/fd (opendir: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83069 ceph_time ceph cwd unknown /proc/32497/task/83069/cwd (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83069 ceph_time ceph rtd unknown /proc/32497/task/83069/root (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83069 ceph_time ceph txt unknown /proc/32497/task/83069/exe (readlink: Permission denied) 2024-03-12T04:17:28.439 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83069 ceph_time ceph NOFD /proc/32497/task/83069/fd (opendir: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83070 ms_dispat ceph cwd unknown /proc/32497/task/83070/cwd (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83070 ms_dispat ceph rtd unknown /proc/32497/task/83070/root (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83070 ms_dispat ceph txt unknown /proc/32497/task/83070/exe (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83070 ms_dispat ceph NOFD /proc/32497/task/83070/fd (opendir: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83071 ms_local ceph cwd unknown /proc/32497/task/83071/cwd (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83071 ms_local ceph rtd unknown /proc/32497/task/83071/root (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83071 ms_local ceph txt unknown /proc/32497/task/83071/exe (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83071 ms_local ceph NOFD /proc/32497/task/83071/fd (opendir: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83072 safe_time ceph cwd unknown /proc/32497/task/83072/cwd (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83072 safe_time ceph rtd unknown /proc/32497/task/83072/root (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83072 safe_time ceph txt unknown /proc/32497/task/83072/exe (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83072 safe_time ceph NOFD /proc/32497/task/83072/fd (opendir: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83073 safe_time ceph cwd unknown /proc/32497/task/83073/cwd (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83073 safe_time ceph rtd unknown /proc/32497/task/83073/root (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83073 safe_time ceph txt unknown /proc/32497/task/83073/exe (readlink: Permission denied) 2024-03-12T04:17:28.440 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83073 safe_time ceph NOFD /proc/32497/task/83073/fd (opendir: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83077 m-fin-sna ceph cwd unknown /proc/32497/task/83077/cwd (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83077 m-fin-sna ceph rtd unknown /proc/32497/task/83077/root (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83077 m-fin-sna ceph txt unknown /proc/32497/task/83077/exe (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83077 m-fin-sna ceph NOFD /proc/32497/task/83077/fd (opendir: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83079 m-fin-sta ceph cwd unknown /proc/32497/task/83079/cwd (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83079 m-fin-sta ceph rtd unknown /proc/32497/task/83079/root (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83079 m-fin-sta ceph txt unknown /proc/32497/task/83079/exe (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83079 m-fin-sta ceph NOFD /proc/32497/task/83079/fd (opendir: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83080 telemetry ceph cwd unknown /proc/32497/task/83080/cwd (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83080 telemetry ceph rtd unknown /proc/32497/task/83080/root (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83080 telemetry ceph txt unknown /proc/32497/task/83080/exe (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83080 telemetry ceph NOFD /proc/32497/task/83080/fd (opendir: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83081 m-fin-tel ceph cwd unknown /proc/32497/task/83081/cwd (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83081 m-fin-tel ceph rtd unknown /proc/32497/task/83081/root (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83081 m-fin-tel ceph txt unknown /proc/32497/task/83081/exe (readlink: Permission denied) 2024-03-12T04:17:28.441 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83081 m-fin-tel ceph NOFD /proc/32497/task/83081/fd (opendir: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83082 mgr-fin ceph cwd unknown /proc/32497/task/83082/cwd (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83082 mgr-fin ceph rtd unknown /proc/32497/task/83082/root (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83082 mgr-fin ceph txt unknown /proc/32497/task/83082/exe (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83082 mgr-fin ceph NOFD /proc/32497/task/83082/fd (opendir: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83086 io_contex ceph cwd unknown /proc/32497/task/83086/cwd (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83086 io_contex ceph rtd unknown /proc/32497/task/83086/root (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83086 io_contex ceph txt unknown /proc/32497/task/83086/exe (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83086 io_contex ceph NOFD /proc/32497/task/83086/fd (opendir: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83087 io_contex ceph cwd unknown /proc/32497/task/83087/cwd (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83087 io_contex ceph rtd unknown /proc/32497/task/83087/root (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83087 io_contex ceph txt unknown /proc/32497/task/83087/exe (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83087 io_contex ceph NOFD /proc/32497/task/83087/fd (opendir: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83088 ceph_time ceph cwd unknown /proc/32497/task/83088/cwd (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83088 ceph_time ceph rtd unknown /proc/32497/task/83088/root (readlink: Permission denied) 2024-03-12T04:17:28.442 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83088 ceph_time ceph txt unknown /proc/32497/task/83088/exe (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83088 ceph_time ceph NOFD /proc/32497/task/83088/fd (opendir: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83089 ms_dispat ceph cwd unknown /proc/32497/task/83089/cwd (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83089 ms_dispat ceph rtd unknown /proc/32497/task/83089/root (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83089 ms_dispat ceph txt unknown /proc/32497/task/83089/exe (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83089 ms_dispat ceph NOFD /proc/32497/task/83089/fd (opendir: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83090 ms_local ceph cwd unknown /proc/32497/task/83090/cwd (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83090 ms_local ceph rtd unknown /proc/32497/task/83090/root (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83090 ms_local ceph txt unknown /proc/32497/task/83090/exe (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83090 ms_local ceph NOFD /proc/32497/task/83090/fd (opendir: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83091 safe_time ceph cwd unknown /proc/32497/task/83091/cwd (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83091 safe_time ceph rtd unknown /proc/32497/task/83091/root (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83091 safe_time ceph txt unknown /proc/32497/task/83091/exe (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83091 safe_time ceph NOFD /proc/32497/task/83091/fd (opendir: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83092 safe_time ceph cwd unknown /proc/32497/task/83092/cwd (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83092 safe_time ceph rtd unknown /proc/32497/task/83092/root (readlink: Permission denied) 2024-03-12T04:17:28.443 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83092 safe_time ceph txt unknown /proc/32497/task/83092/exe (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83092 safe_time ceph NOFD /proc/32497/task/83092/fd (opendir: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83093 mgr-fin ceph cwd unknown /proc/32497/task/83093/cwd (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83093 mgr-fin ceph rtd unknown /proc/32497/task/83093/root (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83093 mgr-fin ceph txt unknown /proc/32497/task/83093/exe (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83093 mgr-fin ceph NOFD /proc/32497/task/83093/fd (opendir: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83094 mgr-fin ceph cwd unknown /proc/32497/task/83094/cwd (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83094 mgr-fin ceph rtd unknown /proc/32497/task/83094/root (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83094 mgr-fin ceph txt unknown /proc/32497/task/83094/exe (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83094 mgr-fin ceph NOFD /proc/32497/task/83094/fd (opendir: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83095 mgr-fin ceph cwd unknown /proc/32497/task/83095/cwd (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83095 mgr-fin ceph rtd unknown /proc/32497/task/83095/root (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83095 mgr-fin ceph txt unknown /proc/32497/task/83095/exe (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83095 mgr-fin ceph NOFD /proc/32497/task/83095/fd (opendir: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83096 mgr-fin ceph cwd unknown /proc/32497/task/83096/cwd (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83096 mgr-fin ceph rtd unknown /proc/32497/task/83096/root (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83096 mgr-fin ceph txt unknown /proc/32497/task/83096/exe (readlink: Permission denied) 2024-03-12T04:17:28.444 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83096 mgr-fin ceph NOFD /proc/32497/task/83096/fd (opendir: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83097 mgr-fin ceph cwd unknown /proc/32497/task/83097/cwd (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83097 mgr-fin ceph rtd unknown /proc/32497/task/83097/root (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83097 mgr-fin ceph txt unknown /proc/32497/task/83097/exe (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83097 mgr-fin ceph NOFD /proc/32497/task/83097/fd (opendir: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83098 mgr-fin ceph cwd unknown /proc/32497/task/83098/cwd (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83098 mgr-fin ceph rtd unknown /proc/32497/task/83098/root (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83098 mgr-fin ceph txt unknown /proc/32497/task/83098/exe (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83098 mgr-fin ceph NOFD /proc/32497/task/83098/fd (opendir: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83099 mgr-fin ceph cwd unknown /proc/32497/task/83099/cwd (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83099 mgr-fin ceph rtd unknown /proc/32497/task/83099/root (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83099 mgr-fin ceph txt unknown /proc/32497/task/83099/exe (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83099 mgr-fin ceph NOFD /proc/32497/task/83099/fd (opendir: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83100 mgr-fin ceph cwd unknown /proc/32497/task/83100/cwd (readlink: Permission denied) 2024-03-12T04:17:28.445 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83100 mgr-fin ceph rtd unknown /proc/32497/task/83100/root (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83100 mgr-fin ceph txt unknown /proc/32497/task/83100/exe (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83100 mgr-fin ceph NOFD /proc/32497/task/83100/fd (opendir: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83101 mgr-fin ceph cwd unknown /proc/32497/task/83101/cwd (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83101 mgr-fin ceph rtd unknown /proc/32497/task/83101/root (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83101 mgr-fin ceph txt unknown /proc/32497/task/83101/exe (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83101 mgr-fin ceph NOFD /proc/32497/task/83101/fd (opendir: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83102 mgr-fin ceph cwd unknown /proc/32497/task/83102/cwd (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83102 mgr-fin ceph rtd unknown /proc/32497/task/83102/root (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83102 mgr-fin ceph txt unknown /proc/32497/task/83102/exe (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83102 mgr-fin ceph NOFD /proc/32497/task/83102/fd (opendir: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83103 mgr-fin ceph cwd unknown /proc/32497/task/83103/cwd (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83103 mgr-fin ceph rtd unknown /proc/32497/task/83103/root (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83103 mgr-fin ceph txt unknown /proc/32497/task/83103/exe (readlink: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83103 mgr-fin ceph NOFD /proc/32497/task/83103/fd (opendir: Permission denied) 2024-03-12T04:17:28.446 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83104 mgr-fin ceph cwd unknown /proc/32497/task/83104/cwd (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83104 mgr-fin ceph rtd unknown /proc/32497/task/83104/root (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83104 mgr-fin ceph txt unknown /proc/32497/task/83104/exe (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83104 mgr-fin ceph NOFD /proc/32497/task/83104/fd (opendir: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83106 m-fin-vol ceph cwd unknown /proc/32497/task/83106/cwd (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83106 m-fin-vol ceph rtd unknown /proc/32497/task/83106/root (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83106 m-fin-vol ceph txt unknown /proc/32497/task/83106/exe (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83106 m-fin-vol ceph NOFD /proc/32497/task/83106/fd (opendir: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83107 io_contex ceph cwd unknown /proc/32497/task/83107/cwd (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83107 io_contex ceph rtd unknown /proc/32497/task/83107/root (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83107 io_contex ceph txt unknown /proc/32497/task/83107/exe (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83107 io_contex ceph NOFD /proc/32497/task/83107/fd (opendir: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83108 io_contex ceph cwd unknown /proc/32497/task/83108/cwd (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83108 io_contex ceph rtd unknown /proc/32497/task/83108/root (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83108 io_contex ceph txt unknown /proc/32497/task/83108/exe (readlink: Permission denied) 2024-03-12T04:17:28.447 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83108 io_contex ceph NOFD /proc/32497/task/83108/fd (opendir: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83136 dashboard ceph cwd unknown /proc/32497/task/83136/cwd (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83136 dashboard ceph rtd unknown /proc/32497/task/83136/root (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83136 dashboard ceph txt unknown /proc/32497/task/83136/exe (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83136 dashboard ceph NOFD /proc/32497/task/83136/fd (opendir: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83137 dashboard ceph cwd unknown /proc/32497/task/83137/cwd (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83137 dashboard ceph rtd unknown /proc/32497/task/83137/root (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83137 dashboard ceph txt unknown /proc/32497/task/83137/exe (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83137 dashboard ceph NOFD /proc/32497/task/83137/fd (opendir: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83138 dashboard ceph cwd unknown /proc/32497/task/83138/cwd (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83138 dashboard ceph rtd unknown /proc/32497/task/83138/root (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83138 dashboard ceph txt unknown /proc/32497/task/83138/exe (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83138 dashboard ceph NOFD /proc/32497/task/83138/fd (opendir: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83139 dashboard ceph cwd unknown /proc/32497/task/83139/cwd (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83139 dashboard ceph rtd unknown /proc/32497/task/83139/root (readlink: Permission denied) 2024-03-12T04:17:28.448 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83139 dashboard ceph txt unknown /proc/32497/task/83139/exe (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83139 dashboard ceph NOFD /proc/32497/task/83139/fd (opendir: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83140 dashboard ceph cwd unknown /proc/32497/task/83140/cwd (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83140 dashboard ceph rtd unknown /proc/32497/task/83140/root (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83140 dashboard ceph txt unknown /proc/32497/task/83140/exe (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83140 dashboard ceph NOFD /proc/32497/task/83140/fd (opendir: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83142 dashboard ceph cwd unknown /proc/32497/task/83142/cwd (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83142 dashboard ceph rtd unknown /proc/32497/task/83142/root (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83142 dashboard ceph txt unknown /proc/32497/task/83142/exe (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83142 dashboard ceph NOFD /proc/32497/task/83142/fd (opendir: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83143 dashboard ceph cwd unknown /proc/32497/task/83143/cwd (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83143 dashboard ceph rtd unknown /proc/32497/task/83143/root (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83143 dashboard ceph txt unknown /proc/32497/task/83143/exe (readlink: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83143 dashboard ceph NOFD /proc/32497/task/83143/fd (opendir: Permission denied) 2024-03-12T04:17:28.449 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83144 dashboard ceph cwd unknown /proc/32497/task/83144/cwd (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83144 dashboard ceph rtd unknown /proc/32497/task/83144/root (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83144 dashboard ceph txt unknown /proc/32497/task/83144/exe (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83144 dashboard ceph NOFD /proc/32497/task/83144/fd (opendir: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83145 dashboard ceph cwd unknown /proc/32497/task/83145/cwd (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83145 dashboard ceph rtd unknown /proc/32497/task/83145/root (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83145 dashboard ceph txt unknown /proc/32497/task/83145/exe (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83145 dashboard ceph NOFD /proc/32497/task/83145/fd (opendir: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83147 dashboard ceph cwd unknown /proc/32497/task/83147/cwd (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83147 dashboard ceph rtd unknown /proc/32497/task/83147/root (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83147 dashboard ceph txt unknown /proc/32497/task/83147/exe (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83147 dashboard ceph NOFD /proc/32497/task/83147/fd (opendir: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83149 dashboard ceph cwd unknown /proc/32497/task/83149/cwd (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83149 dashboard ceph rtd unknown /proc/32497/task/83149/root (readlink: Permission denied) 2024-03-12T04:17:28.450 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83149 dashboard ceph txt unknown /proc/32497/task/83149/exe (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83149 dashboard ceph NOFD /proc/32497/task/83149/fd (opendir: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83213 dashboard ceph cwd unknown /proc/32497/task/83213/cwd (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83213 dashboard ceph rtd unknown /proc/32497/task/83213/root (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83213 dashboard ceph txt unknown /proc/32497/task/83213/exe (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83213 dashboard ceph NOFD /proc/32497/task/83213/fd (opendir: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83554 devicehea ceph cwd unknown /proc/32497/task/83554/cwd (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83554 devicehea ceph rtd unknown /proc/32497/task/83554/root (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83554 devicehea ceph txt unknown /proc/32497/task/83554/exe (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83554 devicehea ceph NOFD /proc/32497/task/83554/fd (opendir: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83564 io_contex ceph cwd unknown /proc/32497/task/83564/cwd (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83564 io_contex ceph rtd unknown /proc/32497/task/83564/root (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83564 io_contex ceph txt unknown /proc/32497/task/83564/exe (readlink: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83564 io_contex ceph NOFD /proc/32497/task/83564/fd (opendir: Permission denied) 2024-03-12T04:17:28.451 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83565 io_contex ceph cwd unknown /proc/32497/task/83565/cwd (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83565 io_contex ceph rtd unknown /proc/32497/task/83565/root (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83565 io_contex ceph txt unknown /proc/32497/task/83565/exe (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83565 io_contex ceph NOFD /proc/32497/task/83565/fd (opendir: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83566 ceph_time ceph cwd unknown /proc/32497/task/83566/cwd (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83566 ceph_time ceph rtd unknown /proc/32497/task/83566/root (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83566 ceph_time ceph txt unknown /proc/32497/task/83566/exe (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83566 ceph_time ceph NOFD /proc/32497/task/83566/fd (opendir: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83567 ms_dispat ceph cwd unknown /proc/32497/task/83567/cwd (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83567 ms_dispat ceph rtd unknown /proc/32497/task/83567/root (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83567 ms_dispat ceph txt unknown /proc/32497/task/83567/exe (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83567 ms_dispat ceph NOFD /proc/32497/task/83567/fd (opendir: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83568 ms_local ceph cwd unknown /proc/32497/task/83568/cwd (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83568 ms_local ceph rtd unknown /proc/32497/task/83568/root (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83568 ms_local ceph txt unknown /proc/32497/task/83568/exe (readlink: Permission denied) 2024-03-12T04:17:28.452 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83568 ms_local ceph NOFD /proc/32497/task/83568/fd (opendir: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83569 safe_time ceph cwd unknown /proc/32497/task/83569/cwd (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83569 safe_time ceph rtd unknown /proc/32497/task/83569/root (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83569 safe_time ceph txt unknown /proc/32497/task/83569/exe (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83569 safe_time ceph NOFD /proc/32497/task/83569/fd (opendir: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83570 safe_time ceph cwd unknown /proc/32497/task/83570/cwd (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83570 safe_time ceph rtd unknown /proc/32497/task/83570/root (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83570 safe_time ceph txt unknown /proc/32497/task/83570/exe (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83570 safe_time ceph NOFD /proc/32497/task/83570/fd (opendir: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83749 mgr-fin ceph cwd unknown /proc/32497/task/83749/cwd (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83749 mgr-fin ceph rtd unknown /proc/32497/task/83749/root (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83749 mgr-fin ceph txt unknown /proc/32497/task/83749/exe (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83749 mgr-fin ceph NOFD /proc/32497/task/83749/fd (opendir: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83750 mgr-fin ceph cwd unknown /proc/32497/task/83750/cwd (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83750 mgr-fin ceph rtd unknown /proc/32497/task/83750/root (readlink: Permission denied) 2024-03-12T04:17:28.453 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83750 mgr-fin ceph txt unknown /proc/32497/task/83750/exe (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83750 mgr-fin ceph NOFD /proc/32497/task/83750/fd (opendir: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83751 mgr-fin ceph cwd unknown /proc/32497/task/83751/cwd (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83751 mgr-fin ceph rtd unknown /proc/32497/task/83751/root (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83751 mgr-fin ceph txt unknown /proc/32497/task/83751/exe (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83751 mgr-fin ceph NOFD /proc/32497/task/83751/fd (opendir: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83752 mgr-fin ceph cwd unknown /proc/32497/task/83752/cwd (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83752 mgr-fin ceph rtd unknown /proc/32497/task/83752/root (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83752 mgr-fin ceph txt unknown /proc/32497/task/83752/exe (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83752 mgr-fin ceph NOFD /proc/32497/task/83752/fd (opendir: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83753 mgr-fin ceph cwd unknown /proc/32497/task/83753/cwd (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83753 mgr-fin ceph rtd unknown /proc/32497/task/83753/root (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83753 mgr-fin ceph txt unknown /proc/32497/task/83753/exe (readlink: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83753 mgr-fin ceph NOFD /proc/32497/task/83753/fd (opendir: Permission denied) 2024-03-12T04:17:28.454 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83754 mgr-fin ceph cwd unknown /proc/32497/task/83754/cwd (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83754 mgr-fin ceph rtd unknown /proc/32497/task/83754/root (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83754 mgr-fin ceph txt unknown /proc/32497/task/83754/exe (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83754 mgr-fin ceph NOFD /proc/32497/task/83754/fd (opendir: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83755 mgr-fin ceph cwd unknown /proc/32497/task/83755/cwd (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83755 mgr-fin ceph rtd unknown /proc/32497/task/83755/root (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83755 mgr-fin ceph txt unknown /proc/32497/task/83755/exe (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83755 mgr-fin ceph NOFD /proc/32497/task/83755/fd (opendir: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83756 mgr-fin ceph cwd unknown /proc/32497/task/83756/cwd (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83756 mgr-fin ceph rtd unknown /proc/32497/task/83756/root (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83756 mgr-fin ceph txt unknown /proc/32497/task/83756/exe (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83756 mgr-fin ceph NOFD /proc/32497/task/83756/fd (opendir: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83757 mgr-fin ceph cwd unknown /proc/32497/task/83757/cwd (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83757 mgr-fin ceph rtd unknown /proc/32497/task/83757/root (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83757 mgr-fin ceph txt unknown /proc/32497/task/83757/exe (readlink: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83757 mgr-fin ceph NOFD /proc/32497/task/83757/fd (opendir: Permission denied) 2024-03-12T04:17:28.455 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83758 mgr-fin ceph cwd unknown /proc/32497/task/83758/cwd (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83758 mgr-fin ceph rtd unknown /proc/32497/task/83758/root (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83758 mgr-fin ceph txt unknown /proc/32497/task/83758/exe (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83758 mgr-fin ceph NOFD /proc/32497/task/83758/fd (opendir: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83759 mgr-fin ceph cwd unknown /proc/32497/task/83759/cwd (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83759 mgr-fin ceph rtd unknown /proc/32497/task/83759/root (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83759 mgr-fin ceph txt unknown /proc/32497/task/83759/exe (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83759 mgr-fin ceph NOFD /proc/32497/task/83759/fd (opendir: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83761 mgr-fin ceph cwd unknown /proc/32497/task/83761/cwd (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83761 mgr-fin ceph rtd unknown /proc/32497/task/83761/root (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83761 mgr-fin ceph txt unknown /proc/32497/task/83761/exe (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83761 mgr-fin ceph NOFD /proc/32497/task/83761/fd (opendir: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83762 mgr-fin ceph cwd unknown /proc/32497/task/83762/cwd (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83762 mgr-fin ceph rtd unknown /proc/32497/task/83762/root (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83762 mgr-fin ceph txt unknown /proc/32497/task/83762/exe (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83762 mgr-fin ceph NOFD /proc/32497/task/83762/fd (opendir: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83763 mgr-fin ceph cwd unknown /proc/32497/task/83763/cwd (readlink: Permission denied) 2024-03-12T04:17:28.456 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83763 mgr-fin ceph rtd unknown /proc/32497/task/83763/root (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83763 mgr-fin ceph txt unknown /proc/32497/task/83763/exe (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83763 mgr-fin ceph NOFD /proc/32497/task/83763/fd (opendir: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83764 mgr-fin ceph cwd unknown /proc/32497/task/83764/cwd (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83764 mgr-fin ceph rtd unknown /proc/32497/task/83764/root (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83764 mgr-fin ceph txt unknown /proc/32497/task/83764/exe (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83764 mgr-fin ceph NOFD /proc/32497/task/83764/fd (opendir: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83765 mgr-fin ceph cwd unknown /proc/32497/task/83765/cwd (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83765 mgr-fin ceph rtd unknown /proc/32497/task/83765/root (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83765 mgr-fin ceph txt unknown /proc/32497/task/83765/exe (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83765 mgr-fin ceph NOFD /proc/32497/task/83765/fd (opendir: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83766 mgr-fin ceph cwd unknown /proc/32497/task/83766/cwd (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83766 mgr-fin ceph rtd unknown /proc/32497/task/83766/root (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83766 mgr-fin ceph txt unknown /proc/32497/task/83766/exe (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83766 mgr-fin ceph NOFD /proc/32497/task/83766/fd (opendir: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83767 mgr-fin ceph cwd unknown /proc/32497/task/83767/cwd (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83767 mgr-fin ceph rtd unknown /proc/32497/task/83767/root (readlink: Permission denied) 2024-03-12T04:17:28.457 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83767 mgr-fin ceph txt unknown /proc/32497/task/83767/exe (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83767 mgr-fin ceph NOFD /proc/32497/task/83767/fd (opendir: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83768 mgr-fin ceph cwd unknown /proc/32497/task/83768/cwd (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83768 mgr-fin ceph rtd unknown /proc/32497/task/83768/root (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83768 mgr-fin ceph txt unknown /proc/32497/task/83768/exe (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83768 mgr-fin ceph NOFD /proc/32497/task/83768/fd (opendir: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83769 mgr-fin ceph cwd unknown /proc/32497/task/83769/cwd (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83769 mgr-fin ceph rtd unknown /proc/32497/task/83769/root (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83769 mgr-fin ceph txt unknown /proc/32497/task/83769/exe (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83769 mgr-fin ceph NOFD /proc/32497/task/83769/fd (opendir: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83770 mgr-fin ceph cwd unknown /proc/32497/task/83770/cwd (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83770 mgr-fin ceph rtd unknown /proc/32497/task/83770/root (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83770 mgr-fin ceph txt unknown /proc/32497/task/83770/exe (readlink: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83770 mgr-fin ceph NOFD /proc/32497/task/83770/fd (opendir: Permission denied) 2024-03-12T04:17:28.458 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83771 mgr-fin ceph cwd unknown /proc/32497/task/83771/cwd (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83771 mgr-fin ceph rtd unknown /proc/32497/task/83771/root (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83771 mgr-fin ceph txt unknown /proc/32497/task/83771/exe (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 83771 mgr-fin ceph NOFD /proc/32497/task/83771/fd (opendir: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85367 ceph_time ceph cwd unknown /proc/32497/task/85367/cwd (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85367 ceph_time ceph rtd unknown /proc/32497/task/85367/root (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85367 ceph_time ceph txt unknown /proc/32497/task/85367/exe (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85367 ceph_time ceph NOFD /proc/32497/task/85367/fd (opendir: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85368 fn_anonym ceph cwd unknown /proc/32497/task/85368/cwd (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85368 fn_anonym ceph rtd unknown /proc/32497/task/85368/root (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85368 fn_anonym ceph txt unknown /proc/32497/task/85368/exe (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85368 fn_anonym ceph NOFD /proc/32497/task/85368/fd (opendir: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85369 safe_time ceph cwd unknown /proc/32497/task/85369/cwd (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85369 safe_time ceph rtd unknown /proc/32497/task/85369/root (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85369 safe_time ceph txt unknown /proc/32497/task/85369/exe (readlink: Permission denied) 2024-03-12T04:17:28.459 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85369 safe_time ceph NOFD /proc/32497/task/85369/fd (opendir: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85370 fn_anonym ceph cwd unknown /proc/32497/task/85370/cwd (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85370 fn_anonym ceph rtd unknown /proc/32497/task/85370/root (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85370 fn_anonym ceph txt unknown /proc/32497/task/85370/exe (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85370 fn_anonym ceph NOFD /proc/32497/task/85370/fd (opendir: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85371 flusher ceph cwd unknown /proc/32497/task/85371/cwd (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85371 flusher ceph rtd unknown /proc/32497/task/85371/root (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85371 flusher ceph txt unknown /proc/32497/task/85371/exe (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85371 flusher ceph NOFD /proc/32497/task/85371/fd (opendir: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85372 ms_dispat ceph cwd unknown /proc/32497/task/85372/cwd (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85372 ms_dispat ceph rtd unknown /proc/32497/task/85372/root (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85372 ms_dispat ceph txt unknown /proc/32497/task/85372/exe (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85372 ms_dispat ceph NOFD /proc/32497/task/85372/fd (opendir: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85373 ms_local ceph cwd unknown /proc/32497/task/85373/cwd (readlink: Permission denied) 2024-03-12T04:17:28.460 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85373 ms_local ceph rtd unknown /proc/32497/task/85373/root (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85373 ms_local ceph txt unknown /proc/32497/task/85373/exe (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85373 ms_local ceph NOFD /proc/32497/task/85373/fd (opendir: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85374 safe_time ceph cwd unknown /proc/32497/task/85374/cwd (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85374 safe_time ceph rtd unknown /proc/32497/task/85374/root (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85374 safe_time ceph txt unknown /proc/32497/task/85374/exe (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85374 safe_time ceph NOFD /proc/32497/task/85374/fd (opendir: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85375 m-fin-sna ceph cwd unknown /proc/32497/task/85375/cwd (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85375 m-fin-sna ceph rtd unknown /proc/32497/task/85375/root (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85375 m-fin-sna ceph txt unknown /proc/32497/task/85375/exe (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 85375 m-fin-sna ceph NOFD /proc/32497/task/85375/fd (opendir: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 86862 m-fin-sna ceph cwd unknown /proc/32497/task/86862/cwd (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 86862 m-fin-sna ceph rtd unknown /proc/32497/task/86862/root (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 86862 m-fin-sna ceph txt unknown /proc/32497/task/86862/exe (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mgr 32497 86862 m-fin-sna ceph NOFD /proc/32497/task/86862/fd (opendir: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 33581 root cwd unknown /proc/33581/cwd (readlink: Permission denied) 2024-03-12T04:17:28.461 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 33581 root rtd unknown /proc/33581/root (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 33581 root txt unknown /proc/33581/exe (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:systemd 33581 root NOFD /proc/33581/fd (opendir: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 33584 root cwd unknown /proc/33584/cwd (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 33584 root rtd unknown /proc/33584/root (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 33584 root txt unknown /proc/33584/exe (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:(sd-pam) 33584 root NOFD /proc/33584/fd (opendir: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 41200 root cwd unknown /proc/41200/cwd (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 41200 root rtd unknown /proc/41200/root (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 41200 root txt unknown /proc/41200/exe (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 41200 root NOFD /proc/41200/fd (opendir: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 44527 root cwd unknown /proc/44527/cwd (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 44527 root rtd unknown /proc/44527/root (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 44527 root txt unknown /proc/44527/exe (readlink: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 44527 root NOFD /proc/44527/fd (opendir: Permission denied) 2024-03-12T04:17:28.462 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 44529 root cwd unknown /proc/44529/cwd (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 44529 root rtd unknown /proc/44529/root (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 44529 root txt unknown /proc/44529/exe (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 44529 root NOFD /proc/44529/fd (opendir: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 ceph cwd unknown /proc/44531/cwd (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 ceph rtd unknown /proc/44531/root (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 ceph txt unknown /proc/44531/exe (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 ceph NOFD /proc/44531/fd (opendir: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44532 log ceph cwd unknown /proc/44531/task/44532/cwd (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44532 log ceph rtd unknown /proc/44531/task/44532/root (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44532 log ceph txt unknown /proc/44531/task/44532/exe (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44532 log ceph NOFD /proc/44531/task/44532/fd (opendir: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44534 msgr-work ceph cwd unknown /proc/44531/task/44534/cwd (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44534 msgr-work ceph rtd unknown /proc/44531/task/44534/root (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44534 msgr-work ceph txt unknown /proc/44531/task/44534/exe (readlink: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44534 msgr-work ceph NOFD /proc/44531/task/44534/fd (opendir: Permission denied) 2024-03-12T04:17:28.463 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44535 msgr-work ceph cwd unknown /proc/44531/task/44535/cwd (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44535 msgr-work ceph rtd unknown /proc/44531/task/44535/root (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44535 msgr-work ceph txt unknown /proc/44531/task/44535/exe (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44535 msgr-work ceph NOFD /proc/44531/task/44535/fd (opendir: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44536 msgr-work ceph cwd unknown /proc/44531/task/44536/cwd (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44536 msgr-work ceph rtd unknown /proc/44531/task/44536/root (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44536 msgr-work ceph txt unknown /proc/44531/task/44536/exe (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44536 msgr-work ceph NOFD /proc/44531/task/44536/fd (opendir: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44540 service ceph cwd unknown /proc/44531/task/44540/cwd (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44540 service ceph rtd unknown /proc/44531/task/44540/root (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44540 service ceph txt unknown /proc/44531/task/44540/exe (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44540 service ceph NOFD /proc/44531/task/44540/fd (opendir: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44541 admin_soc ceph cwd unknown /proc/44531/task/44541/cwd (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44541 admin_soc ceph rtd unknown /proc/44531/task/44541/root (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44541 admin_soc ceph txt unknown /proc/44531/task/44541/exe (readlink: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44541 admin_soc ceph NOFD /proc/44531/task/44541/fd (opendir: Permission denied) 2024-03-12T04:17:28.464 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44683 signal_ha ceph cwd unknown /proc/44531/task/44683/cwd (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44683 signal_ha ceph rtd unknown /proc/44531/task/44683/root (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44683 signal_ha ceph txt unknown /proc/44531/task/44683/exe (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44683 signal_ha ceph NOFD /proc/44531/task/44683/fd (opendir: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44684 io_contex ceph cwd unknown /proc/44531/task/44684/cwd (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44684 io_contex ceph rtd unknown /proc/44531/task/44684/root (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44684 io_contex ceph txt unknown /proc/44531/task/44684/exe (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44684 io_contex ceph NOFD /proc/44531/task/44684/fd (opendir: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44685 io_contex ceph cwd unknown /proc/44531/task/44685/cwd (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44685 io_contex ceph rtd unknown /proc/44531/task/44685/root (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44685 io_contex ceph txt unknown /proc/44531/task/44685/exe (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44685 io_contex ceph NOFD /proc/44531/task/44685/fd (opendir: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44801 OpHistory ceph cwd unknown /proc/44531/task/44801/cwd (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44801 OpHistory ceph rtd unknown /proc/44531/task/44801/root (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44801 OpHistory ceph txt unknown /proc/44531/task/44801/exe (readlink: Permission denied) 2024-03-12T04:17:28.465 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44801 OpHistory ceph NOFD /proc/44531/task/44801/fd (opendir: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44802 ceph_time ceph cwd unknown /proc/44531/task/44802/cwd (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44802 ceph_time ceph rtd unknown /proc/44531/task/44802/root (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44802 ceph_time ceph txt unknown /proc/44531/task/44802/exe (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44802 ceph_time ceph NOFD /proc/44531/task/44802/fd (opendir: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44948 ceph-osd ceph cwd unknown /proc/44531/task/44948/cwd (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44948 ceph-osd ceph rtd unknown /proc/44531/task/44948/root (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44948 ceph-osd ceph txt unknown /proc/44531/task/44948/exe (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44948 ceph-osd ceph NOFD /proc/44531/task/44948/fd (opendir: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44949 safe_time ceph cwd unknown /proc/44531/task/44949/cwd (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44949 safe_time ceph rtd unknown /proc/44531/task/44949/root (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44949 safe_time ceph txt unknown /proc/44531/task/44949/exe (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44949 safe_time ceph NOFD /proc/44531/task/44949/fd (opendir: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44950 safe_time ceph cwd unknown /proc/44531/task/44950/cwd (readlink: Permission denied) 2024-03-12T04:17:28.466 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44950 safe_time ceph rtd unknown /proc/44531/task/44950/root (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44950 safe_time ceph txt unknown /proc/44531/task/44950/exe (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44950 safe_time ceph NOFD /proc/44531/task/44950/fd (opendir: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44951 safe_time ceph cwd unknown /proc/44531/task/44951/cwd (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44951 safe_time ceph rtd unknown /proc/44531/task/44951/root (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44951 safe_time ceph txt unknown /proc/44531/task/44951/exe (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44951 safe_time ceph NOFD /proc/44531/task/44951/fd (opendir: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44953 safe_time ceph cwd unknown /proc/44531/task/44953/cwd (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44953 safe_time ceph rtd unknown /proc/44531/task/44953/root (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44953 safe_time ceph txt unknown /proc/44531/task/44953/exe (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44953 safe_time ceph NOFD /proc/44531/task/44953/fd (opendir: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44954 fn_anonym ceph cwd unknown /proc/44531/task/44954/cwd (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44954 fn_anonym ceph rtd unknown /proc/44531/task/44954/root (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44954 fn_anonym ceph txt unknown /proc/44531/task/44954/exe (readlink: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44954 fn_anonym ceph NOFD /proc/44531/task/44954/fd (opendir: Permission denied) 2024-03-12T04:17:28.467 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44957 rocksdb:l ceph cwd unknown /proc/44531/task/44957/cwd (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44957 rocksdb:l ceph rtd unknown /proc/44531/task/44957/root (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44957 rocksdb:l ceph txt unknown /proc/44531/task/44957/exe (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44957 rocksdb:l ceph NOFD /proc/44531/task/44957/fd (opendir: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44958 rocksdb:l ceph cwd unknown /proc/44531/task/44958/cwd (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44958 rocksdb:l ceph rtd unknown /proc/44531/task/44958/root (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44958 rocksdb:l ceph txt unknown /proc/44531/task/44958/exe (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44958 rocksdb:l ceph NOFD /proc/44531/task/44958/fd (opendir: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44959 rocksdb:l ceph cwd unknown /proc/44531/task/44959/cwd (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44959 rocksdb:l ceph rtd unknown /proc/44531/task/44959/root (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44959 rocksdb:l ceph txt unknown /proc/44531/task/44959/exe (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44959 rocksdb:l ceph NOFD /proc/44531/task/44959/fd (opendir: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44960 rocksdb:h ceph cwd unknown /proc/44531/task/44960/cwd (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44960 rocksdb:h ceph rtd unknown /proc/44531/task/44960/root (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44960 rocksdb:h ceph txt unknown /proc/44531/task/44960/exe (readlink: Permission denied) 2024-03-12T04:17:28.468 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 44960 rocksdb:h ceph NOFD /proc/44531/task/44960/fd (opendir: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45405 bstore_ai ceph cwd unknown /proc/44531/task/45405/cwd (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45405 bstore_ai ceph rtd unknown /proc/44531/task/45405/root (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45405 bstore_ai ceph txt unknown /proc/44531/task/45405/exe (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45405 bstore_ai ceph NOFD /proc/44531/task/45405/fd (opendir: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45612 bstore_ai ceph cwd unknown /proc/44531/task/45612/cwd (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45612 bstore_ai ceph rtd unknown /proc/44531/task/45612/root (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45612 bstore_ai ceph txt unknown /proc/44531/task/45612/exe (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45612 bstore_ai ceph NOFD /proc/44531/task/45612/fd (opendir: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45813 ceph-osd ceph cwd unknown /proc/44531/task/45813/cwd (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45813 ceph-osd ceph rtd unknown /proc/44531/task/45813/root (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45813 ceph-osd ceph txt unknown /proc/44531/task/45813/exe (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45813 ceph-osd ceph NOFD /proc/44531/task/45813/fd (opendir: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45815 cfin ceph cwd unknown /proc/44531/task/45815/cwd (readlink: Permission denied) 2024-03-12T04:17:28.469 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45815 cfin ceph rtd unknown /proc/44531/task/45815/root (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45815 cfin ceph txt unknown /proc/44531/task/45815/exe (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45815 cfin ceph NOFD /proc/44531/task/45815/fd (opendir: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45816 bstore_kv ceph cwd unknown /proc/44531/task/45816/cwd (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45816 bstore_kv ceph rtd unknown /proc/44531/task/45816/root (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45816 bstore_kv ceph txt unknown /proc/44531/task/45816/exe (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45816 bstore_kv ceph NOFD /proc/44531/task/45816/fd (opendir: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45817 bstore_kv ceph cwd unknown /proc/44531/task/45817/cwd (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45817 bstore_kv ceph rtd unknown /proc/44531/task/45817/root (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45817 bstore_kv ceph txt unknown /proc/44531/task/45817/exe (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45817 bstore_kv ceph NOFD /proc/44531/task/45817/fd (opendir: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45818 bstore_me ceph cwd unknown /proc/44531/task/45818/cwd (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45818 bstore_me ceph rtd unknown /proc/44531/task/45818/root (readlink: Permission denied) 2024-03-12T04:17:28.470 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45818 bstore_me ceph txt unknown /proc/44531/task/45818/exe (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45818 bstore_me ceph NOFD /proc/44531/task/45818/fd (opendir: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45835 ms_dispat ceph cwd unknown /proc/44531/task/45835/cwd (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45835 ms_dispat ceph rtd unknown /proc/44531/task/45835/root (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45835 ms_dispat ceph txt unknown /proc/44531/task/45835/exe (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45835 ms_dispat ceph NOFD /proc/44531/task/45835/fd (opendir: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45836 ms_local ceph cwd unknown /proc/44531/task/45836/cwd (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45836 ms_local ceph rtd unknown /proc/44531/task/45836/root (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45836 ms_local ceph txt unknown /proc/44531/task/45836/exe (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45836 ms_local ceph NOFD /proc/44531/task/45836/fd (opendir: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45837 safe_time ceph cwd unknown /proc/44531/task/45837/cwd (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45837 safe_time ceph rtd unknown /proc/44531/task/45837/root (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45837 safe_time ceph txt unknown /proc/44531/task/45837/exe (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45837 safe_time ceph NOFD /proc/44531/task/45837/fd (opendir: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45838 safe_time ceph cwd unknown /proc/44531/task/45838/cwd (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45838 safe_time ceph rtd unknown /proc/44531/task/45838/root (readlink: Permission denied) 2024-03-12T04:17:28.471 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45838 safe_time ceph txt unknown /proc/44531/task/45838/exe (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45838 safe_time ceph NOFD /proc/44531/task/45838/fd (opendir: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45839 ms_dispat ceph cwd unknown /proc/44531/task/45839/cwd (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45839 ms_dispat ceph rtd unknown /proc/44531/task/45839/root (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45839 ms_dispat ceph txt unknown /proc/44531/task/45839/exe (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45839 ms_dispat ceph NOFD /proc/44531/task/45839/fd (opendir: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45840 ms_local ceph cwd unknown /proc/44531/task/45840/cwd (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45840 ms_local ceph rtd unknown /proc/44531/task/45840/root (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45840 ms_local ceph txt unknown /proc/44531/task/45840/exe (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45840 ms_local ceph NOFD /proc/44531/task/45840/fd (opendir: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45841 ms_dispat ceph cwd unknown /proc/44531/task/45841/cwd (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45841 ms_dispat ceph rtd unknown /proc/44531/task/45841/root (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45841 ms_dispat ceph txt unknown /proc/44531/task/45841/exe (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45841 ms_dispat ceph NOFD /proc/44531/task/45841/fd (opendir: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45842 ms_local ceph cwd unknown /proc/44531/task/45842/cwd (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45842 ms_local ceph rtd unknown /proc/44531/task/45842/root (readlink: Permission denied) 2024-03-12T04:17:28.472 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45842 ms_local ceph txt unknown /proc/44531/task/45842/exe (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45842 ms_local ceph NOFD /proc/44531/task/45842/fd (opendir: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45843 ms_dispat ceph cwd unknown /proc/44531/task/45843/cwd (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45843 ms_dispat ceph rtd unknown /proc/44531/task/45843/root (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45843 ms_dispat ceph txt unknown /proc/44531/task/45843/exe (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45843 ms_dispat ceph NOFD /proc/44531/task/45843/fd (opendir: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45844 ms_local ceph cwd unknown /proc/44531/task/45844/cwd (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45844 ms_local ceph rtd unknown /proc/44531/task/45844/root (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45844 ms_local ceph txt unknown /proc/44531/task/45844/exe (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45844 ms_local ceph NOFD /proc/44531/task/45844/fd (opendir: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45845 ms_dispat ceph cwd unknown /proc/44531/task/45845/cwd (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45845 ms_dispat ceph rtd unknown /proc/44531/task/45845/root (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45845 ms_dispat ceph txt unknown /proc/44531/task/45845/exe (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45845 ms_dispat ceph NOFD /proc/44531/task/45845/fd (opendir: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45846 ms_local ceph cwd unknown /proc/44531/task/45846/cwd (readlink: Permission denied) 2024-03-12T04:17:28.473 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45846 ms_local ceph rtd unknown /proc/44531/task/45846/root (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45846 ms_local ceph txt unknown /proc/44531/task/45846/exe (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45846 ms_local ceph NOFD /proc/44531/task/45846/fd (opendir: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45847 ms_dispat ceph cwd unknown /proc/44531/task/45847/cwd (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45847 ms_dispat ceph rtd unknown /proc/44531/task/45847/root (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45847 ms_dispat ceph txt unknown /proc/44531/task/45847/exe (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45847 ms_dispat ceph NOFD /proc/44531/task/45847/fd (opendir: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45848 ms_local ceph cwd unknown /proc/44531/task/45848/cwd (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45848 ms_local ceph rtd unknown /proc/44531/task/45848/root (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45848 ms_local ceph txt unknown /proc/44531/task/45848/exe (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45848 ms_local ceph NOFD /proc/44531/task/45848/fd (opendir: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45849 ms_dispat ceph cwd unknown /proc/44531/task/45849/cwd (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45849 ms_dispat ceph rtd unknown /proc/44531/task/45849/root (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45849 ms_dispat ceph txt unknown /proc/44531/task/45849/exe (readlink: Permission denied) 2024-03-12T04:17:28.474 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45849 ms_dispat ceph NOFD /proc/44531/task/45849/fd (opendir: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45850 ms_local ceph cwd unknown /proc/44531/task/45850/cwd (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45850 ms_local ceph rtd unknown /proc/44531/task/45850/root (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45850 ms_local ceph txt unknown /proc/44531/task/45850/exe (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45850 ms_local ceph NOFD /proc/44531/task/45850/fd (opendir: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45851 fn_anonym ceph cwd unknown /proc/44531/task/45851/cwd (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45851 fn_anonym ceph rtd unknown /proc/44531/task/45851/root (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45851 fn_anonym ceph txt unknown /proc/44531/task/45851/exe (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45851 fn_anonym ceph NOFD /proc/44531/task/45851/fd (opendir: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45852 finisher ceph cwd unknown /proc/44531/task/45852/cwd (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45852 finisher ceph rtd unknown /proc/44531/task/45852/root (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45852 finisher ceph txt unknown /proc/44531/task/45852/exe (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45852 finisher ceph NOFD /proc/44531/task/45852/fd (opendir: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45853 safe_time ceph cwd unknown /proc/44531/task/45853/cwd (readlink: Permission denied) 2024-03-12T04:17:28.475 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45853 safe_time ceph rtd unknown /proc/44531/task/45853/root (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45853 safe_time ceph txt unknown /proc/44531/task/45853/exe (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45853 safe_time ceph NOFD /proc/44531/task/45853/fd (opendir: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45854 safe_time ceph cwd unknown /proc/44531/task/45854/cwd (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45854 safe_time ceph rtd unknown /proc/44531/task/45854/root (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45854 safe_time ceph txt unknown /proc/44531/task/45854/exe (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45854 safe_time ceph NOFD /proc/44531/task/45854/fd (opendir: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45855 ceph-osd ceph cwd unknown /proc/44531/task/45855/cwd (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45855 ceph-osd ceph rtd unknown /proc/44531/task/45855/root (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45855 ceph-osd ceph txt unknown /proc/44531/task/45855/exe (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45855 ceph-osd ceph NOFD /proc/44531/task/45855/fd (opendir: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45856 osd_srv_a ceph cwd unknown /proc/44531/task/45856/cwd (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45856 osd_srv_a ceph rtd unknown /proc/44531/task/45856/root (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45856 osd_srv_a ceph txt unknown /proc/44531/task/45856/exe (readlink: Permission denied) 2024-03-12T04:17:28.476 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45856 osd_srv_a ceph NOFD /proc/44531/task/45856/fd (opendir: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45857 tp_osd_tp ceph cwd unknown /proc/44531/task/45857/cwd (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45857 tp_osd_tp ceph rtd unknown /proc/44531/task/45857/root (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45857 tp_osd_tp ceph txt unknown /proc/44531/task/45857/exe (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45857 tp_osd_tp ceph NOFD /proc/44531/task/45857/fd (opendir: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45858 tp_osd_tp ceph cwd unknown /proc/44531/task/45858/cwd (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45858 tp_osd_tp ceph rtd unknown /proc/44531/task/45858/root (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45858 tp_osd_tp ceph txt unknown /proc/44531/task/45858/exe (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45858 tp_osd_tp ceph NOFD /proc/44531/task/45858/fd (opendir: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45859 tp_osd_tp ceph cwd unknown /proc/44531/task/45859/cwd (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45859 tp_osd_tp ceph rtd unknown /proc/44531/task/45859/root (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45859 tp_osd_tp ceph txt unknown /proc/44531/task/45859/exe (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45859 tp_osd_tp ceph NOFD /proc/44531/task/45859/fd (opendir: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45860 tp_osd_tp ceph cwd unknown /proc/44531/task/45860/cwd (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45860 tp_osd_tp ceph rtd unknown /proc/44531/task/45860/root (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45860 tp_osd_tp ceph txt unknown /proc/44531/task/45860/exe (readlink: Permission denied) 2024-03-12T04:17:28.477 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45860 tp_osd_tp ceph NOFD /proc/44531/task/45860/fd (opendir: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45861 tp_osd_tp ceph cwd unknown /proc/44531/task/45861/cwd (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45861 tp_osd_tp ceph rtd unknown /proc/44531/task/45861/root (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45861 tp_osd_tp ceph txt unknown /proc/44531/task/45861/exe (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45861 tp_osd_tp ceph NOFD /proc/44531/task/45861/fd (opendir: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45862 tp_osd_tp ceph cwd unknown /proc/44531/task/45862/cwd (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45862 tp_osd_tp ceph rtd unknown /proc/44531/task/45862/root (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45862 tp_osd_tp ceph txt unknown /proc/44531/task/45862/exe (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45862 tp_osd_tp ceph NOFD /proc/44531/task/45862/fd (opendir: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45863 tp_osd_tp ceph cwd unknown /proc/44531/task/45863/cwd (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45863 tp_osd_tp ceph rtd unknown /proc/44531/task/45863/root (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45863 tp_osd_tp ceph txt unknown /proc/44531/task/45863/exe (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45863 tp_osd_tp ceph NOFD /proc/44531/task/45863/fd (opendir: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45864 tp_osd_tp ceph cwd unknown /proc/44531/task/45864/cwd (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45864 tp_osd_tp ceph rtd unknown /proc/44531/task/45864/root (readlink: Permission denied) 2024-03-12T04:17:28.478 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45864 tp_osd_tp ceph txt unknown /proc/44531/task/45864/exe (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45864 tp_osd_tp ceph NOFD /proc/44531/task/45864/fd (opendir: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45865 tp_osd_tp ceph cwd unknown /proc/44531/task/45865/cwd (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45865 tp_osd_tp ceph rtd unknown /proc/44531/task/45865/root (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45865 tp_osd_tp ceph txt unknown /proc/44531/task/45865/exe (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45865 tp_osd_tp ceph NOFD /proc/44531/task/45865/fd (opendir: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45866 tp_osd_tp ceph cwd unknown /proc/44531/task/45866/cwd (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45866 tp_osd_tp ceph rtd unknown /proc/44531/task/45866/root (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45866 tp_osd_tp ceph txt unknown /proc/44531/task/45866/exe (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45866 tp_osd_tp ceph NOFD /proc/44531/task/45866/fd (opendir: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45867 tp_osd_tp ceph cwd unknown /proc/44531/task/45867/cwd (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45867 tp_osd_tp ceph rtd unknown /proc/44531/task/45867/root (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45867 tp_osd_tp ceph txt unknown /proc/44531/task/45867/exe (readlink: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45867 tp_osd_tp ceph NOFD /proc/44531/task/45867/fd (opendir: Permission denied) 2024-03-12T04:17:28.479 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45868 tp_osd_tp ceph cwd unknown /proc/44531/task/45868/cwd (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45868 tp_osd_tp ceph rtd unknown /proc/44531/task/45868/root (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45868 tp_osd_tp ceph txt unknown /proc/44531/task/45868/exe (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45868 tp_osd_tp ceph NOFD /proc/44531/task/45868/fd (opendir: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45869 tp_osd_tp ceph cwd unknown /proc/44531/task/45869/cwd (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45869 tp_osd_tp ceph rtd unknown /proc/44531/task/45869/root (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45869 tp_osd_tp ceph txt unknown /proc/44531/task/45869/exe (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45869 tp_osd_tp ceph NOFD /proc/44531/task/45869/fd (opendir: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45870 tp_osd_tp ceph cwd unknown /proc/44531/task/45870/cwd (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45870 tp_osd_tp ceph rtd unknown /proc/44531/task/45870/root (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45870 tp_osd_tp ceph txt unknown /proc/44531/task/45870/exe (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45870 tp_osd_tp ceph NOFD /proc/44531/task/45870/fd (opendir: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45871 tp_osd_tp ceph cwd unknown /proc/44531/task/45871/cwd (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45871 tp_osd_tp ceph rtd unknown /proc/44531/task/45871/root (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45871 tp_osd_tp ceph txt unknown /proc/44531/task/45871/exe (readlink: Permission denied) 2024-03-12T04:17:28.480 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45871 tp_osd_tp ceph NOFD /proc/44531/task/45871/fd (opendir: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45872 tp_osd_tp ceph cwd unknown /proc/44531/task/45872/cwd (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45872 tp_osd_tp ceph rtd unknown /proc/44531/task/45872/root (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45872 tp_osd_tp ceph txt unknown /proc/44531/task/45872/exe (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45872 tp_osd_tp ceph NOFD /proc/44531/task/45872/fd (opendir: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45873 osd_srv_h ceph cwd unknown /proc/44531/task/45873/cwd (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45873 osd_srv_h ceph rtd unknown /proc/44531/task/45873/root (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45873 osd_srv_h ceph txt unknown /proc/44531/task/45873/exe (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 44531 45873 osd_srv_h ceph NOFD /proc/44531/task/45873/fd (opendir: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 45909 root cwd unknown /proc/45909/cwd (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 45909 root rtd unknown /proc/45909/root (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 45909 root txt unknown /proc/45909/exe (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 45909 root NOFD /proc/45909/fd (opendir: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 45940 root cwd unknown /proc/45940/cwd (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 45940 root rtd unknown /proc/45940/root (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 45940 root txt unknown /proc/45940/exe (readlink: Permission denied) 2024-03-12T04:17:28.481 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 45940 root NOFD /proc/45940/fd (opendir: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 51351 root cwd unknown /proc/51351/cwd (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 51351 root rtd unknown /proc/51351/root (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 51351 root txt unknown /proc/51351/exe (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 51351 root NOFD /proc/51351/fd (opendir: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 51353 root cwd unknown /proc/51353/cwd (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 51353 root rtd unknown /proc/51353/root (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 51353 root txt unknown /proc/51353/exe (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 51353 root NOFD /proc/51353/fd (opendir: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 ceph cwd unknown /proc/51355/cwd (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 ceph rtd unknown /proc/51355/root (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 ceph txt unknown /proc/51355/exe (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 ceph NOFD /proc/51355/fd (opendir: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51356 log ceph cwd unknown /proc/51355/task/51356/cwd (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51356 log ceph rtd unknown /proc/51355/task/51356/root (readlink: Permission denied) 2024-03-12T04:17:28.482 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51356 log ceph txt unknown /proc/51355/task/51356/exe (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51356 log ceph NOFD /proc/51355/task/51356/fd (opendir: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51358 msgr-work ceph cwd unknown /proc/51355/task/51358/cwd (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51358 msgr-work ceph rtd unknown /proc/51355/task/51358/root (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51358 msgr-work ceph txt unknown /proc/51355/task/51358/exe (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51358 msgr-work ceph NOFD /proc/51355/task/51358/fd (opendir: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51359 msgr-work ceph cwd unknown /proc/51355/task/51359/cwd (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51359 msgr-work ceph rtd unknown /proc/51355/task/51359/root (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51359 msgr-work ceph txt unknown /proc/51355/task/51359/exe (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51359 msgr-work ceph NOFD /proc/51355/task/51359/fd (opendir: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51360 msgr-work ceph cwd unknown /proc/51355/task/51360/cwd (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51360 msgr-work ceph rtd unknown /proc/51355/task/51360/root (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51360 msgr-work ceph txt unknown /proc/51355/task/51360/exe (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51360 msgr-work ceph NOFD /proc/51355/task/51360/fd (opendir: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51364 service ceph cwd unknown /proc/51355/task/51364/cwd (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51364 service ceph rtd unknown /proc/51355/task/51364/root (readlink: Permission denied) 2024-03-12T04:17:28.483 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51364 service ceph txt unknown /proc/51355/task/51364/exe (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51364 service ceph NOFD /proc/51355/task/51364/fd (opendir: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51365 admin_soc ceph cwd unknown /proc/51355/task/51365/cwd (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51365 admin_soc ceph rtd unknown /proc/51355/task/51365/root (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51365 admin_soc ceph txt unknown /proc/51355/task/51365/exe (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51365 admin_soc ceph NOFD /proc/51355/task/51365/fd (opendir: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51507 signal_ha ceph cwd unknown /proc/51355/task/51507/cwd (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51507 signal_ha ceph rtd unknown /proc/51355/task/51507/root (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51507 signal_ha ceph txt unknown /proc/51355/task/51507/exe (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51507 signal_ha ceph NOFD /proc/51355/task/51507/fd (opendir: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51508 io_contex ceph cwd unknown /proc/51355/task/51508/cwd (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51508 io_contex ceph rtd unknown /proc/51355/task/51508/root (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51508 io_contex ceph txt unknown /proc/51355/task/51508/exe (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51508 io_contex ceph NOFD /proc/51355/task/51508/fd (opendir: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51509 io_contex ceph cwd unknown /proc/51355/task/51509/cwd (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51509 io_contex ceph rtd unknown /proc/51355/task/51509/root (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51509 io_contex ceph txt unknown /proc/51355/task/51509/exe (readlink: Permission denied) 2024-03-12T04:17:28.484 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51509 io_contex ceph NOFD /proc/51355/task/51509/fd (opendir: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51623 OpHistory ceph cwd unknown /proc/51355/task/51623/cwd (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51623 OpHistory ceph rtd unknown /proc/51355/task/51623/root (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51623 OpHistory ceph txt unknown /proc/51355/task/51623/exe (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51623 OpHistory ceph NOFD /proc/51355/task/51623/fd (opendir: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51624 ceph_time ceph cwd unknown /proc/51355/task/51624/cwd (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51624 ceph_time ceph rtd unknown /proc/51355/task/51624/root (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51624 ceph_time ceph txt unknown /proc/51355/task/51624/exe (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51624 ceph_time ceph NOFD /proc/51355/task/51624/fd (opendir: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51660 ceph-osd ceph cwd unknown /proc/51355/task/51660/cwd (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51660 ceph-osd ceph rtd unknown /proc/51355/task/51660/root (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51660 ceph-osd ceph txt unknown /proc/51355/task/51660/exe (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51660 ceph-osd ceph NOFD /proc/51355/task/51660/fd (opendir: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51664 ceph-osd ceph cwd unknown /proc/51355/task/51664/cwd (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51664 ceph-osd ceph rtd unknown /proc/51355/task/51664/root (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51664 ceph-osd ceph txt unknown /proc/51355/task/51664/exe (readlink: Permission denied) 2024-03-12T04:17:28.485 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51664 ceph-osd ceph NOFD /proc/51355/task/51664/fd (opendir: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51685 ceph-osd ceph cwd unknown /proc/51355/task/51685/cwd (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51685 ceph-osd ceph rtd unknown /proc/51355/task/51685/root (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51685 ceph-osd ceph txt unknown /proc/51355/task/51685/exe (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51685 ceph-osd ceph NOFD /proc/51355/task/51685/fd (opendir: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51707 ceph-osd ceph cwd unknown /proc/51355/task/51707/cwd (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51707 ceph-osd ceph rtd unknown /proc/51355/task/51707/root (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51707 ceph-osd ceph txt unknown /proc/51355/task/51707/exe (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51707 ceph-osd ceph NOFD /proc/51355/task/51707/fd (opendir: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51730 ceph-osd ceph cwd unknown /proc/51355/task/51730/cwd (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51730 ceph-osd ceph rtd unknown /proc/51355/task/51730/root (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51730 ceph-osd ceph txt unknown /proc/51355/task/51730/exe (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51730 ceph-osd ceph NOFD /proc/51355/task/51730/fd (opendir: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51738 ceph-osd ceph cwd unknown /proc/51355/task/51738/cwd (readlink: Permission denied) 2024-03-12T04:17:28.486 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51738 ceph-osd ceph rtd unknown /proc/51355/task/51738/root (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51738 ceph-osd ceph txt unknown /proc/51355/task/51738/exe (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51738 ceph-osd ceph NOFD /proc/51355/task/51738/fd (opendir: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51765 ceph-osd ceph cwd unknown /proc/51355/task/51765/cwd (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51765 ceph-osd ceph rtd unknown /proc/51355/task/51765/root (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51765 ceph-osd ceph txt unknown /proc/51355/task/51765/exe (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51765 ceph-osd ceph NOFD /proc/51355/task/51765/fd (opendir: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51804 ceph-osd ceph cwd unknown /proc/51355/task/51804/cwd (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51804 ceph-osd ceph rtd unknown /proc/51355/task/51804/root (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51804 ceph-osd ceph txt unknown /proc/51355/task/51804/exe (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51804 ceph-osd ceph NOFD /proc/51355/task/51804/fd (opendir: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51806 ceph-osd ceph cwd unknown /proc/51355/task/51806/cwd (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51806 ceph-osd ceph rtd unknown /proc/51355/task/51806/root (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51806 ceph-osd ceph txt unknown /proc/51355/task/51806/exe (readlink: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51806 ceph-osd ceph NOFD /proc/51355/task/51806/fd (opendir: Permission denied) 2024-03-12T04:17:28.487 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51807 safe_time ceph cwd unknown /proc/51355/task/51807/cwd (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51807 safe_time ceph rtd unknown /proc/51355/task/51807/root (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51807 safe_time ceph txt unknown /proc/51355/task/51807/exe (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51807 safe_time ceph NOFD /proc/51355/task/51807/fd (opendir: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51808 safe_time ceph cwd unknown /proc/51355/task/51808/cwd (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51808 safe_time ceph rtd unknown /proc/51355/task/51808/root (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51808 safe_time ceph txt unknown /proc/51355/task/51808/exe (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51808 safe_time ceph NOFD /proc/51355/task/51808/fd (opendir: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51809 safe_time ceph cwd unknown /proc/51355/task/51809/cwd (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51809 safe_time ceph rtd unknown /proc/51355/task/51809/root (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51809 safe_time ceph txt unknown /proc/51355/task/51809/exe (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51809 safe_time ceph NOFD /proc/51355/task/51809/fd (opendir: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51810 safe_time ceph cwd unknown /proc/51355/task/51810/cwd (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51810 safe_time ceph rtd unknown /proc/51355/task/51810/root (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51810 safe_time ceph txt unknown /proc/51355/task/51810/exe (readlink: Permission denied) 2024-03-12T04:17:28.488 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51810 safe_time ceph NOFD /proc/51355/task/51810/fd (opendir: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51811 fn_anonym ceph cwd unknown /proc/51355/task/51811/cwd (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51811 fn_anonym ceph rtd unknown /proc/51355/task/51811/root (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51811 fn_anonym ceph txt unknown /proc/51355/task/51811/exe (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51811 fn_anonym ceph NOFD /proc/51355/task/51811/fd (opendir: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51814 rocksdb:l ceph cwd unknown /proc/51355/task/51814/cwd (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51814 rocksdb:l ceph rtd unknown /proc/51355/task/51814/root (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51814 rocksdb:l ceph txt unknown /proc/51355/task/51814/exe (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51814 rocksdb:l ceph NOFD /proc/51355/task/51814/fd (opendir: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51815 rocksdb:l ceph cwd unknown /proc/51355/task/51815/cwd (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51815 rocksdb:l ceph rtd unknown /proc/51355/task/51815/root (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51815 rocksdb:l ceph txt unknown /proc/51355/task/51815/exe (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51815 rocksdb:l ceph NOFD /proc/51355/task/51815/fd (opendir: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51816 rocksdb:l ceph cwd unknown /proc/51355/task/51816/cwd (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51816 rocksdb:l ceph rtd unknown /proc/51355/task/51816/root (readlink: Permission denied) 2024-03-12T04:17:28.489 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51816 rocksdb:l ceph txt unknown /proc/51355/task/51816/exe (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51816 rocksdb:l ceph NOFD /proc/51355/task/51816/fd (opendir: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51817 rocksdb:h ceph cwd unknown /proc/51355/task/51817/cwd (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51817 rocksdb:h ceph rtd unknown /proc/51355/task/51817/root (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51817 rocksdb:h ceph txt unknown /proc/51355/task/51817/exe (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 51817 rocksdb:h ceph NOFD /proc/51355/task/51817/fd (opendir: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52257 bstore_ai ceph cwd unknown /proc/51355/task/52257/cwd (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52257 bstore_ai ceph rtd unknown /proc/51355/task/52257/root (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52257 bstore_ai ceph txt unknown /proc/51355/task/52257/exe (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52257 bstore_ai ceph NOFD /proc/51355/task/52257/fd (opendir: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52497 bstore_ai ceph cwd unknown /proc/51355/task/52497/cwd (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52497 bstore_ai ceph rtd unknown /proc/51355/task/52497/root (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52497 bstore_ai ceph txt unknown /proc/51355/task/52497/exe (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52497 bstore_ai ceph NOFD /proc/51355/task/52497/fd (opendir: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52690 ceph-osd ceph cwd unknown /proc/51355/task/52690/cwd (readlink: Permission denied) 2024-03-12T04:17:28.490 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52690 ceph-osd ceph rtd unknown /proc/51355/task/52690/root (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52690 ceph-osd ceph txt unknown /proc/51355/task/52690/exe (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52690 ceph-osd ceph NOFD /proc/51355/task/52690/fd (opendir: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52691 cfin ceph cwd unknown /proc/51355/task/52691/cwd (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52691 cfin ceph rtd unknown /proc/51355/task/52691/root (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52691 cfin ceph txt unknown /proc/51355/task/52691/exe (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52691 cfin ceph NOFD /proc/51355/task/52691/fd (opendir: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52692 bstore_kv ceph cwd unknown /proc/51355/task/52692/cwd (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52692 bstore_kv ceph rtd unknown /proc/51355/task/52692/root (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52692 bstore_kv ceph txt unknown /proc/51355/task/52692/exe (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52692 bstore_kv ceph NOFD /proc/51355/task/52692/fd (opendir: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52693 bstore_kv ceph cwd unknown /proc/51355/task/52693/cwd (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52693 bstore_kv ceph rtd unknown /proc/51355/task/52693/root (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52693 bstore_kv ceph txt unknown /proc/51355/task/52693/exe (readlink: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52693 bstore_kv ceph NOFD /proc/51355/task/52693/fd (opendir: Permission denied) 2024-03-12T04:17:28.491 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52694 bstore_me ceph cwd unknown /proc/51355/task/52694/cwd (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52694 bstore_me ceph rtd unknown /proc/51355/task/52694/root (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52694 bstore_me ceph txt unknown /proc/51355/task/52694/exe (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52694 bstore_me ceph NOFD /proc/51355/task/52694/fd (opendir: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52705 ms_dispat ceph cwd unknown /proc/51355/task/52705/cwd (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52705 ms_dispat ceph rtd unknown /proc/51355/task/52705/root (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52705 ms_dispat ceph txt unknown /proc/51355/task/52705/exe (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52705 ms_dispat ceph NOFD /proc/51355/task/52705/fd (opendir: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52706 ms_local ceph cwd unknown /proc/51355/task/52706/cwd (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52706 ms_local ceph rtd unknown /proc/51355/task/52706/root (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52706 ms_local ceph txt unknown /proc/51355/task/52706/exe (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52706 ms_local ceph NOFD /proc/51355/task/52706/fd (opendir: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52707 safe_time ceph cwd unknown /proc/51355/task/52707/cwd (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52707 safe_time ceph rtd unknown /proc/51355/task/52707/root (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52707 safe_time ceph txt unknown /proc/51355/task/52707/exe (readlink: Permission denied) 2024-03-12T04:17:28.492 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52707 safe_time ceph NOFD /proc/51355/task/52707/fd (opendir: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52708 safe_time ceph cwd unknown /proc/51355/task/52708/cwd (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52708 safe_time ceph rtd unknown /proc/51355/task/52708/root (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52708 safe_time ceph txt unknown /proc/51355/task/52708/exe (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52708 safe_time ceph NOFD /proc/51355/task/52708/fd (opendir: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52709 ms_dispat ceph cwd unknown /proc/51355/task/52709/cwd (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52709 ms_dispat ceph rtd unknown /proc/51355/task/52709/root (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52709 ms_dispat ceph txt unknown /proc/51355/task/52709/exe (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52709 ms_dispat ceph NOFD /proc/51355/task/52709/fd (opendir: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52710 ms_local ceph cwd unknown /proc/51355/task/52710/cwd (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52710 ms_local ceph rtd unknown /proc/51355/task/52710/root (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52710 ms_local ceph txt unknown /proc/51355/task/52710/exe (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52710 ms_local ceph NOFD /proc/51355/task/52710/fd (opendir: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52711 ms_dispat ceph cwd unknown /proc/51355/task/52711/cwd (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52711 ms_dispat ceph rtd unknown /proc/51355/task/52711/root (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52711 ms_dispat ceph txt unknown /proc/51355/task/52711/exe (readlink: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52711 ms_dispat ceph NOFD /proc/51355/task/52711/fd (opendir: Permission denied) 2024-03-12T04:17:28.493 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52712 ms_local ceph cwd unknown /proc/51355/task/52712/cwd (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52712 ms_local ceph rtd unknown /proc/51355/task/52712/root (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52712 ms_local ceph txt unknown /proc/51355/task/52712/exe (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52712 ms_local ceph NOFD /proc/51355/task/52712/fd (opendir: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52713 ms_dispat ceph cwd unknown /proc/51355/task/52713/cwd (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52713 ms_dispat ceph rtd unknown /proc/51355/task/52713/root (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52713 ms_dispat ceph txt unknown /proc/51355/task/52713/exe (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52713 ms_dispat ceph NOFD /proc/51355/task/52713/fd (opendir: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52714 ms_local ceph cwd unknown /proc/51355/task/52714/cwd (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52714 ms_local ceph rtd unknown /proc/51355/task/52714/root (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52714 ms_local ceph txt unknown /proc/51355/task/52714/exe (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52714 ms_local ceph NOFD /proc/51355/task/52714/fd (opendir: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52715 ms_dispat ceph cwd unknown /proc/51355/task/52715/cwd (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52715 ms_dispat ceph rtd unknown /proc/51355/task/52715/root (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52715 ms_dispat ceph txt unknown /proc/51355/task/52715/exe (readlink: Permission denied) 2024-03-12T04:17:28.494 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52715 ms_dispat ceph NOFD /proc/51355/task/52715/fd (opendir: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52716 ms_local ceph cwd unknown /proc/51355/task/52716/cwd (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52716 ms_local ceph rtd unknown /proc/51355/task/52716/root (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52716 ms_local ceph txt unknown /proc/51355/task/52716/exe (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52716 ms_local ceph NOFD /proc/51355/task/52716/fd (opendir: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52717 ms_dispat ceph cwd unknown /proc/51355/task/52717/cwd (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52717 ms_dispat ceph rtd unknown /proc/51355/task/52717/root (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52717 ms_dispat ceph txt unknown /proc/51355/task/52717/exe (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52717 ms_dispat ceph NOFD /proc/51355/task/52717/fd (opendir: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52718 ms_local ceph cwd unknown /proc/51355/task/52718/cwd (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52718 ms_local ceph rtd unknown /proc/51355/task/52718/root (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52718 ms_local ceph txt unknown /proc/51355/task/52718/exe (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52718 ms_local ceph NOFD /proc/51355/task/52718/fd (opendir: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52719 ms_dispat ceph cwd unknown /proc/51355/task/52719/cwd (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52719 ms_dispat ceph rtd unknown /proc/51355/task/52719/root (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52719 ms_dispat ceph txt unknown /proc/51355/task/52719/exe (readlink: Permission denied) 2024-03-12T04:17:28.495 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52719 ms_dispat ceph NOFD /proc/51355/task/52719/fd (opendir: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52720 ms_local ceph cwd unknown /proc/51355/task/52720/cwd (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52720 ms_local ceph rtd unknown /proc/51355/task/52720/root (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52720 ms_local ceph txt unknown /proc/51355/task/52720/exe (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52720 ms_local ceph NOFD /proc/51355/task/52720/fd (opendir: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52721 fn_anonym ceph cwd unknown /proc/51355/task/52721/cwd (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52721 fn_anonym ceph rtd unknown /proc/51355/task/52721/root (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52721 fn_anonym ceph txt unknown /proc/51355/task/52721/exe (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52721 fn_anonym ceph NOFD /proc/51355/task/52721/fd (opendir: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52722 finisher ceph cwd unknown /proc/51355/task/52722/cwd (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52722 finisher ceph rtd unknown /proc/51355/task/52722/root (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52722 finisher ceph txt unknown /proc/51355/task/52722/exe (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52722 finisher ceph NOFD /proc/51355/task/52722/fd (opendir: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52723 safe_time ceph cwd unknown /proc/51355/task/52723/cwd (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52723 safe_time ceph rtd unknown /proc/51355/task/52723/root (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52723 safe_time ceph txt unknown /proc/51355/task/52723/exe (readlink: Permission denied) 2024-03-12T04:17:28.496 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52723 safe_time ceph NOFD /proc/51355/task/52723/fd (opendir: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52724 safe_time ceph cwd unknown /proc/51355/task/52724/cwd (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52724 safe_time ceph rtd unknown /proc/51355/task/52724/root (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52724 safe_time ceph txt unknown /proc/51355/task/52724/exe (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52724 safe_time ceph NOFD /proc/51355/task/52724/fd (opendir: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52725 ceph-osd ceph cwd unknown /proc/51355/task/52725/cwd (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52725 ceph-osd ceph rtd unknown /proc/51355/task/52725/root (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52725 ceph-osd ceph txt unknown /proc/51355/task/52725/exe (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52725 ceph-osd ceph NOFD /proc/51355/task/52725/fd (opendir: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52726 osd_srv_a ceph cwd unknown /proc/51355/task/52726/cwd (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52726 osd_srv_a ceph rtd unknown /proc/51355/task/52726/root (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52726 osd_srv_a ceph txt unknown /proc/51355/task/52726/exe (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52726 osd_srv_a ceph NOFD /proc/51355/task/52726/fd (opendir: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52727 tp_osd_tp ceph cwd unknown /proc/51355/task/52727/cwd (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52727 tp_osd_tp ceph rtd unknown /proc/51355/task/52727/root (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52727 tp_osd_tp ceph txt unknown /proc/51355/task/52727/exe (readlink: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52727 tp_osd_tp ceph NOFD /proc/51355/task/52727/fd (opendir: Permission denied) 2024-03-12T04:17:28.497 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52728 tp_osd_tp ceph cwd unknown /proc/51355/task/52728/cwd (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52728 tp_osd_tp ceph rtd unknown /proc/51355/task/52728/root (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52728 tp_osd_tp ceph txt unknown /proc/51355/task/52728/exe (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52728 tp_osd_tp ceph NOFD /proc/51355/task/52728/fd (opendir: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52729 tp_osd_tp ceph cwd unknown /proc/51355/task/52729/cwd (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52729 tp_osd_tp ceph rtd unknown /proc/51355/task/52729/root (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52729 tp_osd_tp ceph txt unknown /proc/51355/task/52729/exe (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52729 tp_osd_tp ceph NOFD /proc/51355/task/52729/fd (opendir: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52730 tp_osd_tp ceph cwd unknown /proc/51355/task/52730/cwd (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52730 tp_osd_tp ceph rtd unknown /proc/51355/task/52730/root (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52730 tp_osd_tp ceph txt unknown /proc/51355/task/52730/exe (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52730 tp_osd_tp ceph NOFD /proc/51355/task/52730/fd (opendir: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52731 tp_osd_tp ceph cwd unknown /proc/51355/task/52731/cwd (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52731 tp_osd_tp ceph rtd unknown /proc/51355/task/52731/root (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52731 tp_osd_tp ceph txt unknown /proc/51355/task/52731/exe (readlink: Permission denied) 2024-03-12T04:17:28.498 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52731 tp_osd_tp ceph NOFD /proc/51355/task/52731/fd (opendir: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52732 tp_osd_tp ceph cwd unknown /proc/51355/task/52732/cwd (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52732 tp_osd_tp ceph rtd unknown /proc/51355/task/52732/root (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52732 tp_osd_tp ceph txt unknown /proc/51355/task/52732/exe (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52732 tp_osd_tp ceph NOFD /proc/51355/task/52732/fd (opendir: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52733 tp_osd_tp ceph cwd unknown /proc/51355/task/52733/cwd (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52733 tp_osd_tp ceph rtd unknown /proc/51355/task/52733/root (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52733 tp_osd_tp ceph txt unknown /proc/51355/task/52733/exe (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52733 tp_osd_tp ceph NOFD /proc/51355/task/52733/fd (opendir: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52734 tp_osd_tp ceph cwd unknown /proc/51355/task/52734/cwd (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52734 tp_osd_tp ceph rtd unknown /proc/51355/task/52734/root (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52734 tp_osd_tp ceph txt unknown /proc/51355/task/52734/exe (readlink: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52734 tp_osd_tp ceph NOFD /proc/51355/task/52734/fd (opendir: Permission denied) 2024-03-12T04:17:28.499 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52735 tp_osd_tp ceph cwd unknown /proc/51355/task/52735/cwd (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52735 tp_osd_tp ceph rtd unknown /proc/51355/task/52735/root (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52735 tp_osd_tp ceph txt unknown /proc/51355/task/52735/exe (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52735 tp_osd_tp ceph NOFD /proc/51355/task/52735/fd (opendir: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52736 tp_osd_tp ceph cwd unknown /proc/51355/task/52736/cwd (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52736 tp_osd_tp ceph rtd unknown /proc/51355/task/52736/root (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52736 tp_osd_tp ceph txt unknown /proc/51355/task/52736/exe (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52736 tp_osd_tp ceph NOFD /proc/51355/task/52736/fd (opendir: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52737 tp_osd_tp ceph cwd unknown /proc/51355/task/52737/cwd (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52737 tp_osd_tp ceph rtd unknown /proc/51355/task/52737/root (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52737 tp_osd_tp ceph txt unknown /proc/51355/task/52737/exe (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52737 tp_osd_tp ceph NOFD /proc/51355/task/52737/fd (opendir: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52738 tp_osd_tp ceph cwd unknown /proc/51355/task/52738/cwd (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52738 tp_osd_tp ceph rtd unknown /proc/51355/task/52738/root (readlink: Permission denied) 2024-03-12T04:17:28.500 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52738 tp_osd_tp ceph txt unknown /proc/51355/task/52738/exe (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52738 tp_osd_tp ceph NOFD /proc/51355/task/52738/fd (opendir: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52739 tp_osd_tp ceph cwd unknown /proc/51355/task/52739/cwd (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52739 tp_osd_tp ceph rtd unknown /proc/51355/task/52739/root (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52739 tp_osd_tp ceph txt unknown /proc/51355/task/52739/exe (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52739 tp_osd_tp ceph NOFD /proc/51355/task/52739/fd (opendir: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52740 tp_osd_tp ceph cwd unknown /proc/51355/task/52740/cwd (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52740 tp_osd_tp ceph rtd unknown /proc/51355/task/52740/root (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52740 tp_osd_tp ceph txt unknown /proc/51355/task/52740/exe (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52740 tp_osd_tp ceph NOFD /proc/51355/task/52740/fd (opendir: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52741 tp_osd_tp ceph cwd unknown /proc/51355/task/52741/cwd (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52741 tp_osd_tp ceph rtd unknown /proc/51355/task/52741/root (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52741 tp_osd_tp ceph txt unknown /proc/51355/task/52741/exe (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52741 tp_osd_tp ceph NOFD /proc/51355/task/52741/fd (opendir: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52742 tp_osd_tp ceph cwd unknown /proc/51355/task/52742/cwd (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52742 tp_osd_tp ceph rtd unknown /proc/51355/task/52742/root (readlink: Permission denied) 2024-03-12T04:17:28.501 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52742 tp_osd_tp ceph txt unknown /proc/51355/task/52742/exe (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52742 tp_osd_tp ceph NOFD /proc/51355/task/52742/fd (opendir: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52743 osd_srv_h ceph cwd unknown /proc/51355/task/52743/cwd (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52743 osd_srv_h ceph rtd unknown /proc/51355/task/52743/root (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52743 osd_srv_h ceph txt unknown /proc/51355/task/52743/exe (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 51355 52743 osd_srv_h ceph NOFD /proc/51355/task/52743/fd (opendir: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 52762 root cwd unknown /proc/52762/cwd (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 52762 root rtd unknown /proc/52762/root (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 52762 root txt unknown /proc/52762/exe (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 52762 root NOFD /proc/52762/fd (opendir: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 52779 root cwd unknown /proc/52779/cwd (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 52779 root rtd unknown /proc/52779/root (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 52779 root txt unknown /proc/52779/exe (readlink: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 52779 root NOFD /proc/52779/fd (opendir: Permission denied) 2024-03-12T04:17:28.502 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 55258 root cwd unknown /proc/55258/cwd (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 55258 root rtd unknown /proc/55258/root (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 55258 root txt unknown /proc/55258/exe (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/7 55258 root NOFD /proc/55258/fd (opendir: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 55368 root cwd unknown /proc/55368/cwd (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 55368 root rtd unknown /proc/55368/root (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 55368 root txt unknown /proc/55368/exe (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 55368 root NOFD /proc/55368/fd (opendir: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57464 root cwd unknown /proc/57464/cwd (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57464 root rtd unknown /proc/57464/root (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57464 root txt unknown /proc/57464/exe (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 57464 root NOFD /proc/57464/fd (opendir: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 57487 root cwd unknown /proc/57487/cwd (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 57487 root rtd unknown /proc/57487/root (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 57487 root txt unknown /proc/57487/exe (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 57487 root NOFD /proc/57487/fd (opendir: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 58066 root cwd unknown /proc/58066/cwd (readlink: Permission denied) 2024-03-12T04:17:28.503 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 58066 root rtd unknown /proc/58066/root (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 58066 root txt unknown /proc/58066/exe (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 58066 root NOFD /proc/58066/fd (opendir: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 58068 root cwd unknown /proc/58068/cwd (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 58068 root rtd unknown /proc/58068/root (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 58068 root txt unknown /proc/58068/exe (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 58068 root NOFD /proc/58068/fd (opendir: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 ceph cwd unknown /proc/58070/cwd (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 ceph rtd unknown /proc/58070/root (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 ceph txt unknown /proc/58070/exe (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 ceph NOFD /proc/58070/fd (opendir: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58071 log ceph cwd unknown /proc/58070/task/58071/cwd (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58071 log ceph rtd unknown /proc/58070/task/58071/root (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58071 log ceph txt unknown /proc/58070/task/58071/exe (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58071 log ceph NOFD /proc/58070/task/58071/fd (opendir: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58073 msgr-work ceph cwd unknown /proc/58070/task/58073/cwd (readlink: Permission denied) 2024-03-12T04:17:28.504 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58073 msgr-work ceph rtd unknown /proc/58070/task/58073/root (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58073 msgr-work ceph txt unknown /proc/58070/task/58073/exe (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58073 msgr-work ceph NOFD /proc/58070/task/58073/fd (opendir: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58074 msgr-work ceph cwd unknown /proc/58070/task/58074/cwd (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58074 msgr-work ceph rtd unknown /proc/58070/task/58074/root (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58074 msgr-work ceph txt unknown /proc/58070/task/58074/exe (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58074 msgr-work ceph NOFD /proc/58070/task/58074/fd (opendir: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58075 msgr-work ceph cwd unknown /proc/58070/task/58075/cwd (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58075 msgr-work ceph rtd unknown /proc/58070/task/58075/root (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58075 msgr-work ceph txt unknown /proc/58070/task/58075/exe (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58075 msgr-work ceph NOFD /proc/58070/task/58075/fd (opendir: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58079 service ceph cwd unknown /proc/58070/task/58079/cwd (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58079 service ceph rtd unknown /proc/58070/task/58079/root (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58079 service ceph txt unknown /proc/58070/task/58079/exe (readlink: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58079 service ceph NOFD /proc/58070/task/58079/fd (opendir: Permission denied) 2024-03-12T04:17:28.505 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58080 admin_soc ceph cwd unknown /proc/58070/task/58080/cwd (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58080 admin_soc ceph rtd unknown /proc/58070/task/58080/root (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58080 admin_soc ceph txt unknown /proc/58070/task/58080/exe (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58080 admin_soc ceph NOFD /proc/58070/task/58080/fd (opendir: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58222 signal_ha ceph cwd unknown /proc/58070/task/58222/cwd (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58222 signal_ha ceph rtd unknown /proc/58070/task/58222/root (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58222 signal_ha ceph txt unknown /proc/58070/task/58222/exe (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58222 signal_ha ceph NOFD /proc/58070/task/58222/fd (opendir: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58223 io_contex ceph cwd unknown /proc/58070/task/58223/cwd (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58223 io_contex ceph rtd unknown /proc/58070/task/58223/root (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58223 io_contex ceph txt unknown /proc/58070/task/58223/exe (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58223 io_contex ceph NOFD /proc/58070/task/58223/fd (opendir: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58224 io_contex ceph cwd unknown /proc/58070/task/58224/cwd (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58224 io_contex ceph rtd unknown /proc/58070/task/58224/root (readlink: Permission denied) 2024-03-12T04:17:28.506 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58224 io_contex ceph txt unknown /proc/58070/task/58224/exe (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58224 io_contex ceph NOFD /proc/58070/task/58224/fd (opendir: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58339 OpHistory ceph cwd unknown /proc/58070/task/58339/cwd (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58339 OpHistory ceph rtd unknown /proc/58070/task/58339/root (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58339 OpHistory ceph txt unknown /proc/58070/task/58339/exe (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58339 OpHistory ceph NOFD /proc/58070/task/58339/fd (opendir: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58340 ceph_time ceph cwd unknown /proc/58070/task/58340/cwd (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58340 ceph_time ceph rtd unknown /proc/58070/task/58340/root (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58340 ceph_time ceph txt unknown /proc/58070/task/58340/exe (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58340 ceph_time ceph NOFD /proc/58070/task/58340/fd (opendir: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58390 ceph-osd ceph cwd unknown /proc/58070/task/58390/cwd (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58390 ceph-osd ceph rtd unknown /proc/58070/task/58390/root (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58390 ceph-osd ceph txt unknown /proc/58070/task/58390/exe (readlink: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58390 ceph-osd ceph NOFD /proc/58070/task/58390/fd (opendir: Permission denied) 2024-03-12T04:17:28.507 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58394 ceph-osd ceph cwd unknown /proc/58070/task/58394/cwd (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58394 ceph-osd ceph rtd unknown /proc/58070/task/58394/root (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58394 ceph-osd ceph txt unknown /proc/58070/task/58394/exe (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58394 ceph-osd ceph NOFD /proc/58070/task/58394/fd (opendir: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58403 ceph-osd ceph cwd unknown /proc/58070/task/58403/cwd (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58403 ceph-osd ceph rtd unknown /proc/58070/task/58403/root (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58403 ceph-osd ceph txt unknown /proc/58070/task/58403/exe (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58403 ceph-osd ceph NOFD /proc/58070/task/58403/fd (opendir: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58413 ceph-osd ceph cwd unknown /proc/58070/task/58413/cwd (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58413 ceph-osd ceph rtd unknown /proc/58070/task/58413/root (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58413 ceph-osd ceph txt unknown /proc/58070/task/58413/exe (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58413 ceph-osd ceph NOFD /proc/58070/task/58413/fd (opendir: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58430 ceph-osd ceph cwd unknown /proc/58070/task/58430/cwd (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58430 ceph-osd ceph rtd unknown /proc/58070/task/58430/root (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58430 ceph-osd ceph txt unknown /proc/58070/task/58430/exe (readlink: Permission denied) 2024-03-12T04:17:28.508 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58430 ceph-osd ceph NOFD /proc/58070/task/58430/fd (opendir: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58452 ceph-osd ceph cwd unknown /proc/58070/task/58452/cwd (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58452 ceph-osd ceph rtd unknown /proc/58070/task/58452/root (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58452 ceph-osd ceph txt unknown /proc/58070/task/58452/exe (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58452 ceph-osd ceph NOFD /proc/58070/task/58452/fd (opendir: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58461 ceph-osd ceph cwd unknown /proc/58070/task/58461/cwd (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58461 ceph-osd ceph rtd unknown /proc/58070/task/58461/root (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58461 ceph-osd ceph txt unknown /proc/58070/task/58461/exe (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58461 ceph-osd ceph NOFD /proc/58070/task/58461/fd (opendir: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58503 ceph-osd ceph cwd unknown /proc/58070/task/58503/cwd (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58503 ceph-osd ceph rtd unknown /proc/58070/task/58503/root (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58503 ceph-osd ceph txt unknown /proc/58070/task/58503/exe (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58503 ceph-osd ceph NOFD /proc/58070/task/58503/fd (opendir: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58504 ceph-osd ceph cwd unknown /proc/58070/task/58504/cwd (readlink: Permission denied) 2024-03-12T04:17:28.509 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58504 ceph-osd ceph rtd unknown /proc/58070/task/58504/root (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58504 ceph-osd ceph txt unknown /proc/58070/task/58504/exe (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58504 ceph-osd ceph NOFD /proc/58070/task/58504/fd (opendir: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58505 safe_time ceph cwd unknown /proc/58070/task/58505/cwd (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58505 safe_time ceph rtd unknown /proc/58070/task/58505/root (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58505 safe_time ceph txt unknown /proc/58070/task/58505/exe (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58505 safe_time ceph NOFD /proc/58070/task/58505/fd (opendir: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58506 safe_time ceph cwd unknown /proc/58070/task/58506/cwd (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58506 safe_time ceph rtd unknown /proc/58070/task/58506/root (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58506 safe_time ceph txt unknown /proc/58070/task/58506/exe (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58506 safe_time ceph NOFD /proc/58070/task/58506/fd (opendir: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58508 safe_time ceph cwd unknown /proc/58070/task/58508/cwd (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58508 safe_time ceph rtd unknown /proc/58070/task/58508/root (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58508 safe_time ceph txt unknown /proc/58070/task/58508/exe (readlink: Permission denied) 2024-03-12T04:17:28.510 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58508 safe_time ceph NOFD /proc/58070/task/58508/fd (opendir: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58509 safe_time ceph cwd unknown /proc/58070/task/58509/cwd (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58509 safe_time ceph rtd unknown /proc/58070/task/58509/root (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58509 safe_time ceph txt unknown /proc/58070/task/58509/exe (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58509 safe_time ceph NOFD /proc/58070/task/58509/fd (opendir: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58510 fn_anonym ceph cwd unknown /proc/58070/task/58510/cwd (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58510 fn_anonym ceph rtd unknown /proc/58070/task/58510/root (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58510 fn_anonym ceph txt unknown /proc/58070/task/58510/exe (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58510 fn_anonym ceph NOFD /proc/58070/task/58510/fd (opendir: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58513 rocksdb:l ceph cwd unknown /proc/58070/task/58513/cwd (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58513 rocksdb:l ceph rtd unknown /proc/58070/task/58513/root (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58513 rocksdb:l ceph txt unknown /proc/58070/task/58513/exe (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58513 rocksdb:l ceph NOFD /proc/58070/task/58513/fd (opendir: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58514 rocksdb:l ceph cwd unknown /proc/58070/task/58514/cwd (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58514 rocksdb:l ceph rtd unknown /proc/58070/task/58514/root (readlink: Permission denied) 2024-03-12T04:17:28.511 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58514 rocksdb:l ceph txt unknown /proc/58070/task/58514/exe (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58514 rocksdb:l ceph NOFD /proc/58070/task/58514/fd (opendir: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58515 rocksdb:l ceph cwd unknown /proc/58070/task/58515/cwd (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58515 rocksdb:l ceph rtd unknown /proc/58070/task/58515/root (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58515 rocksdb:l ceph txt unknown /proc/58070/task/58515/exe (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58515 rocksdb:l ceph NOFD /proc/58070/task/58515/fd (opendir: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58516 rocksdb:h ceph cwd unknown /proc/58070/task/58516/cwd (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58516 rocksdb:h ceph rtd unknown /proc/58070/task/58516/root (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58516 rocksdb:h ceph txt unknown /proc/58070/task/58516/exe (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58516 rocksdb:h ceph NOFD /proc/58070/task/58516/fd (opendir: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58995 bstore_ai ceph cwd unknown /proc/58070/task/58995/cwd (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58995 bstore_ai ceph rtd unknown /proc/58070/task/58995/root (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58995 bstore_ai ceph txt unknown /proc/58070/task/58995/exe (readlink: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 58995 bstore_ai ceph NOFD /proc/58070/task/58995/fd (opendir: Permission denied) 2024-03-12T04:17:28.512 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59178 bstore_ai ceph cwd unknown /proc/58070/task/59178/cwd (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59178 bstore_ai ceph rtd unknown /proc/58070/task/59178/root (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59178 bstore_ai ceph txt unknown /proc/58070/task/59178/exe (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59178 bstore_ai ceph NOFD /proc/58070/task/59178/fd (opendir: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59360 ceph-osd ceph cwd unknown /proc/58070/task/59360/cwd (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59360 ceph-osd ceph rtd unknown /proc/58070/task/59360/root (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59360 ceph-osd ceph txt unknown /proc/58070/task/59360/exe (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59360 ceph-osd ceph NOFD /proc/58070/task/59360/fd (opendir: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59361 cfin ceph cwd unknown /proc/58070/task/59361/cwd (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59361 cfin ceph rtd unknown /proc/58070/task/59361/root (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59361 cfin ceph txt unknown /proc/58070/task/59361/exe (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59361 cfin ceph NOFD /proc/58070/task/59361/fd (opendir: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59362 bstore_kv ceph cwd unknown /proc/58070/task/59362/cwd (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59362 bstore_kv ceph rtd unknown /proc/58070/task/59362/root (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59362 bstore_kv ceph txt unknown /proc/58070/task/59362/exe (readlink: Permission denied) 2024-03-12T04:17:28.513 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59362 bstore_kv ceph NOFD /proc/58070/task/59362/fd (opendir: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59363 bstore_kv ceph cwd unknown /proc/58070/task/59363/cwd (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59363 bstore_kv ceph rtd unknown /proc/58070/task/59363/root (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59363 bstore_kv ceph txt unknown /proc/58070/task/59363/exe (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59363 bstore_kv ceph NOFD /proc/58070/task/59363/fd (opendir: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59364 bstore_me ceph cwd unknown /proc/58070/task/59364/cwd (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59364 bstore_me ceph rtd unknown /proc/58070/task/59364/root (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59364 bstore_me ceph txt unknown /proc/58070/task/59364/exe (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59364 bstore_me ceph NOFD /proc/58070/task/59364/fd (opendir: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59365 ms_dispat ceph cwd unknown /proc/58070/task/59365/cwd (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59365 ms_dispat ceph rtd unknown /proc/58070/task/59365/root (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59365 ms_dispat ceph txt unknown /proc/58070/task/59365/exe (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59365 ms_dispat ceph NOFD /proc/58070/task/59365/fd (opendir: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59366 ms_local ceph cwd unknown /proc/58070/task/59366/cwd (readlink: Permission denied) 2024-03-12T04:17:28.514 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59366 ms_local ceph rtd unknown /proc/58070/task/59366/root (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59366 ms_local ceph txt unknown /proc/58070/task/59366/exe (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59366 ms_local ceph NOFD /proc/58070/task/59366/fd (opendir: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59367 safe_time ceph cwd unknown /proc/58070/task/59367/cwd (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59367 safe_time ceph rtd unknown /proc/58070/task/59367/root (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59367 safe_time ceph txt unknown /proc/58070/task/59367/exe (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59367 safe_time ceph NOFD /proc/58070/task/59367/fd (opendir: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59368 safe_time ceph cwd unknown /proc/58070/task/59368/cwd (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59368 safe_time ceph rtd unknown /proc/58070/task/59368/root (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59368 safe_time ceph txt unknown /proc/58070/task/59368/exe (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59368 safe_time ceph NOFD /proc/58070/task/59368/fd (opendir: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59369 ms_dispat ceph cwd unknown /proc/58070/task/59369/cwd (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59369 ms_dispat ceph rtd unknown /proc/58070/task/59369/root (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59369 ms_dispat ceph txt unknown /proc/58070/task/59369/exe (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59369 ms_dispat ceph NOFD /proc/58070/task/59369/fd (opendir: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59370 ms_local ceph cwd unknown /proc/58070/task/59370/cwd (readlink: Permission denied) 2024-03-12T04:17:28.515 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59370 ms_local ceph rtd unknown /proc/58070/task/59370/root (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59370 ms_local ceph txt unknown /proc/58070/task/59370/exe (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59370 ms_local ceph NOFD /proc/58070/task/59370/fd (opendir: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59371 ms_dispat ceph cwd unknown /proc/58070/task/59371/cwd (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59371 ms_dispat ceph rtd unknown /proc/58070/task/59371/root (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59371 ms_dispat ceph txt unknown /proc/58070/task/59371/exe (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59371 ms_dispat ceph NOFD /proc/58070/task/59371/fd (opendir: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59372 ms_local ceph cwd unknown /proc/58070/task/59372/cwd (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59372 ms_local ceph rtd unknown /proc/58070/task/59372/root (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59372 ms_local ceph txt unknown /proc/58070/task/59372/exe (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59372 ms_local ceph NOFD /proc/58070/task/59372/fd (opendir: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59373 ms_dispat ceph cwd unknown /proc/58070/task/59373/cwd (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59373 ms_dispat ceph rtd unknown /proc/58070/task/59373/root (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59373 ms_dispat ceph txt unknown /proc/58070/task/59373/exe (readlink: Permission denied) 2024-03-12T04:17:28.516 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59373 ms_dispat ceph NOFD /proc/58070/task/59373/fd (opendir: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59374 ms_local ceph cwd unknown /proc/58070/task/59374/cwd (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59374 ms_local ceph rtd unknown /proc/58070/task/59374/root (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59374 ms_local ceph txt unknown /proc/58070/task/59374/exe (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59374 ms_local ceph NOFD /proc/58070/task/59374/fd (opendir: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59375 ms_dispat ceph cwd unknown /proc/58070/task/59375/cwd (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59375 ms_dispat ceph rtd unknown /proc/58070/task/59375/root (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59375 ms_dispat ceph txt unknown /proc/58070/task/59375/exe (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59375 ms_dispat ceph NOFD /proc/58070/task/59375/fd (opendir: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59376 ms_local ceph cwd unknown /proc/58070/task/59376/cwd (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59376 ms_local ceph rtd unknown /proc/58070/task/59376/root (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59376 ms_local ceph txt unknown /proc/58070/task/59376/exe (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59376 ms_local ceph NOFD /proc/58070/task/59376/fd (opendir: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59377 ms_dispat ceph cwd unknown /proc/58070/task/59377/cwd (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59377 ms_dispat ceph rtd unknown /proc/58070/task/59377/root (readlink: Permission denied) 2024-03-12T04:17:28.517 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59377 ms_dispat ceph txt unknown /proc/58070/task/59377/exe (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59377 ms_dispat ceph NOFD /proc/58070/task/59377/fd (opendir: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59378 ms_local ceph cwd unknown /proc/58070/task/59378/cwd (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59378 ms_local ceph rtd unknown /proc/58070/task/59378/root (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59378 ms_local ceph txt unknown /proc/58070/task/59378/exe (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59378 ms_local ceph NOFD /proc/58070/task/59378/fd (opendir: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59379 ms_dispat ceph cwd unknown /proc/58070/task/59379/cwd (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59379 ms_dispat ceph rtd unknown /proc/58070/task/59379/root (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59379 ms_dispat ceph txt unknown /proc/58070/task/59379/exe (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59379 ms_dispat ceph NOFD /proc/58070/task/59379/fd (opendir: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59380 ms_local ceph cwd unknown /proc/58070/task/59380/cwd (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59380 ms_local ceph rtd unknown /proc/58070/task/59380/root (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59380 ms_local ceph txt unknown /proc/58070/task/59380/exe (readlink: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59380 ms_local ceph NOFD /proc/58070/task/59380/fd (opendir: Permission denied) 2024-03-12T04:17:28.518 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59381 fn_anonym ceph cwd unknown /proc/58070/task/59381/cwd (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59381 fn_anonym ceph rtd unknown /proc/58070/task/59381/root (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59381 fn_anonym ceph txt unknown /proc/58070/task/59381/exe (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59381 fn_anonym ceph NOFD /proc/58070/task/59381/fd (opendir: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59382 finisher ceph cwd unknown /proc/58070/task/59382/cwd (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59382 finisher ceph rtd unknown /proc/58070/task/59382/root (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59382 finisher ceph txt unknown /proc/58070/task/59382/exe (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59382 finisher ceph NOFD /proc/58070/task/59382/fd (opendir: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59383 safe_time ceph cwd unknown /proc/58070/task/59383/cwd (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59383 safe_time ceph rtd unknown /proc/58070/task/59383/root (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59383 safe_time ceph txt unknown /proc/58070/task/59383/exe (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59383 safe_time ceph NOFD /proc/58070/task/59383/fd (opendir: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59384 safe_time ceph cwd unknown /proc/58070/task/59384/cwd (readlink: Permission denied) 2024-03-12T04:17:28.519 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59384 safe_time ceph rtd unknown /proc/58070/task/59384/root (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59384 safe_time ceph txt unknown /proc/58070/task/59384/exe (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59384 safe_time ceph NOFD /proc/58070/task/59384/fd (opendir: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59385 ceph-osd ceph cwd unknown /proc/58070/task/59385/cwd (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59385 ceph-osd ceph rtd unknown /proc/58070/task/59385/root (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59385 ceph-osd ceph txt unknown /proc/58070/task/59385/exe (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59385 ceph-osd ceph NOFD /proc/58070/task/59385/fd (opendir: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59386 osd_srv_a ceph cwd unknown /proc/58070/task/59386/cwd (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59386 osd_srv_a ceph rtd unknown /proc/58070/task/59386/root (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59386 osd_srv_a ceph txt unknown /proc/58070/task/59386/exe (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59386 osd_srv_a ceph NOFD /proc/58070/task/59386/fd (opendir: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59387 tp_osd_tp ceph cwd unknown /proc/58070/task/59387/cwd (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59387 tp_osd_tp ceph rtd unknown /proc/58070/task/59387/root (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59387 tp_osd_tp ceph txt unknown /proc/58070/task/59387/exe (readlink: Permission denied) 2024-03-12T04:17:28.520 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59387 tp_osd_tp ceph NOFD /proc/58070/task/59387/fd (opendir: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59388 tp_osd_tp ceph cwd unknown /proc/58070/task/59388/cwd (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59388 tp_osd_tp ceph rtd unknown /proc/58070/task/59388/root (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59388 tp_osd_tp ceph txt unknown /proc/58070/task/59388/exe (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59388 tp_osd_tp ceph NOFD /proc/58070/task/59388/fd (opendir: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59389 tp_osd_tp ceph cwd unknown /proc/58070/task/59389/cwd (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59389 tp_osd_tp ceph rtd unknown /proc/58070/task/59389/root (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59389 tp_osd_tp ceph txt unknown /proc/58070/task/59389/exe (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59389 tp_osd_tp ceph NOFD /proc/58070/task/59389/fd (opendir: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59390 tp_osd_tp ceph cwd unknown /proc/58070/task/59390/cwd (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59390 tp_osd_tp ceph rtd unknown /proc/58070/task/59390/root (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59390 tp_osd_tp ceph txt unknown /proc/58070/task/59390/exe (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59390 tp_osd_tp ceph NOFD /proc/58070/task/59390/fd (opendir: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59391 tp_osd_tp ceph cwd unknown /proc/58070/task/59391/cwd (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59391 tp_osd_tp ceph rtd unknown /proc/58070/task/59391/root (readlink: Permission denied) 2024-03-12T04:17:28.521 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59391 tp_osd_tp ceph txt unknown /proc/58070/task/59391/exe (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59391 tp_osd_tp ceph NOFD /proc/58070/task/59391/fd (opendir: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59392 tp_osd_tp ceph cwd unknown /proc/58070/task/59392/cwd (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59392 tp_osd_tp ceph rtd unknown /proc/58070/task/59392/root (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59392 tp_osd_tp ceph txt unknown /proc/58070/task/59392/exe (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59392 tp_osd_tp ceph NOFD /proc/58070/task/59392/fd (opendir: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59393 tp_osd_tp ceph cwd unknown /proc/58070/task/59393/cwd (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59393 tp_osd_tp ceph rtd unknown /proc/58070/task/59393/root (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59393 tp_osd_tp ceph txt unknown /proc/58070/task/59393/exe (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59393 tp_osd_tp ceph NOFD /proc/58070/task/59393/fd (opendir: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59394 tp_osd_tp ceph cwd unknown /proc/58070/task/59394/cwd (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59394 tp_osd_tp ceph rtd unknown /proc/58070/task/59394/root (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59394 tp_osd_tp ceph txt unknown /proc/58070/task/59394/exe (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59394 tp_osd_tp ceph NOFD /proc/58070/task/59394/fd (opendir: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59395 tp_osd_tp ceph cwd unknown /proc/58070/task/59395/cwd (readlink: Permission denied) 2024-03-12T04:17:28.522 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59395 tp_osd_tp ceph rtd unknown /proc/58070/task/59395/root (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59395 tp_osd_tp ceph txt unknown /proc/58070/task/59395/exe (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59395 tp_osd_tp ceph NOFD /proc/58070/task/59395/fd (opendir: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59396 tp_osd_tp ceph cwd unknown /proc/58070/task/59396/cwd (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59396 tp_osd_tp ceph rtd unknown /proc/58070/task/59396/root (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59396 tp_osd_tp ceph txt unknown /proc/58070/task/59396/exe (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59396 tp_osd_tp ceph NOFD /proc/58070/task/59396/fd (opendir: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59397 tp_osd_tp ceph cwd unknown /proc/58070/task/59397/cwd (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59397 tp_osd_tp ceph rtd unknown /proc/58070/task/59397/root (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59397 tp_osd_tp ceph txt unknown /proc/58070/task/59397/exe (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59397 tp_osd_tp ceph NOFD /proc/58070/task/59397/fd (opendir: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59398 tp_osd_tp ceph cwd unknown /proc/58070/task/59398/cwd (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59398 tp_osd_tp ceph rtd unknown /proc/58070/task/59398/root (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59398 tp_osd_tp ceph txt unknown /proc/58070/task/59398/exe (readlink: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59398 tp_osd_tp ceph NOFD /proc/58070/task/59398/fd (opendir: Permission denied) 2024-03-12T04:17:28.523 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59399 tp_osd_tp ceph cwd unknown /proc/58070/task/59399/cwd (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59399 tp_osd_tp ceph rtd unknown /proc/58070/task/59399/root (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59399 tp_osd_tp ceph txt unknown /proc/58070/task/59399/exe (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59399 tp_osd_tp ceph NOFD /proc/58070/task/59399/fd (opendir: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59400 tp_osd_tp ceph cwd unknown /proc/58070/task/59400/cwd (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59400 tp_osd_tp ceph rtd unknown /proc/58070/task/59400/root (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59400 tp_osd_tp ceph txt unknown /proc/58070/task/59400/exe (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59400 tp_osd_tp ceph NOFD /proc/58070/task/59400/fd (opendir: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59401 tp_osd_tp ceph cwd unknown /proc/58070/task/59401/cwd (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59401 tp_osd_tp ceph rtd unknown /proc/58070/task/59401/root (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59401 tp_osd_tp ceph txt unknown /proc/58070/task/59401/exe (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59401 tp_osd_tp ceph NOFD /proc/58070/task/59401/fd (opendir: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59402 tp_osd_tp ceph cwd unknown /proc/58070/task/59402/cwd (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59402 tp_osd_tp ceph rtd unknown /proc/58070/task/59402/root (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59402 tp_osd_tp ceph txt unknown /proc/58070/task/59402/exe (readlink: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59402 tp_osd_tp ceph NOFD /proc/58070/task/59402/fd (opendir: Permission denied) 2024-03-12T04:17:28.524 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59403 osd_srv_h ceph cwd unknown /proc/58070/task/59403/cwd (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59403 osd_srv_h ceph rtd unknown /proc/58070/task/59403/root (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59403 osd_srv_h ceph txt unknown /proc/58070/task/59403/exe (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 58070 59403 osd_srv_h ceph NOFD /proc/58070/task/59403/fd (opendir: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 58389 root cwd unknown /proc/58389/cwd (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 58389 root rtd unknown /proc/58389/root (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 58389 root txt unknown /proc/58389/exe (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/1 58389 root NOFD /proc/58389/fd (opendir: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 59498 root cwd unknown /proc/59498/cwd (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 59498 root rtd unknown /proc/59498/root (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 59498 root txt unknown /proc/59498/exe (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 59498 root NOFD /proc/59498/fd (opendir: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 59516 root cwd unknown /proc/59516/cwd (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 59516 root rtd unknown /proc/59516/root (readlink: Permission denied) 2024-03-12T04:17:28.525 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 59516 root txt unknown /proc/59516/exe (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 59516 root NOFD /proc/59516/fd (opendir: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 61852 root cwd unknown /proc/61852/cwd (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 61852 root rtd unknown /proc/61852/root (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 61852 root txt unknown /proc/61852/exe (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/4 61852 root NOFD /proc/61852/fd (opendir: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 64592 root cwd unknown /proc/64592/cwd (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 64592 root rtd unknown /proc/64592/root (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 64592 root txt unknown /proc/64592/exe (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/2 64592 root NOFD /proc/64592/fd (opendir: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 64659 root cwd unknown /proc/64659/cwd (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 64659 root rtd unknown /proc/64659/root (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 64659 root txt unknown /proc/64659/exe (readlink: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 64659 root NOFD /proc/64659/fd (opendir: Permission denied) 2024-03-12T04:17:28.526 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 64661 root cwd unknown /proc/64661/cwd (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 64661 root rtd unknown /proc/64661/root (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 64661 root txt unknown /proc/64661/exe (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 64661 root NOFD /proc/64661/fd (opendir: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 ceph cwd unknown /proc/64663/cwd (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 ceph rtd unknown /proc/64663/root (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 ceph txt unknown /proc/64663/exe (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 ceph NOFD /proc/64663/fd (opendir: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64664 log ceph cwd unknown /proc/64663/task/64664/cwd (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64664 log ceph rtd unknown /proc/64663/task/64664/root (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64664 log ceph txt unknown /proc/64663/task/64664/exe (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64664 log ceph NOFD /proc/64663/task/64664/fd (opendir: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64666 msgr-work ceph cwd unknown /proc/64663/task/64666/cwd (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64666 msgr-work ceph rtd unknown /proc/64663/task/64666/root (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64666 msgr-work ceph txt unknown /proc/64663/task/64666/exe (readlink: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64666 msgr-work ceph NOFD /proc/64663/task/64666/fd (opendir: Permission denied) 2024-03-12T04:17:28.527 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64667 msgr-work ceph cwd unknown /proc/64663/task/64667/cwd (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64667 msgr-work ceph rtd unknown /proc/64663/task/64667/root (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64667 msgr-work ceph txt unknown /proc/64663/task/64667/exe (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64667 msgr-work ceph NOFD /proc/64663/task/64667/fd (opendir: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64668 msgr-work ceph cwd unknown /proc/64663/task/64668/cwd (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64668 msgr-work ceph rtd unknown /proc/64663/task/64668/root (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64668 msgr-work ceph txt unknown /proc/64663/task/64668/exe (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64668 msgr-work ceph NOFD /proc/64663/task/64668/fd (opendir: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64672 service ceph cwd unknown /proc/64663/task/64672/cwd (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64672 service ceph rtd unknown /proc/64663/task/64672/root (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64672 service ceph txt unknown /proc/64663/task/64672/exe (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64672 service ceph NOFD /proc/64663/task/64672/fd (opendir: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64673 admin_soc ceph cwd unknown /proc/64663/task/64673/cwd (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64673 admin_soc ceph rtd unknown /proc/64663/task/64673/root (readlink: Permission denied) 2024-03-12T04:17:28.528 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64673 admin_soc ceph txt unknown /proc/64663/task/64673/exe (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64673 admin_soc ceph NOFD /proc/64663/task/64673/fd (opendir: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64817 signal_ha ceph cwd unknown /proc/64663/task/64817/cwd (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64817 signal_ha ceph rtd unknown /proc/64663/task/64817/root (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64817 signal_ha ceph txt unknown /proc/64663/task/64817/exe (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64817 signal_ha ceph NOFD /proc/64663/task/64817/fd (opendir: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64818 io_contex ceph cwd unknown /proc/64663/task/64818/cwd (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64818 io_contex ceph rtd unknown /proc/64663/task/64818/root (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64818 io_contex ceph txt unknown /proc/64663/task/64818/exe (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64818 io_contex ceph NOFD /proc/64663/task/64818/fd (opendir: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64819 io_contex ceph cwd unknown /proc/64663/task/64819/cwd (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64819 io_contex ceph rtd unknown /proc/64663/task/64819/root (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64819 io_contex ceph txt unknown /proc/64663/task/64819/exe (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64819 io_contex ceph NOFD /proc/64663/task/64819/fd (opendir: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64936 OpHistory ceph cwd unknown /proc/64663/task/64936/cwd (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64936 OpHistory ceph rtd unknown /proc/64663/task/64936/root (readlink: Permission denied) 2024-03-12T04:17:28.529 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64936 OpHistory ceph txt unknown /proc/64663/task/64936/exe (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64936 OpHistory ceph NOFD /proc/64663/task/64936/fd (opendir: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64937 ceph_time ceph cwd unknown /proc/64663/task/64937/cwd (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64937 ceph_time ceph rtd unknown /proc/64663/task/64937/root (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64937 ceph_time ceph txt unknown /proc/64663/task/64937/exe (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 64937 ceph_time ceph NOFD /proc/64663/task/64937/fd (opendir: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65070 ceph-osd ceph cwd unknown /proc/64663/task/65070/cwd (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65070 ceph-osd ceph rtd unknown /proc/64663/task/65070/root (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65070 ceph-osd ceph txt unknown /proc/64663/task/65070/exe (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65070 ceph-osd ceph NOFD /proc/64663/task/65070/fd (opendir: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65072 safe_time ceph cwd unknown /proc/64663/task/65072/cwd (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65072 safe_time ceph rtd unknown /proc/64663/task/65072/root (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65072 safe_time ceph txt unknown /proc/64663/task/65072/exe (readlink: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65072 safe_time ceph NOFD /proc/64663/task/65072/fd (opendir: Permission denied) 2024-03-12T04:17:28.530 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65073 safe_time ceph cwd unknown /proc/64663/task/65073/cwd (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65073 safe_time ceph rtd unknown /proc/64663/task/65073/root (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65073 safe_time ceph txt unknown /proc/64663/task/65073/exe (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65073 safe_time ceph NOFD /proc/64663/task/65073/fd (opendir: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65074 safe_time ceph cwd unknown /proc/64663/task/65074/cwd (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65074 safe_time ceph rtd unknown /proc/64663/task/65074/root (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65074 safe_time ceph txt unknown /proc/64663/task/65074/exe (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65074 safe_time ceph NOFD /proc/64663/task/65074/fd (opendir: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65075 safe_time ceph cwd unknown /proc/64663/task/65075/cwd (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65075 safe_time ceph rtd unknown /proc/64663/task/65075/root (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65075 safe_time ceph txt unknown /proc/64663/task/65075/exe (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65075 safe_time ceph NOFD /proc/64663/task/65075/fd (opendir: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65076 fn_anonym ceph cwd unknown /proc/64663/task/65076/cwd (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65076 fn_anonym ceph rtd unknown /proc/64663/task/65076/root (readlink: Permission denied) 2024-03-12T04:17:28.531 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65076 fn_anonym ceph txt unknown /proc/64663/task/65076/exe (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65076 fn_anonym ceph NOFD /proc/64663/task/65076/fd (opendir: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65079 rocksdb:l ceph cwd unknown /proc/64663/task/65079/cwd (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65079 rocksdb:l ceph rtd unknown /proc/64663/task/65079/root (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65079 rocksdb:l ceph txt unknown /proc/64663/task/65079/exe (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65079 rocksdb:l ceph NOFD /proc/64663/task/65079/fd (opendir: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65080 rocksdb:l ceph cwd unknown /proc/64663/task/65080/cwd (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65080 rocksdb:l ceph rtd unknown /proc/64663/task/65080/root (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65080 rocksdb:l ceph txt unknown /proc/64663/task/65080/exe (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65080 rocksdb:l ceph NOFD /proc/64663/task/65080/fd (opendir: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65081 rocksdb:l ceph cwd unknown /proc/64663/task/65081/cwd (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65081 rocksdb:l ceph rtd unknown /proc/64663/task/65081/root (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65081 rocksdb:l ceph txt unknown /proc/64663/task/65081/exe (readlink: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65081 rocksdb:l ceph NOFD /proc/64663/task/65081/fd (opendir: Permission denied) 2024-03-12T04:17:28.532 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65082 rocksdb:h ceph cwd unknown /proc/64663/task/65082/cwd (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65082 rocksdb:h ceph rtd unknown /proc/64663/task/65082/root (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65082 rocksdb:h ceph txt unknown /proc/64663/task/65082/exe (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65082 rocksdb:h ceph NOFD /proc/64663/task/65082/fd (opendir: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65555 bstore_ai ceph cwd unknown /proc/64663/task/65555/cwd (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65555 bstore_ai ceph rtd unknown /proc/64663/task/65555/root (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65555 bstore_ai ceph txt unknown /proc/64663/task/65555/exe (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65555 bstore_ai ceph NOFD /proc/64663/task/65555/fd (opendir: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65761 bstore_ai ceph cwd unknown /proc/64663/task/65761/cwd (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65761 bstore_ai ceph rtd unknown /proc/64663/task/65761/root (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65761 bstore_ai ceph txt unknown /proc/64663/task/65761/exe (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65761 bstore_ai ceph NOFD /proc/64663/task/65761/fd (opendir: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65944 ceph-osd ceph cwd unknown /proc/64663/task/65944/cwd (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65944 ceph-osd ceph rtd unknown /proc/64663/task/65944/root (readlink: Permission denied) 2024-03-12T04:17:28.533 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65944 ceph-osd ceph txt unknown /proc/64663/task/65944/exe (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65944 ceph-osd ceph NOFD /proc/64663/task/65944/fd (opendir: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65945 cfin ceph cwd unknown /proc/64663/task/65945/cwd (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65945 cfin ceph rtd unknown /proc/64663/task/65945/root (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65945 cfin ceph txt unknown /proc/64663/task/65945/exe (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65945 cfin ceph NOFD /proc/64663/task/65945/fd (opendir: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65946 bstore_kv ceph cwd unknown /proc/64663/task/65946/cwd (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65946 bstore_kv ceph rtd unknown /proc/64663/task/65946/root (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65946 bstore_kv ceph txt unknown /proc/64663/task/65946/exe (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65946 bstore_kv ceph NOFD /proc/64663/task/65946/fd (opendir: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65947 bstore_kv ceph cwd unknown /proc/64663/task/65947/cwd (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65947 bstore_kv ceph rtd unknown /proc/64663/task/65947/root (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65947 bstore_kv ceph txt unknown /proc/64663/task/65947/exe (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65947 bstore_kv ceph NOFD /proc/64663/task/65947/fd (opendir: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65948 bstore_me ceph cwd unknown /proc/64663/task/65948/cwd (readlink: Permission denied) 2024-03-12T04:17:28.534 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65948 bstore_me ceph rtd unknown /proc/64663/task/65948/root (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65948 bstore_me ceph txt unknown /proc/64663/task/65948/exe (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65948 bstore_me ceph NOFD /proc/64663/task/65948/fd (opendir: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65949 ms_dispat ceph cwd unknown /proc/64663/task/65949/cwd (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65949 ms_dispat ceph rtd unknown /proc/64663/task/65949/root (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65949 ms_dispat ceph txt unknown /proc/64663/task/65949/exe (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65949 ms_dispat ceph NOFD /proc/64663/task/65949/fd (opendir: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65950 ms_local ceph cwd unknown /proc/64663/task/65950/cwd (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65950 ms_local ceph rtd unknown /proc/64663/task/65950/root (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65950 ms_local ceph txt unknown /proc/64663/task/65950/exe (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65950 ms_local ceph NOFD /proc/64663/task/65950/fd (opendir: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65951 safe_time ceph cwd unknown /proc/64663/task/65951/cwd (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65951 safe_time ceph rtd unknown /proc/64663/task/65951/root (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65951 safe_time ceph txt unknown /proc/64663/task/65951/exe (readlink: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65951 safe_time ceph NOFD /proc/64663/task/65951/fd (opendir: Permission denied) 2024-03-12T04:17:28.535 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65952 safe_time ceph cwd unknown /proc/64663/task/65952/cwd (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65952 safe_time ceph rtd unknown /proc/64663/task/65952/root (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65952 safe_time ceph txt unknown /proc/64663/task/65952/exe (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65952 safe_time ceph NOFD /proc/64663/task/65952/fd (opendir: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65953 ms_dispat ceph cwd unknown /proc/64663/task/65953/cwd (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65953 ms_dispat ceph rtd unknown /proc/64663/task/65953/root (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65953 ms_dispat ceph txt unknown /proc/64663/task/65953/exe (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65953 ms_dispat ceph NOFD /proc/64663/task/65953/fd (opendir: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65954 ms_local ceph cwd unknown /proc/64663/task/65954/cwd (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65954 ms_local ceph rtd unknown /proc/64663/task/65954/root (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65954 ms_local ceph txt unknown /proc/64663/task/65954/exe (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65954 ms_local ceph NOFD /proc/64663/task/65954/fd (opendir: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65955 ms_dispat ceph cwd unknown /proc/64663/task/65955/cwd (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65955 ms_dispat ceph rtd unknown /proc/64663/task/65955/root (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65955 ms_dispat ceph txt unknown /proc/64663/task/65955/exe (readlink: Permission denied) 2024-03-12T04:17:28.536 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65955 ms_dispat ceph NOFD /proc/64663/task/65955/fd (opendir: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65956 ms_local ceph cwd unknown /proc/64663/task/65956/cwd (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65956 ms_local ceph rtd unknown /proc/64663/task/65956/root (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65956 ms_local ceph txt unknown /proc/64663/task/65956/exe (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65956 ms_local ceph NOFD /proc/64663/task/65956/fd (opendir: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65957 ms_dispat ceph cwd unknown /proc/64663/task/65957/cwd (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65957 ms_dispat ceph rtd unknown /proc/64663/task/65957/root (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65957 ms_dispat ceph txt unknown /proc/64663/task/65957/exe (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65957 ms_dispat ceph NOFD /proc/64663/task/65957/fd (opendir: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65958 ms_local ceph cwd unknown /proc/64663/task/65958/cwd (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65958 ms_local ceph rtd unknown /proc/64663/task/65958/root (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65958 ms_local ceph txt unknown /proc/64663/task/65958/exe (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65958 ms_local ceph NOFD /proc/64663/task/65958/fd (opendir: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65959 ms_dispat ceph cwd unknown /proc/64663/task/65959/cwd (readlink: Permission denied) 2024-03-12T04:17:28.537 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65959 ms_dispat ceph rtd unknown /proc/64663/task/65959/root (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65959 ms_dispat ceph txt unknown /proc/64663/task/65959/exe (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65959 ms_dispat ceph NOFD /proc/64663/task/65959/fd (opendir: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65960 ms_local ceph cwd unknown /proc/64663/task/65960/cwd (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65960 ms_local ceph rtd unknown /proc/64663/task/65960/root (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65960 ms_local ceph txt unknown /proc/64663/task/65960/exe (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65960 ms_local ceph NOFD /proc/64663/task/65960/fd (opendir: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65961 ms_dispat ceph cwd unknown /proc/64663/task/65961/cwd (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65961 ms_dispat ceph rtd unknown /proc/64663/task/65961/root (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65961 ms_dispat ceph txt unknown /proc/64663/task/65961/exe (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65961 ms_dispat ceph NOFD /proc/64663/task/65961/fd (opendir: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65962 ms_local ceph cwd unknown /proc/64663/task/65962/cwd (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65962 ms_local ceph rtd unknown /proc/64663/task/65962/root (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65962 ms_local ceph txt unknown /proc/64663/task/65962/exe (readlink: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65962 ms_local ceph NOFD /proc/64663/task/65962/fd (opendir: Permission denied) 2024-03-12T04:17:28.538 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65963 ms_dispat ceph cwd unknown /proc/64663/task/65963/cwd (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65963 ms_dispat ceph rtd unknown /proc/64663/task/65963/root (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65963 ms_dispat ceph txt unknown /proc/64663/task/65963/exe (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65963 ms_dispat ceph NOFD /proc/64663/task/65963/fd (opendir: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65964 ms_local ceph cwd unknown /proc/64663/task/65964/cwd (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65964 ms_local ceph rtd unknown /proc/64663/task/65964/root (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65964 ms_local ceph txt unknown /proc/64663/task/65964/exe (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65964 ms_local ceph NOFD /proc/64663/task/65964/fd (opendir: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65965 fn_anonym ceph cwd unknown /proc/64663/task/65965/cwd (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65965 fn_anonym ceph rtd unknown /proc/64663/task/65965/root (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65965 fn_anonym ceph txt unknown /proc/64663/task/65965/exe (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65965 fn_anonym ceph NOFD /proc/64663/task/65965/fd (opendir: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65966 finisher ceph cwd unknown /proc/64663/task/65966/cwd (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65966 finisher ceph rtd unknown /proc/64663/task/65966/root (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65966 finisher ceph txt unknown /proc/64663/task/65966/exe (readlink: Permission denied) 2024-03-12T04:17:28.539 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65966 finisher ceph NOFD /proc/64663/task/65966/fd (opendir: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65967 safe_time ceph cwd unknown /proc/64663/task/65967/cwd (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65967 safe_time ceph rtd unknown /proc/64663/task/65967/root (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65967 safe_time ceph txt unknown /proc/64663/task/65967/exe (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65967 safe_time ceph NOFD /proc/64663/task/65967/fd (opendir: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65968 safe_time ceph cwd unknown /proc/64663/task/65968/cwd (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65968 safe_time ceph rtd unknown /proc/64663/task/65968/root (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65968 safe_time ceph txt unknown /proc/64663/task/65968/exe (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65968 safe_time ceph NOFD /proc/64663/task/65968/fd (opendir: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65969 ceph-osd ceph cwd unknown /proc/64663/task/65969/cwd (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65969 ceph-osd ceph rtd unknown /proc/64663/task/65969/root (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65969 ceph-osd ceph txt unknown /proc/64663/task/65969/exe (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65969 ceph-osd ceph NOFD /proc/64663/task/65969/fd (opendir: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65970 osd_srv_a ceph cwd unknown /proc/64663/task/65970/cwd (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65970 osd_srv_a ceph rtd unknown /proc/64663/task/65970/root (readlink: Permission denied) 2024-03-12T04:17:28.540 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65970 osd_srv_a ceph txt unknown /proc/64663/task/65970/exe (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65970 osd_srv_a ceph NOFD /proc/64663/task/65970/fd (opendir: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65971 tp_osd_tp ceph cwd unknown /proc/64663/task/65971/cwd (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65971 tp_osd_tp ceph rtd unknown /proc/64663/task/65971/root (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65971 tp_osd_tp ceph txt unknown /proc/64663/task/65971/exe (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65971 tp_osd_tp ceph NOFD /proc/64663/task/65971/fd (opendir: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65972 tp_osd_tp ceph cwd unknown /proc/64663/task/65972/cwd (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65972 tp_osd_tp ceph rtd unknown /proc/64663/task/65972/root (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65972 tp_osd_tp ceph txt unknown /proc/64663/task/65972/exe (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65972 tp_osd_tp ceph NOFD /proc/64663/task/65972/fd (opendir: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65973 tp_osd_tp ceph cwd unknown /proc/64663/task/65973/cwd (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65973 tp_osd_tp ceph rtd unknown /proc/64663/task/65973/root (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65973 tp_osd_tp ceph txt unknown /proc/64663/task/65973/exe (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65973 tp_osd_tp ceph NOFD /proc/64663/task/65973/fd (opendir: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65974 tp_osd_tp ceph cwd unknown /proc/64663/task/65974/cwd (readlink: Permission denied) 2024-03-12T04:17:28.541 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65974 tp_osd_tp ceph rtd unknown /proc/64663/task/65974/root (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65974 tp_osd_tp ceph txt unknown /proc/64663/task/65974/exe (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65974 tp_osd_tp ceph NOFD /proc/64663/task/65974/fd (opendir: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65975 tp_osd_tp ceph cwd unknown /proc/64663/task/65975/cwd (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65975 tp_osd_tp ceph rtd unknown /proc/64663/task/65975/root (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65975 tp_osd_tp ceph txt unknown /proc/64663/task/65975/exe (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65975 tp_osd_tp ceph NOFD /proc/64663/task/65975/fd (opendir: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65976 tp_osd_tp ceph cwd unknown /proc/64663/task/65976/cwd (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65976 tp_osd_tp ceph rtd unknown /proc/64663/task/65976/root (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65976 tp_osd_tp ceph txt unknown /proc/64663/task/65976/exe (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65976 tp_osd_tp ceph NOFD /proc/64663/task/65976/fd (opendir: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65977 tp_osd_tp ceph cwd unknown /proc/64663/task/65977/cwd (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65977 tp_osd_tp ceph rtd unknown /proc/64663/task/65977/root (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65977 tp_osd_tp ceph txt unknown /proc/64663/task/65977/exe (readlink: Permission denied) 2024-03-12T04:17:28.542 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65977 tp_osd_tp ceph NOFD /proc/64663/task/65977/fd (opendir: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65978 tp_osd_tp ceph cwd unknown /proc/64663/task/65978/cwd (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65978 tp_osd_tp ceph rtd unknown /proc/64663/task/65978/root (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65978 tp_osd_tp ceph txt unknown /proc/64663/task/65978/exe (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65978 tp_osd_tp ceph NOFD /proc/64663/task/65978/fd (opendir: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65979 tp_osd_tp ceph cwd unknown /proc/64663/task/65979/cwd (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65979 tp_osd_tp ceph rtd unknown /proc/64663/task/65979/root (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65979 tp_osd_tp ceph txt unknown /proc/64663/task/65979/exe (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65979 tp_osd_tp ceph NOFD /proc/64663/task/65979/fd (opendir: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65980 tp_osd_tp ceph cwd unknown /proc/64663/task/65980/cwd (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65980 tp_osd_tp ceph rtd unknown /proc/64663/task/65980/root (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65980 tp_osd_tp ceph txt unknown /proc/64663/task/65980/exe (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65980 tp_osd_tp ceph NOFD /proc/64663/task/65980/fd (opendir: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65981 tp_osd_tp ceph cwd unknown /proc/64663/task/65981/cwd (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65981 tp_osd_tp ceph rtd unknown /proc/64663/task/65981/root (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65981 tp_osd_tp ceph txt unknown /proc/64663/task/65981/exe (readlink: Permission denied) 2024-03-12T04:17:28.543 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65981 tp_osd_tp ceph NOFD /proc/64663/task/65981/fd (opendir: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65982 tp_osd_tp ceph cwd unknown /proc/64663/task/65982/cwd (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65982 tp_osd_tp ceph rtd unknown /proc/64663/task/65982/root (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65982 tp_osd_tp ceph txt unknown /proc/64663/task/65982/exe (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65982 tp_osd_tp ceph NOFD /proc/64663/task/65982/fd (opendir: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65983 tp_osd_tp ceph cwd unknown /proc/64663/task/65983/cwd (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65983 tp_osd_tp ceph rtd unknown /proc/64663/task/65983/root (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65983 tp_osd_tp ceph txt unknown /proc/64663/task/65983/exe (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65983 tp_osd_tp ceph NOFD /proc/64663/task/65983/fd (opendir: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65984 tp_osd_tp ceph cwd unknown /proc/64663/task/65984/cwd (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65984 tp_osd_tp ceph rtd unknown /proc/64663/task/65984/root (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65984 tp_osd_tp ceph txt unknown /proc/64663/task/65984/exe (readlink: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65984 tp_osd_tp ceph NOFD /proc/64663/task/65984/fd (opendir: Permission denied) 2024-03-12T04:17:28.544 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65985 tp_osd_tp ceph cwd unknown /proc/64663/task/65985/cwd (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65985 tp_osd_tp ceph rtd unknown /proc/64663/task/65985/root (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65985 tp_osd_tp ceph txt unknown /proc/64663/task/65985/exe (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65985 tp_osd_tp ceph NOFD /proc/64663/task/65985/fd (opendir: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65986 tp_osd_tp ceph cwd unknown /proc/64663/task/65986/cwd (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65986 tp_osd_tp ceph rtd unknown /proc/64663/task/65986/root (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65986 tp_osd_tp ceph txt unknown /proc/64663/task/65986/exe (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65986 tp_osd_tp ceph NOFD /proc/64663/task/65986/fd (opendir: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65987 osd_srv_h ceph cwd unknown /proc/64663/task/65987/cwd (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65987 osd_srv_h ceph rtd unknown /proc/64663/task/65987/root (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65987 osd_srv_h ceph txt unknown /proc/64663/task/65987/exe (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-osd 64663 65987 osd_srv_h ceph NOFD /proc/64663/task/65987/fd (opendir: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 65989 root cwd unknown /proc/65989/cwd (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 65989 root rtd unknown /proc/65989/root (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 65989 root txt unknown /proc/65989/exe (readlink: Permission denied) 2024-03-12T04:17:28.545 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 65989 root NOFD /proc/65989/fd (opendir: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 66006 root cwd unknown /proc/66006/cwd (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 66006 root rtd unknown /proc/66006/root (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 66006 root txt unknown /proc/66006/exe (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 66006 root NOFD /proc/66006/fd (opendir: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69293 root cwd unknown /proc/69293/cwd (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69293 root rtd unknown /proc/69293/root (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69293 root txt unknown /proc/69293/exe (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69293 root NOFD /proc/69293/fd (opendir: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69310 root cwd unknown /proc/69310/cwd (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69310 root rtd unknown /proc/69310/root (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69310 root txt unknown /proc/69310/exe (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69310 root NOFD /proc/69310/fd (opendir: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69311 root cwd unknown /proc/69311/cwd (readlink: Permission denied) 2024-03-12T04:17:28.546 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69311 root rtd unknown /proc/69311/root (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69311 root txt unknown /proc/69311/exe (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69311 root NOFD /proc/69311/fd (opendir: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69328 root cwd unknown /proc/69328/cwd (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69328 root rtd unknown /proc/69328/root (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69328 root txt unknown /proc/69328/exe (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69328 root NOFD /proc/69328/fd (opendir: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69329 root cwd unknown /proc/69329/cwd (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69329 root rtd unknown /proc/69329/root (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69329 root txt unknown /proc/69329/exe (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69329 root NOFD /proc/69329/fd (opendir: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69346 root cwd unknown /proc/69346/cwd (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69346 root rtd unknown /proc/69346/root (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69346 root txt unknown /proc/69346/exe (readlink: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69346 root NOFD /proc/69346/fd (opendir: Permission denied) 2024-03-12T04:17:28.547 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69347 root cwd unknown /proc/69347/cwd (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69347 root rtd unknown /proc/69347/root (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69347 root txt unknown /proc/69347/exe (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 69347 root NOFD /proc/69347/fd (opendir: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69364 root cwd unknown /proc/69364/cwd (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69364 root rtd unknown /proc/69364/root (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69364 root txt unknown /proc/69364/exe (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:journalct 69364 root NOFD /proc/69364/fd (opendir: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 69892 root cwd unknown /proc/69892/cwd (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 69892 root rtd unknown /proc/69892/root (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 69892 root txt unknown /proc/69892/exe (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 69892 root NOFD /proc/69892/fd (opendir: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 69894 root cwd unknown /proc/69894/cwd (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 69894 root rtd unknown /proc/69894/root (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 69894 root txt unknown /proc/69894/exe (readlink: Permission denied) 2024-03-12T04:17:28.548 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 69894 root NOFD /proc/69894/fd (opendir: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 ceph cwd unknown /proc/69896/cwd (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 ceph rtd unknown /proc/69896/root (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 ceph txt unknown /proc/69896/exe (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 ceph NOFD /proc/69896/fd (opendir: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69897 log ceph cwd unknown /proc/69896/task/69897/cwd (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69897 log ceph rtd unknown /proc/69896/task/69897/root (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69897 log ceph txt unknown /proc/69896/task/69897/exe (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69897 log ceph NOFD /proc/69896/task/69897/fd (opendir: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69899 msgr-work ceph cwd unknown /proc/69896/task/69899/cwd (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69899 msgr-work ceph rtd unknown /proc/69896/task/69899/root (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69899 msgr-work ceph txt unknown /proc/69896/task/69899/exe (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69899 msgr-work ceph NOFD /proc/69896/task/69899/fd (opendir: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69900 msgr-work ceph cwd unknown /proc/69896/task/69900/cwd (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69900 msgr-work ceph rtd unknown /proc/69896/task/69900/root (readlink: Permission denied) 2024-03-12T04:17:28.549 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69900 msgr-work ceph txt unknown /proc/69896/task/69900/exe (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69900 msgr-work ceph NOFD /proc/69896/task/69900/fd (opendir: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69901 msgr-work ceph cwd unknown /proc/69896/task/69901/cwd (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69901 msgr-work ceph rtd unknown /proc/69896/task/69901/root (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69901 msgr-work ceph txt unknown /proc/69896/task/69901/exe (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69901 msgr-work ceph NOFD /proc/69896/task/69901/fd (opendir: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69905 service ceph cwd unknown /proc/69896/task/69905/cwd (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69905 service ceph rtd unknown /proc/69896/task/69905/root (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69905 service ceph txt unknown /proc/69896/task/69905/exe (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69905 service ceph NOFD /proc/69896/task/69905/fd (opendir: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69906 admin_soc ceph cwd unknown /proc/69896/task/69906/cwd (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69906 admin_soc ceph rtd unknown /proc/69896/task/69906/root (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69906 admin_soc ceph txt unknown /proc/69896/task/69906/exe (readlink: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69906 admin_soc ceph NOFD /proc/69896/task/69906/fd (opendir: Permission denied) 2024-03-12T04:17:28.550 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69907 signal_ha ceph cwd unknown /proc/69896/task/69907/cwd (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69907 signal_ha ceph rtd unknown /proc/69896/task/69907/root (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69907 signal_ha ceph txt unknown /proc/69896/task/69907/exe (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69907 signal_ha ceph NOFD /proc/69896/task/69907/fd (opendir: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69908 io_contex ceph cwd unknown /proc/69896/task/69908/cwd (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69908 io_contex ceph rtd unknown /proc/69896/task/69908/root (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69908 io_contex ceph txt unknown /proc/69896/task/69908/exe (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69908 io_contex ceph NOFD /proc/69896/task/69908/fd (opendir: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69909 io_contex ceph cwd unknown /proc/69896/task/69909/cwd (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69909 io_contex ceph rtd unknown /proc/69896/task/69909/root (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69909 io_contex ceph txt unknown /proc/69896/task/69909/exe (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69909 io_contex ceph NOFD /proc/69896/task/69909/fd (opendir: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69910 ms_dispat ceph cwd unknown /proc/69896/task/69910/cwd (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69910 ms_dispat ceph rtd unknown /proc/69896/task/69910/root (readlink: Permission denied) 2024-03-12T04:17:28.551 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69910 ms_dispat ceph txt unknown /proc/69896/task/69910/exe (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69910 ms_dispat ceph NOFD /proc/69896/task/69910/fd (opendir: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69911 ms_local ceph cwd unknown /proc/69896/task/69911/cwd (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69911 ms_local ceph rtd unknown /proc/69896/task/69911/root (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69911 ms_local ceph txt unknown /proc/69896/task/69911/exe (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69911 ms_local ceph NOFD /proc/69896/task/69911/fd (opendir: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69912 safe_time ceph cwd unknown /proc/69896/task/69912/cwd (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69912 safe_time ceph rtd unknown /proc/69896/task/69912/root (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69912 safe_time ceph txt unknown /proc/69896/task/69912/exe (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69912 safe_time ceph NOFD /proc/69896/task/69912/fd (opendir: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69913 safe_time ceph cwd unknown /proc/69896/task/69913/cwd (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69913 safe_time ceph rtd unknown /proc/69896/task/69913/root (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69913 safe_time ceph txt unknown /proc/69896/task/69913/exe (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69913 safe_time ceph NOFD /proc/69896/task/69913/fd (opendir: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69914 ceph-mds ceph cwd unknown /proc/69896/task/69914/cwd (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69914 ceph-mds ceph rtd unknown /proc/69896/task/69914/root (readlink: Permission denied) 2024-03-12T04:17:28.552 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69914 ceph-mds ceph txt unknown /proc/69896/task/69914/exe (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 69914 ceph-mds ceph NOFD /proc/69896/task/69914/fd (opendir: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 70024 safe_time ceph cwd unknown /proc/69896/task/70024/cwd (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 70024 safe_time ceph rtd unknown /proc/69896/task/70024/root (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 70024 safe_time ceph txt unknown /proc/69896/task/70024/exe (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 69896 70024 safe_time ceph NOFD /proc/69896/task/70024/fd (opendir: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70246 root cwd unknown /proc/70246/cwd (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70246 root rtd unknown /proc/70246/root (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70246 root txt unknown /proc/70246/exe (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70246 root NOFD /proc/70246/fd (opendir: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70248 root cwd unknown /proc/70248/cwd (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70248 root rtd unknown /proc/70248/root (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70248 root txt unknown /proc/70248/exe (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70248 root NOFD /proc/70248/fd (opendir: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 ceph cwd unknown /proc/70250/cwd (readlink: Permission denied) 2024-03-12T04:17:28.553 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 ceph rtd unknown /proc/70250/root (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 ceph txt unknown /proc/70250/exe (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 ceph NOFD /proc/70250/fd (opendir: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70251 log ceph cwd unknown /proc/70250/task/70251/cwd (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70251 log ceph rtd unknown /proc/70250/task/70251/root (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70251 log ceph txt unknown /proc/70250/task/70251/exe (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70251 log ceph NOFD /proc/70250/task/70251/fd (opendir: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70253 msgr-work ceph cwd unknown /proc/70250/task/70253/cwd (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70253 msgr-work ceph rtd unknown /proc/70250/task/70253/root (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70253 msgr-work ceph txt unknown /proc/70250/task/70253/exe (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70253 msgr-work ceph NOFD /proc/70250/task/70253/fd (opendir: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70254 msgr-work ceph cwd unknown /proc/70250/task/70254/cwd (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70254 msgr-work ceph rtd unknown /proc/70250/task/70254/root (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70254 msgr-work ceph txt unknown /proc/70250/task/70254/exe (readlink: Permission denied) 2024-03-12T04:17:28.554 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70254 msgr-work ceph NOFD /proc/70250/task/70254/fd (opendir: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70255 msgr-work ceph cwd unknown /proc/70250/task/70255/cwd (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70255 msgr-work ceph rtd unknown /proc/70250/task/70255/root (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70255 msgr-work ceph txt unknown /proc/70250/task/70255/exe (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70255 msgr-work ceph NOFD /proc/70250/task/70255/fd (opendir: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70259 service ceph cwd unknown /proc/70250/task/70259/cwd (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70259 service ceph rtd unknown /proc/70250/task/70259/root (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70259 service ceph txt unknown /proc/70250/task/70259/exe (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70259 service ceph NOFD /proc/70250/task/70259/fd (opendir: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70260 admin_soc ceph cwd unknown /proc/70250/task/70260/cwd (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70260 admin_soc ceph rtd unknown /proc/70250/task/70260/root (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70260 admin_soc ceph txt unknown /proc/70250/task/70260/exe (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70260 admin_soc ceph NOFD /proc/70250/task/70260/fd (opendir: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70261 signal_ha ceph cwd unknown /proc/70250/task/70261/cwd (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70261 signal_ha ceph rtd unknown /proc/70250/task/70261/root (readlink: Permission denied) 2024-03-12T04:17:28.555 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70261 signal_ha ceph txt unknown /proc/70250/task/70261/exe (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70261 signal_ha ceph NOFD /proc/70250/task/70261/fd (opendir: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70262 io_contex ceph cwd unknown /proc/70250/task/70262/cwd (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70262 io_contex ceph rtd unknown /proc/70250/task/70262/root (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70262 io_contex ceph txt unknown /proc/70250/task/70262/exe (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70262 io_contex ceph NOFD /proc/70250/task/70262/fd (opendir: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70263 io_contex ceph cwd unknown /proc/70250/task/70263/cwd (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70263 io_contex ceph rtd unknown /proc/70250/task/70263/root (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70263 io_contex ceph txt unknown /proc/70250/task/70263/exe (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70263 io_contex ceph NOFD /proc/70250/task/70263/fd (opendir: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70264 ms_dispat ceph cwd unknown /proc/70250/task/70264/cwd (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70264 ms_dispat ceph rtd unknown /proc/70250/task/70264/root (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70264 ms_dispat ceph txt unknown /proc/70250/task/70264/exe (readlink: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70264 ms_dispat ceph NOFD /proc/70250/task/70264/fd (opendir: Permission denied) 2024-03-12T04:17:28.556 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70265 ms_local ceph cwd unknown /proc/70250/task/70265/cwd (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70265 ms_local ceph rtd unknown /proc/70250/task/70265/root (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70265 ms_local ceph txt unknown /proc/70250/task/70265/exe (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70265 ms_local ceph NOFD /proc/70250/task/70265/fd (opendir: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70266 safe_time ceph cwd unknown /proc/70250/task/70266/cwd (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70266 safe_time ceph rtd unknown /proc/70250/task/70266/root (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70266 safe_time ceph txt unknown /proc/70250/task/70266/exe (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70266 safe_time ceph NOFD /proc/70250/task/70266/fd (opendir: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70267 safe_time ceph cwd unknown /proc/70250/task/70267/cwd (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70267 safe_time ceph rtd unknown /proc/70250/task/70267/root (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70267 safe_time ceph txt unknown /proc/70250/task/70267/exe (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70267 safe_time ceph NOFD /proc/70250/task/70267/fd (opendir: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70268 ceph-mds ceph cwd unknown /proc/70250/task/70268/cwd (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70268 ceph-mds ceph rtd unknown /proc/70250/task/70268/root (readlink: Permission denied) 2024-03-12T04:17:28.557 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70268 ceph-mds ceph txt unknown /proc/70250/task/70268/exe (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70268 ceph-mds ceph NOFD /proc/70250/task/70268/fd (opendir: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70273 safe_time ceph cwd unknown /proc/70250/task/70273/cwd (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70273 safe_time ceph rtd unknown /proc/70250/task/70273/root (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70273 safe_time ceph txt unknown /proc/70250/task/70273/exe (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70250 70273 safe_time ceph NOFD /proc/70250/task/70273/fd (opendir: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70855 root cwd unknown /proc/70855/cwd (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70855 root rtd unknown /proc/70855/root (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70855 root txt unknown /proc/70855/exe (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 70855 root NOFD /proc/70855/fd (opendir: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70857 root cwd unknown /proc/70857/cwd (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70857 root rtd unknown /proc/70857/root (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70857 root txt unknown /proc/70857/exe (readlink: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 70857 root NOFD /proc/70857/fd (opendir: Permission denied) 2024-03-12T04:17:28.558 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 ceph cwd unknown /proc/70859/cwd (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 ceph rtd unknown /proc/70859/root (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 ceph txt unknown /proc/70859/exe (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 ceph NOFD /proc/70859/fd (opendir: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70860 log ceph cwd unknown /proc/70859/task/70860/cwd (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70860 log ceph rtd unknown /proc/70859/task/70860/root (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70860 log ceph txt unknown /proc/70859/task/70860/exe (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70860 log ceph NOFD /proc/70859/task/70860/fd (opendir: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70862 msgr-work ceph cwd unknown /proc/70859/task/70862/cwd (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70862 msgr-work ceph rtd unknown /proc/70859/task/70862/root (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70862 msgr-work ceph txt unknown /proc/70859/task/70862/exe (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70862 msgr-work ceph NOFD /proc/70859/task/70862/fd (opendir: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70863 msgr-work ceph cwd unknown /proc/70859/task/70863/cwd (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70863 msgr-work ceph rtd unknown /proc/70859/task/70863/root (readlink: Permission denied) 2024-03-12T04:17:28.559 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70863 msgr-work ceph txt unknown /proc/70859/task/70863/exe (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70863 msgr-work ceph NOFD /proc/70859/task/70863/fd (opendir: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70864 msgr-work ceph cwd unknown /proc/70859/task/70864/cwd (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70864 msgr-work ceph rtd unknown /proc/70859/task/70864/root (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70864 msgr-work ceph txt unknown /proc/70859/task/70864/exe (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70864 msgr-work ceph NOFD /proc/70859/task/70864/fd (opendir: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70868 service ceph cwd unknown /proc/70859/task/70868/cwd (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70868 service ceph rtd unknown /proc/70859/task/70868/root (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70868 service ceph txt unknown /proc/70859/task/70868/exe (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70868 service ceph NOFD /proc/70859/task/70868/fd (opendir: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70869 admin_soc ceph cwd unknown /proc/70859/task/70869/cwd (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70869 admin_soc ceph rtd unknown /proc/70859/task/70869/root (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70869 admin_soc ceph txt unknown /proc/70859/task/70869/exe (readlink: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70869 admin_soc ceph NOFD /proc/70859/task/70869/fd (opendir: Permission denied) 2024-03-12T04:17:28.560 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70870 signal_ha ceph cwd unknown /proc/70859/task/70870/cwd (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70870 signal_ha ceph rtd unknown /proc/70859/task/70870/root (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70870 signal_ha ceph txt unknown /proc/70859/task/70870/exe (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70870 signal_ha ceph NOFD /proc/70859/task/70870/fd (opendir: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70871 io_contex ceph cwd unknown /proc/70859/task/70871/cwd (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70871 io_contex ceph rtd unknown /proc/70859/task/70871/root (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70871 io_contex ceph txt unknown /proc/70859/task/70871/exe (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70871 io_contex ceph NOFD /proc/70859/task/70871/fd (opendir: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70872 io_contex ceph cwd unknown /proc/70859/task/70872/cwd (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70872 io_contex ceph rtd unknown /proc/70859/task/70872/root (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70872 io_contex ceph txt unknown /proc/70859/task/70872/exe (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70872 io_contex ceph NOFD /proc/70859/task/70872/fd (opendir: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70873 ms_dispat ceph cwd unknown /proc/70859/task/70873/cwd (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70873 ms_dispat ceph rtd unknown /proc/70859/task/70873/root (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70873 ms_dispat ceph txt unknown /proc/70859/task/70873/exe (readlink: Permission denied) 2024-03-12T04:17:28.561 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70873 ms_dispat ceph NOFD /proc/70859/task/70873/fd (opendir: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70874 ms_local ceph cwd unknown /proc/70859/task/70874/cwd (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70874 ms_local ceph rtd unknown /proc/70859/task/70874/root (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70874 ms_local ceph txt unknown /proc/70859/task/70874/exe (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70874 ms_local ceph NOFD /proc/70859/task/70874/fd (opendir: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70875 safe_time ceph cwd unknown /proc/70859/task/70875/cwd (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70875 safe_time ceph rtd unknown /proc/70859/task/70875/root (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70875 safe_time ceph txt unknown /proc/70859/task/70875/exe (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70875 safe_time ceph NOFD /proc/70859/task/70875/fd (opendir: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70876 safe_time ceph cwd unknown /proc/70859/task/70876/cwd (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70876 safe_time ceph rtd unknown /proc/70859/task/70876/root (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70876 safe_time ceph txt unknown /proc/70859/task/70876/exe (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70876 safe_time ceph NOFD /proc/70859/task/70876/fd (opendir: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70877 ceph-mds ceph cwd unknown /proc/70859/task/70877/cwd (readlink: Permission denied) 2024-03-12T04:17:28.562 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70877 ceph-mds ceph rtd unknown /proc/70859/task/70877/root (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70877 ceph-mds ceph txt unknown /proc/70859/task/70877/exe (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70877 ceph-mds ceph NOFD /proc/70859/task/70877/fd (opendir: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70948 safe_time ceph cwd unknown /proc/70859/task/70948/cwd (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70948 safe_time ceph rtd unknown /proc/70859/task/70948/root (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70948 safe_time ceph txt unknown /proc/70859/task/70948/exe (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 70948 safe_time ceph NOFD /proc/70859/task/70948/fd (opendir: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72891 ceph_time ceph cwd unknown /proc/70859/task/72891/cwd (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72891 ceph_time ceph rtd unknown /proc/70859/task/72891/root (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72891 ceph_time ceph txt unknown /proc/70859/task/72891/exe (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72891 ceph_time ceph NOFD /proc/70859/task/72891/fd (opendir: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72892 OpHistory ceph cwd unknown /proc/70859/task/72892/cwd (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72892 OpHistory ceph rtd unknown /proc/70859/task/72892/root (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72892 OpHistory ceph txt unknown /proc/70859/task/72892/exe (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72892 OpHistory ceph NOFD /proc/70859/task/72892/fd (opendir: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72893 ms_dispat ceph cwd unknown /proc/70859/task/72893/cwd (readlink: Permission denied) 2024-03-12T04:17:28.563 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72893 ms_dispat ceph rtd unknown /proc/70859/task/72893/root (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72893 ms_dispat ceph txt unknown /proc/70859/task/72893/exe (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72893 ms_dispat ceph NOFD /proc/70859/task/72893/fd (opendir: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72894 ms_dispat ceph cwd unknown /proc/70859/task/72894/cwd (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72894 ms_dispat ceph rtd unknown /proc/70859/task/72894/root (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72894 ms_dispat ceph txt unknown /proc/70859/task/72894/exe (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72894 ms_dispat ceph NOFD /proc/70859/task/72894/fd (opendir: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72895 mds_rank_ ceph cwd unknown /proc/70859/task/72895/cwd (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72895 mds_rank_ ceph rtd unknown /proc/70859/task/72895/root (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72895 mds_rank_ ceph txt unknown /proc/70859/task/72895/exe (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72895 mds_rank_ ceph NOFD /proc/70859/task/72895/fd (opendir: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72896 PQ_Finish ceph cwd unknown /proc/70859/task/72896/cwd (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72896 PQ_Finish ceph rtd unknown /proc/70859/task/72896/root (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72896 PQ_Finish ceph txt unknown /proc/70859/task/72896/exe (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72896 PQ_Finish ceph NOFD /proc/70859/task/72896/fd (opendir: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72897 safe_time ceph cwd unknown /proc/70859/task/72897/cwd (readlink: Permission denied) 2024-03-12T04:17:28.564 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72897 safe_time ceph rtd unknown /proc/70859/task/72897/root (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72897 safe_time ceph txt unknown /proc/70859/task/72897/exe (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72897 safe_time ceph NOFD /proc/70859/task/72897/fd (opendir: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72898 MR_Finish ceph cwd unknown /proc/70859/task/72898/cwd (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72898 MR_Finish ceph rtd unknown /proc/70859/task/72898/root (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72898 MR_Finish ceph txt unknown /proc/70859/task/72898/exe (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72898 MR_Finish ceph NOFD /proc/70859/task/72898/fd (opendir: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72900 md_submit ceph cwd unknown /proc/70859/task/72900/cwd (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72900 md_submit ceph rtd unknown /proc/70859/task/72900/root (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72900 md_submit ceph txt unknown /proc/70859/task/72900/exe (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 70859 72900 md_submit ceph NOFD /proc/70859/task/72900/fd (opendir: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 71204 root cwd unknown /proc/71204/cwd (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 71204 root rtd unknown /proc/71204/root (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 71204 root txt unknown /proc/71204/exe (readlink: Permission denied) 2024-03-12T04:17:28.565 INFO:tasks.workunit.client.0.smithi042.stdout:conmon 71204 root NOFD /proc/71204/fd (opendir: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 71206 root cwd unknown /proc/71206/cwd (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 71206 root rtd unknown /proc/71206/root (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 71206 root txt unknown /proc/71206/exe (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:podman-in 71206 root NOFD /proc/71206/fd (opendir: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 ceph cwd unknown /proc/71208/cwd (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 ceph rtd unknown /proc/71208/root (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 ceph txt unknown /proc/71208/exe (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 ceph NOFD /proc/71208/fd (opendir: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71209 log ceph cwd unknown /proc/71208/task/71209/cwd (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71209 log ceph rtd unknown /proc/71208/task/71209/root (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71209 log ceph txt unknown /proc/71208/task/71209/exe (readlink: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71209 log ceph NOFD /proc/71208/task/71209/fd (opendir: Permission denied) 2024-03-12T04:17:28.566 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71211 msgr-work ceph cwd unknown /proc/71208/task/71211/cwd (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71211 msgr-work ceph rtd unknown /proc/71208/task/71211/root (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71211 msgr-work ceph txt unknown /proc/71208/task/71211/exe (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71211 msgr-work ceph NOFD /proc/71208/task/71211/fd (opendir: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71212 msgr-work ceph cwd unknown /proc/71208/task/71212/cwd (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71212 msgr-work ceph rtd unknown /proc/71208/task/71212/root (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71212 msgr-work ceph txt unknown /proc/71208/task/71212/exe (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71212 msgr-work ceph NOFD /proc/71208/task/71212/fd (opendir: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71213 msgr-work ceph cwd unknown /proc/71208/task/71213/cwd (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71213 msgr-work ceph rtd unknown /proc/71208/task/71213/root (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71213 msgr-work ceph txt unknown /proc/71208/task/71213/exe (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71213 msgr-work ceph NOFD /proc/71208/task/71213/fd (opendir: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71217 service ceph cwd unknown /proc/71208/task/71217/cwd (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71217 service ceph rtd unknown /proc/71208/task/71217/root (readlink: Permission denied) 2024-03-12T04:17:28.567 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71217 service ceph txt unknown /proc/71208/task/71217/exe (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71217 service ceph NOFD /proc/71208/task/71217/fd (opendir: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71218 admin_soc ceph cwd unknown /proc/71208/task/71218/cwd (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71218 admin_soc ceph rtd unknown /proc/71208/task/71218/root (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71218 admin_soc ceph txt unknown /proc/71208/task/71218/exe (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71218 admin_soc ceph NOFD /proc/71208/task/71218/fd (opendir: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71219 signal_ha ceph cwd unknown /proc/71208/task/71219/cwd (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71219 signal_ha ceph rtd unknown /proc/71208/task/71219/root (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71219 signal_ha ceph txt unknown /proc/71208/task/71219/exe (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71219 signal_ha ceph NOFD /proc/71208/task/71219/fd (opendir: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71220 io_contex ceph cwd unknown /proc/71208/task/71220/cwd (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71220 io_contex ceph rtd unknown /proc/71208/task/71220/root (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71220 io_contex ceph txt unknown /proc/71208/task/71220/exe (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71220 io_contex ceph NOFD /proc/71208/task/71220/fd (opendir: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71221 io_contex ceph cwd unknown /proc/71208/task/71221/cwd (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71221 io_contex ceph rtd unknown /proc/71208/task/71221/root (readlink: Permission denied) 2024-03-12T04:17:28.568 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71221 io_contex ceph txt unknown /proc/71208/task/71221/exe (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71221 io_contex ceph NOFD /proc/71208/task/71221/fd (opendir: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71222 ms_dispat ceph cwd unknown /proc/71208/task/71222/cwd (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71222 ms_dispat ceph rtd unknown /proc/71208/task/71222/root (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71222 ms_dispat ceph txt unknown /proc/71208/task/71222/exe (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71222 ms_dispat ceph NOFD /proc/71208/task/71222/fd (opendir: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71223 ms_local ceph cwd unknown /proc/71208/task/71223/cwd (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71223 ms_local ceph rtd unknown /proc/71208/task/71223/root (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71223 ms_local ceph txt unknown /proc/71208/task/71223/exe (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71223 ms_local ceph NOFD /proc/71208/task/71223/fd (opendir: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71224 safe_time ceph cwd unknown /proc/71208/task/71224/cwd (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71224 safe_time ceph rtd unknown /proc/71208/task/71224/root (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71224 safe_time ceph txt unknown /proc/71208/task/71224/exe (readlink: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71224 safe_time ceph NOFD /proc/71208/task/71224/fd (opendir: Permission denied) 2024-03-12T04:17:28.569 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71225 safe_time ceph cwd unknown /proc/71208/task/71225/cwd (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71225 safe_time ceph rtd unknown /proc/71208/task/71225/root (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71225 safe_time ceph txt unknown /proc/71208/task/71225/exe (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71225 safe_time ceph NOFD /proc/71208/task/71225/fd (opendir: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71226 ceph-mds ceph cwd unknown /proc/71208/task/71226/cwd (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71226 ceph-mds ceph rtd unknown /proc/71208/task/71226/root (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71226 ceph-mds ceph txt unknown /proc/71208/task/71226/exe (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71226 ceph-mds ceph NOFD /proc/71208/task/71226/fd (opendir: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71272 safe_time ceph cwd unknown /proc/71208/task/71272/cwd (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71272 safe_time ceph rtd unknown /proc/71208/task/71272/root (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71272 safe_time ceph txt unknown /proc/71208/task/71272/exe (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 71272 safe_time ceph NOFD /proc/71208/task/71272/fd (opendir: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72757 ceph_time ceph cwd unknown /proc/71208/task/72757/cwd (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72757 ceph_time ceph rtd unknown /proc/71208/task/72757/root (readlink: Permission denied) 2024-03-12T04:17:28.570 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72757 ceph_time ceph txt unknown /proc/71208/task/72757/exe (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72757 ceph_time ceph NOFD /proc/71208/task/72757/fd (opendir: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72758 OpHistory ceph cwd unknown /proc/71208/task/72758/cwd (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72758 OpHistory ceph rtd unknown /proc/71208/task/72758/root (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72758 OpHistory ceph txt unknown /proc/71208/task/72758/exe (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72758 OpHistory ceph NOFD /proc/71208/task/72758/fd (opendir: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72759 ms_dispat ceph cwd unknown /proc/71208/task/72759/cwd (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72759 ms_dispat ceph rtd unknown /proc/71208/task/72759/root (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72759 ms_dispat ceph txt unknown /proc/71208/task/72759/exe (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72759 ms_dispat ceph NOFD /proc/71208/task/72759/fd (opendir: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72760 ms_dispat ceph cwd unknown /proc/71208/task/72760/cwd (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72760 ms_dispat ceph rtd unknown /proc/71208/task/72760/root (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72760 ms_dispat ceph txt unknown /proc/71208/task/72760/exe (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72760 ms_dispat ceph NOFD /proc/71208/task/72760/fd (opendir: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72761 mds_rank_ ceph cwd unknown /proc/71208/task/72761/cwd (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72761 mds_rank_ ceph rtd unknown /proc/71208/task/72761/root (readlink: Permission denied) 2024-03-12T04:17:28.571 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72761 mds_rank_ ceph txt unknown /proc/71208/task/72761/exe (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72761 mds_rank_ ceph NOFD /proc/71208/task/72761/fd (opendir: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72762 PQ_Finish ceph cwd unknown /proc/71208/task/72762/cwd (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72762 PQ_Finish ceph rtd unknown /proc/71208/task/72762/root (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72762 PQ_Finish ceph txt unknown /proc/71208/task/72762/exe (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72762 PQ_Finish ceph NOFD /proc/71208/task/72762/fd (opendir: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72763 safe_time ceph cwd unknown /proc/71208/task/72763/cwd (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72763 safe_time ceph rtd unknown /proc/71208/task/72763/root (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72763 safe_time ceph txt unknown /proc/71208/task/72763/exe (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72763 safe_time ceph NOFD /proc/71208/task/72763/fd (opendir: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72764 MR_Finish ceph cwd unknown /proc/71208/task/72764/cwd (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72764 MR_Finish ceph rtd unknown /proc/71208/task/72764/root (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72764 MR_Finish ceph txt unknown /proc/71208/task/72764/exe (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72764 MR_Finish ceph NOFD /proc/71208/task/72764/fd (opendir: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72765 md_submit ceph cwd unknown /proc/71208/task/72765/cwd (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72765 md_submit ceph rtd unknown /proc/71208/task/72765/root (readlink: Permission denied) 2024-03-12T04:17:28.572 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72765 md_submit ceph txt unknown /proc/71208/task/72765/exe (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72765 md_submit ceph NOFD /proc/71208/task/72765/fd (opendir: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72890 ms_dispat ceph cwd unknown /proc/71208/task/72890/cwd (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72890 ms_dispat ceph rtd unknown /proc/71208/task/72890/root (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72890 ms_dispat ceph txt unknown /proc/71208/task/72890/exe (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-mds 71208 72890 ms_dispat ceph NOFD /proc/71208/task/72890/fd (opendir: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 72807 root cwd unknown /proc/72807/cwd (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 72807 root rtd unknown /proc/72807/root (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 72807 root txt unknown /proc/72807/exe (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 72807 root NOFD /proc/72807/fd (opendir: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 74547 root cwd unknown /proc/74547/cwd (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 74547 root rtd unknown /proc/74547/root (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 74547 root txt unknown /proc/74547/exe (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/3 74547 root NOFD /proc/74547/fd (opendir: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 79308 root cwd unknown /proc/79308/cwd (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 79308 root rtd unknown /proc/79308/root (readlink: Permission denied) 2024-03-12T04:17:28.573 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 79308 root txt unknown /proc/79308/exe (readlink: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/5 79308 root NOFD /proc/79308/fd (opendir: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 81223 root cwd unknown /proc/81223/cwd (readlink: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 81223 root rtd unknown /proc/81223/root (readlink: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 81223 root txt unknown /proc/81223/exe (readlink: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/6 81223 root NOFD /proc/81223/fd (opendir: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 81502 root cwd unknown /proc/81502/cwd (readlink: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 81502 root rtd unknown /proc/81502/root (readlink: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 81502 root txt unknown /proc/81502/exe (readlink: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/0 81502 root NOFD /proc/81502/fd (opendir: Permission denied) 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu cwd DIR 8,1 4096 1026 /home/ubuntu 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu rtd DIR 8,1 4096 2 / 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-12T04:17:28.574 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-12T04:17:28.575 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu 0r FIFO 0,13 0t0 224408 pipe 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu 1w FIFO 0,13 0t0 224409 pipe 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:bash 81587 ubuntu 2w FIFO 0,13 0t0 224410 pipe 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81617 root cwd unknown /proc/81617/cwd (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81617 root rtd unknown /proc/81617/root (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81617 root txt unknown /proc/81617/exe (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81617 root NOFD /proc/81617/fd (opendir: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81622 root cwd unknown /proc/81622/cwd (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81622 root rtd unknown /proc/81622/root (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81622 root txt unknown /proc/81622/exe (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:sudo 81622 root NOFD /proc/81622/fd (opendir: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:daemon-he 81625 root cwd unknown /proc/81625/cwd (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:daemon-he 81625 root rtd unknown /proc/81625/root (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:daemon-he 81625 root txt unknown /proc/81625/exe (readlink: Permission denied) 2024-03-12T04:17:28.576 INFO:tasks.workunit.client.0.smithi042.stdout:daemon-he 81625 root NOFD /proc/81625/fd (opendir: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 root cwd unknown /proc/81627/cwd (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 root rtd unknown /proc/81627/root (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 root txt unknown /proc/81627/exe (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 root NOFD /proc/81627/fd (opendir: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81646 log root cwd unknown /proc/81627/task/81646/cwd (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81646 log root rtd unknown /proc/81627/task/81646/root (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81646 log root txt unknown /proc/81627/task/81646/exe (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81646 log root NOFD /proc/81627/task/81646/fd (opendir: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81648 msgr-work root cwd unknown /proc/81627/task/81648/cwd (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81648 msgr-work root rtd unknown /proc/81627/task/81648/root (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81648 msgr-work root txt unknown /proc/81627/task/81648/exe (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81648 msgr-work root NOFD /proc/81627/task/81648/fd (opendir: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81649 msgr-work root cwd unknown /proc/81627/task/81649/cwd (readlink: Permission denied) 2024-03-12T04:17:28.577 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81649 msgr-work root rtd unknown /proc/81627/task/81649/root (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81649 msgr-work root txt unknown /proc/81627/task/81649/exe (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81649 msgr-work root NOFD /proc/81627/task/81649/fd (opendir: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81650 msgr-work root cwd unknown /proc/81627/task/81650/cwd (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81650 msgr-work root rtd unknown /proc/81627/task/81650/root (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81650 msgr-work root txt unknown /proc/81627/task/81650/exe (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81650 msgr-work root NOFD /proc/81627/task/81650/fd (opendir: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81654 service root cwd unknown /proc/81627/task/81654/cwd (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81654 service root rtd unknown /proc/81627/task/81654/root (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81654 service root txt unknown /proc/81627/task/81654/exe (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81654 service root NOFD /proc/81627/task/81654/fd (opendir: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81655 admin_soc root cwd unknown /proc/81627/task/81655/cwd (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81655 admin_soc root rtd unknown /proc/81627/task/81655/root (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81655 admin_soc root txt unknown /proc/81627/task/81655/exe (readlink: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81655 admin_soc root NOFD /proc/81627/task/81655/fd (opendir: Permission denied) 2024-03-12T04:17:28.578 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81656 signal_ha root cwd unknown /proc/81627/task/81656/cwd (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81656 signal_ha root rtd unknown /proc/81627/task/81656/root (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81656 signal_ha root txt unknown /proc/81627/task/81656/exe (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81656 signal_ha root NOFD /proc/81627/task/81656/fd (opendir: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81657 io_contex root cwd unknown /proc/81627/task/81657/cwd (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81657 io_contex root rtd unknown /proc/81627/task/81657/root (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81657 io_contex root txt unknown /proc/81627/task/81657/exe (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81657 io_contex root NOFD /proc/81627/task/81657/fd (opendir: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81658 io_contex root cwd unknown /proc/81627/task/81658/cwd (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81658 io_contex root rtd unknown /proc/81627/task/81658/root (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81658 io_contex root txt unknown /proc/81627/task/81658/exe (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81658 io_contex root NOFD /proc/81627/task/81658/fd (opendir: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81659 ceph_time root cwd unknown /proc/81627/task/81659/cwd (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81659 ceph_time root rtd unknown /proc/81627/task/81659/root (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81659 ceph_time root txt unknown /proc/81627/task/81659/exe (readlink: Permission denied) 2024-03-12T04:17:28.579 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81659 ceph_time root NOFD /proc/81627/task/81659/fd (opendir: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81660 fn_anonym root cwd unknown /proc/81627/task/81660/cwd (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81660 fn_anonym root rtd unknown /proc/81627/task/81660/root (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81660 fn_anonym root txt unknown /proc/81627/task/81660/exe (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81660 fn_anonym root NOFD /proc/81627/task/81660/fd (opendir: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81661 fn_anonym root cwd unknown /proc/81627/task/81661/cwd (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81661 fn_anonym root rtd unknown /proc/81627/task/81661/root (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81661 fn_anonym root txt unknown /proc/81627/task/81661/exe (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81661 fn_anonym root NOFD /proc/81627/task/81661/fd (opendir: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81662 fn_anonym root cwd unknown /proc/81627/task/81662/cwd (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81662 fn_anonym root rtd unknown /proc/81627/task/81662/root (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81662 fn_anonym root txt unknown /proc/81627/task/81662/exe (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81662 fn_anonym root NOFD /proc/81627/task/81662/fd (opendir: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81663 fn_anonym root cwd unknown /proc/81627/task/81663/cwd (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81663 fn_anonym root rtd unknown /proc/81627/task/81663/root (readlink: Permission denied) 2024-03-12T04:17:28.580 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81663 fn_anonym root txt unknown /proc/81627/task/81663/exe (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81663 fn_anonym root NOFD /proc/81627/task/81663/fd (opendir: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81664 fn_anonym root cwd unknown /proc/81627/task/81664/cwd (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81664 fn_anonym root rtd unknown /proc/81627/task/81664/root (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81664 fn_anonym root txt unknown /proc/81627/task/81664/exe (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81664 fn_anonym root NOFD /proc/81627/task/81664/fd (opendir: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81665 safe_time root cwd unknown /proc/81627/task/81665/cwd (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81665 safe_time root rtd unknown /proc/81627/task/81665/root (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81665 safe_time root txt unknown /proc/81627/task/81665/exe (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81665 safe_time root NOFD /proc/81627/task/81665/fd (opendir: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81666 fn_anonym root cwd unknown /proc/81627/task/81666/cwd (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81666 fn_anonym root rtd unknown /proc/81627/task/81666/root (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81666 fn_anonym root txt unknown /proc/81627/task/81666/exe (readlink: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81666 fn_anonym root NOFD /proc/81627/task/81666/fd (opendir: Permission denied) 2024-03-12T04:17:28.581 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81667 flusher root cwd unknown /proc/81627/task/81667/cwd (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81667 flusher root rtd unknown /proc/81627/task/81667/root (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81667 flusher root txt unknown /proc/81627/task/81667/exe (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81667 flusher root NOFD /proc/81627/task/81667/fd (opendir: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81668 ms_dispat root cwd unknown /proc/81627/task/81668/cwd (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81668 ms_dispat root rtd unknown /proc/81627/task/81668/root (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81668 ms_dispat root txt unknown /proc/81627/task/81668/exe (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81668 ms_dispat root NOFD /proc/81627/task/81668/fd (opendir: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81669 ms_local root cwd unknown /proc/81627/task/81669/cwd (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81669 ms_local root rtd unknown /proc/81627/task/81669/root (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81669 ms_local root txt unknown /proc/81627/task/81669/exe (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81669 ms_local root NOFD /proc/81627/task/81669/fd (opendir: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81670 safe_time root cwd unknown /proc/81627/task/81670/cwd (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81670 safe_time root rtd unknown /proc/81627/task/81670/root (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81670 safe_time root txt unknown /proc/81627/task/81670/exe (readlink: Permission denied) 2024-03-12T04:17:28.582 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81670 safe_time root NOFD /proc/81627/task/81670/fd (opendir: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81672 ceph-fuse root cwd unknown /proc/81627/task/81672/cwd (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81672 ceph-fuse root rtd unknown /proc/81627/task/81672/root (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81672 ceph-fuse root txt unknown /proc/81627/task/81672/exe (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81672 ceph-fuse root NOFD /proc/81627/task/81672/fd (opendir: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81689 ceph-fuse root cwd unknown /proc/81627/task/81689/cwd (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81689 ceph-fuse root rtd unknown /proc/81627/task/81689/root (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81689 ceph-fuse root txt unknown /proc/81627/task/81689/exe (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81689 ceph-fuse root NOFD /proc/81627/task/81689/fd (opendir: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81691 ceph-fuse root cwd unknown /proc/81627/task/81691/cwd (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81691 ceph-fuse root rtd unknown /proc/81627/task/81691/root (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81691 ceph-fuse root txt unknown /proc/81627/task/81691/exe (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 81691 ceph-fuse root NOFD /proc/81627/task/81691/fd (opendir: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86288 ceph-fuse root cwd unknown /proc/81627/task/86288/cwd (readlink: Permission denied) 2024-03-12T04:17:28.583 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86288 ceph-fuse root rtd unknown /proc/81627/task/86288/root (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86288 ceph-fuse root txt unknown /proc/81627/task/86288/exe (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86288 ceph-fuse root NOFD /proc/81627/task/86288/fd (opendir: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86296 ceph-fuse root cwd unknown /proc/81627/task/86296/cwd (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86296 ceph-fuse root rtd unknown /proc/81627/task/86296/root (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86296 ceph-fuse root txt unknown /proc/81627/task/86296/exe (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86296 ceph-fuse root NOFD /proc/81627/task/86296/fd (opendir: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86297 ceph-fuse root cwd unknown /proc/81627/task/86297/cwd (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86297 ceph-fuse root rtd unknown /proc/81627/task/86297/root (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86297 ceph-fuse root txt unknown /proc/81627/task/86297/exe (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86297 ceph-fuse root NOFD /proc/81627/task/86297/fd (opendir: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86403 ceph-fuse root cwd unknown /proc/81627/task/86403/cwd (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86403 ceph-fuse root rtd unknown /proc/81627/task/86403/root (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86403 ceph-fuse root txt unknown /proc/81627/task/86403/exe (readlink: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 86403 ceph-fuse root NOFD /proc/81627/task/86403/fd (opendir: Permission denied) 2024-03-12T04:17:28.584 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 87257 ceph-fuse root cwd unknown /proc/81627/task/87257/cwd (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 87257 ceph-fuse root rtd unknown /proc/81627/task/87257/root (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 87257 ceph-fuse root txt unknown /proc/81627/task/87257/exe (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:ceph-fuse 81627 87257 ceph-fuse root NOFD /proc/81627/task/87257/fd (opendir: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 82018 root cwd unknown /proc/82018/cwd (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 82018 root rtd unknown /proc/82018/root (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 82018 root txt unknown /proc/82018/exe (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 82018 root NOFD /proc/82018/fd (opendir: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83074 root cwd unknown /proc/83074/cwd (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83074 root rtd unknown /proc/83074/root (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83074 root txt unknown /proc/83074/exe (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83074 root NOFD /proc/83074/fd (opendir: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83134 root cwd unknown /proc/83134/cwd (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83134 root rtd unknown /proc/83134/root (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83134 root txt unknown /proc/83134/exe (readlink: Permission denied) 2024-03-12T04:17:28.585 INFO:tasks.workunit.client.0.smithi042.stdout:sshd 83134 root NOFD /proc/83134/fd (opendir: Permission denied) 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu cwd DIR 0,119 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu rtd DIR 8,1 4096 2 / 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu txt REG 8,1 40944 10098 /usr/bin/timeout 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-12T04:17:28.586 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu 0r FIFO 0,13 0t0 231419 pipe 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu 1w FIFO 0,13 0t0 231420 pipe 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:timeout 86268 ubuntu 2w FIFO 0,13 0t0 231421 pipe 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu cwd DIR 0,119 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu rtd DIR 8,1 4096 2 / 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-12T04:17:28.587 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu 0r FIFO 0,13 0t0 231419 pipe 2024-03-12T04:17:28.588 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu 1w FIFO 0,13 0t0 231420 pipe 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu 2w FIFO 0,13 0t0 231421 pipe 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:fsync-tes 86286 ubuntu 255r REG 8,1 207 530666 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 86298 root cwd unknown /proc/86298/cwd (readlink: Permission denied) 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 86298 root rtd unknown /proc/86298/root (readlink: Permission denied) 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 86298 root txt unknown /proc/86298/exe (readlink: Permission denied) 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:kworker/u 86298 root NOFD /proc/86298/fd (opendir: Permission denied) 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu cwd DIR 0,119 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu rtd DIR 8,1 4096 2 / 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 4487176 2249 /usr/lib64/libcrypto.so.3.0.7 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-12T04:17:28.589 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu 0r FIFO 0,13 0t0 231419 pipe 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu 1w FIFO 0,13 0t0 231420 pipe 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu 2w FIFO 0,13 0t0 231421 pipe 2024-03-12T04:17:28.590 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu 3r DIR 0,20 0 1 /proc 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu 4r DIR 0,20 0 237053 /proc/87556/fd 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu 5w FIFO 0,13 0t0 237058 pipe 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87556 ubuntu 6r FIFO 0,13 0t0 237059 pipe 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu cwd DIR 0,119 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu rtd DIR 8,1 4096 2 / 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 4487176 2249 /usr/lib64/libcrypto.so.3.0.7 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-12T04:17:28.591 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu 4r FIFO 0,13 0t0 237058 pipe 2024-03-12T04:17:28.592 INFO:tasks.workunit.client.0.smithi042.stdout:lsof 87557 ubuntu 7w FIFO 0,13 0t0 237059 pipe 2024-03-12T04:17:28.592 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-12T04:17:28.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-12T04:17:28.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:28 smithi042 ceph-mon[32260]: pgmap v71: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.03k op/s 2024-03-12T04:17:28.594 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:28 smithi136 ceph-mon[35571]: pgmap v71: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.03k op/s 2024-03-12T04:17:28.663 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-03-12T04:17:28.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-12T04:17:28.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:28 smithi119 ceph-mon[36119]: pgmap v71: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 5.03k op/s 2024-03-12T04:17:28.959 DEBUG:teuthology.parallel:result is None 2024-03-12T04:17:28.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-12T04:17:28.994 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-12T04:17:28.994 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-12T04:17:29.009 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-12T04:17:29.009 INFO:teuthology.task.exec:Executing custom commands... 2024-03-12T04:17:29.009 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T04:17:29.010 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-12T04:17:29.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-12T04:17:30.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:30 smithi042 ceph-mon[32260]: pgmap v72: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-03-12T04:17:30.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:30 smithi119 ceph-mon[36119]: pgmap v72: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-03-12T04:17:30.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:30 smithi136 ceph-mon[35571]: pgmap v72: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-03-12T04:17:32.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:32 smithi042 ceph-mon[32260]: pgmap v73: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-03-12T04:17:32.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:32 smithi119 ceph-mon[36119]: pgmap v73: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-03-12T04:17:32.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:32 smithi136 ceph-mon[35571]: pgmap v73: 97 pgs: 97 active+clean; 471 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.39k op/s 2024-03-12T04:17:34.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:34 smithi042 ceph-mon[32260]: pgmap v74: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 3.90k op/s 2024-03-12T04:17:34.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:34 smithi119 ceph-mon[36119]: pgmap v74: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 3.90k op/s 2024-03-12T04:17:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:34 smithi136 ceph-mon[35571]: pgmap v74: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 3.90k op/s 2024-03-12T04:17:36.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:36 smithi042 ceph-mon[32260]: pgmap v75: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 2.22k op/s 2024-03-12T04:17:36.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:36 smithi119 ceph-mon[36119]: pgmap v75: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 2.22k op/s 2024-03-12T04:17:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:36 smithi136 ceph-mon[35571]: pgmap v75: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 2.22k op/s 2024-03-12T04:17:38.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:38 smithi042 ceph-mon[32260]: pgmap v76: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.7 MiB/s wr, 2.23k op/s 2024-03-12T04:17:38.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:38 smithi119 ceph-mon[36119]: pgmap v76: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.7 MiB/s wr, 2.23k op/s 2024-03-12T04:17:38.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:38 smithi136 ceph-mon[35571]: pgmap v76: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.7 MiB/s wr, 2.23k op/s 2024-03-12T04:17:40.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:40 smithi042 ceph-mon[32260]: pgmap v77: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-03-12T04:17:40.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:40 smithi119 ceph-mon[36119]: pgmap v77: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-03-12T04:17:40.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:40 smithi136 ceph-mon[35571]: pgmap v77: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-03-12T04:17:42.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:42 smithi042 ceph-mon[32260]: pgmap v78: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-03-12T04:17:42.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:42 smithi119 ceph-mon[36119]: pgmap v78: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-03-12T04:17:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:42 smithi136 ceph-mon[35571]: pgmap v78: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-03-12T04:17:44.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:44 smithi042 ceph-mon[32260]: pgmap v79: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.1 MiB/s wr, 516 op/s 2024-03-12T04:17:44.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:44 smithi119 ceph-mon[36119]: pgmap v79: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.1 MiB/s wr, 516 op/s 2024-03-12T04:17:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:44 smithi136 ceph-mon[35571]: pgmap v79: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.1 MiB/s wr, 516 op/s 2024-03-12T04:17:45.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:17:45.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:17:45.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 -- 172.21.15.42:0/1259135198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f0101d10 msgr2=0x7f11f0105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 --2- 172.21.15.42:0/1259135198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f0101d10 0x7f11f0105d50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009930 tx=0x7f11e002bc30 comp rx=0 tx=0).stop 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 -- 172.21.15.42:0/1259135198 shutdown_connections 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 --2- 172.21.15.42:0/1259135198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f0106430 0x7f11f010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 --2- 172.21.15.42:0/1259135198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f0101d10 0x7f11f0105d50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 --2- 172.21.15.42:0/1259135198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f0101360 0x7f11f0101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 -- 172.21.15.42:0/1259135198 >> 172.21.15.42:0/1259135198 conn(0x7f11f00fcc20 msgr2=0x7f11f00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 -- 172.21.15.42:0/1259135198 shutdown_connections 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.493+0000 7f11f770f640 1 -- 172.21.15.42:0/1259135198 wait complete. 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 Processor -- start 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 -- start start 2024-03-12T04:17:45.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f0101360 0x7f11f0197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f0106430 0x7f11f0197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f019bfd0 0x7f11f019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f019c9c0 con 0x7f11f019bfd0 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f019cb30 con 0x7f11f0101360 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f770f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f019cca0 con 0x7f11f0106430 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f5cac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f019bfd0 0x7f11f019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f5cac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f019bfd0 0x7f11f019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54620/0 (socket says 172.21.15.42:54620) 2024-03-12T04:17:45.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f5cac640 1 -- 172.21.15.42:0/1466000560 learned_addr learned my addr 172.21.15.42:0/1466000560 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f54ab640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f0101360 0x7f11f0197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f4caa640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f0106430 0x7f11f0197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f5cac640 1 -- 172.21.15.42:0/1466000560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f0106430 msgr2=0x7f11f0197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f5cac640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f0106430 0x7f11f0197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.494+0000 7f11f5cac640 1 -- 172.21.15.42:0/1466000560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f0101360 msgr2=0x7f11f0197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11f5cac640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f0101360 0x7f11f0197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11f5cac640 1 -- 172.21.15.42:0/1466000560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0009560 con 0x7f11f019bfd0 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11f54ab640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f0101360 0x7f11f0197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11f5cac640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f019bfd0 0x7f11f019c480 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f11e800eaa0 tx=0x7f11e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11e67fc640 1 -- 172.21.15.42:0/1466000560 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800cdb0 con 0x7f11f019bfd0 2024-03-12T04:17:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f01a4280 con 0x7f11f019bfd0 2024-03-12T04:17:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11e67fc640 1 -- 172.21.15.42:0/1466000560 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11e8005020 con 0x7f11f019bfd0 2024-03-12T04:17:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11e67fc640 1 -- 172.21.15.42:0/1466000560 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80116b0 con 0x7f11f019bfd0 2024-03-12T04:17:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.495+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f01a46e0 con 0x7f11f019bfd0 2024-03-12T04:17:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.497+0000 7f11e67fc640 1 -- 172.21.15.42:0/1466000560 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f11e8005190 con 0x7f11f019bfd0 2024-03-12T04:17:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 Processor -- start 2024-03-12T04:17:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 -- start start 2024-03-12T04:17:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6f2b0 0x55bc28c6f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11e67fc640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f11dc0a1ba0 0x7f11dc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6fbd0 0x55bc28c78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28bde280 con 0x55bc28c6f2b0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28bdb7b0 con 0x55bc28c787b0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28bd6f90 con 0x55bc28c6fbd0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11f4caa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6f2b0 0x55bc28c6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11f54ab640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f11dc0a1ba0 0x7f11dc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11f4caa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6f2b0 0x55bc28c6f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54626/0 (socket says 172.21.15.42:54626) 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11f4caa640 1 -- 172.21.15.42:0/4194316168 learned_addr learned my addr 172.21.15.42:0/4194316168 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11e67fc640 1 -- 172.21.15.42:0/1466000560 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f11e80cda00 con 0x7f11f019bfd0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11f5cac640 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11f4caa640 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6fbd0 0x55bc28c78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.498+0000 7f11f4caa640 1 -- 172.21.15.42:0/4194316168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6fbd0 msgr2=0x55bc28c78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11f4caa640 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6fbd0 0x55bc28c78270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11f4caa640 1 -- 172.21.15.42:0/4194316168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 msgr2=0x55bc28c7aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11f4caa640 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11f4caa640 1 -- 172.21.15.42:0/4194316168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc28b95570 con 0x55bc28c6f2b0 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11f54ab640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f11dc0a1ba0 0x7f11dc0a4060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00abe0 tx=0x7f11ec00a5c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11f4caa640 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6f2b0 0x55bc28c6f690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f11e0005ec0 tx=0x7f11e0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20e8227c4cf8fee server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11db7fe640 1 -- 172.21.15.42:0/4194316168 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e002c970 con 0x55bc28c6f2b0 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11db7fe640 1 -- 172.21.15.42:0/4194316168 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11e00044d0 con 0x55bc28c6f2b0 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.499+0000 7f11db7fe640 1 -- 172.21.15.42:0/4194316168 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0003d40 con 0x55bc28c6f2b0 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 -- 172.21.15.42:0/4194316168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6f2b0 msgr2=0x55bc28c6f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6f2b0 0x55bc28c6f690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f11e0005ec0 tx=0x7f11e0005a20 comp rx=0 tx=0).stop 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 -- 172.21.15.42:0/4194316168 shutdown_connections 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6fbd0 0x55bc28c78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/4194316168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6f2b0 0x55bc28c6f690 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 -- 172.21.15.42:0/4194316168 >> 172.21.15.42:0/4194316168 conn(0x55bc28c69990 msgr2=0x55bc28c69d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 -- 172.21.15.42:0/4194316168 shutdown_connections 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 -- 172.21.15.42:0/4194316168 wait complete. 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 -- start start 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.500+0000 7f11fa3b6740 1 Processor -- start 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6f2b0 0x55bc28c75640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c787b0 0x55bc28c760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc288aa430 con 0x55bc28c6fbd0 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28866e00 con 0x55bc28c6f2b0 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28b5bfd0 con 0x55bc28c787b0 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 10 client.0 _ll_register_callbacks cb 0x55bc28bb1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54636/0 (socket says 172.21.15.42:54636) 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f54ab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6f2b0 0x55bc28c75640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 -- 172.21.15.42:0/3048980030 learned_addr learned my addr 172.21.15.42:0/3048980030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f5cac640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c787b0 0x55bc28c760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 -- 172.21.15.42:0/3048980030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c787b0 msgr2=0x55bc28c760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c787b0 0x55bc28c760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 -- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6f2b0 msgr2=0x55bc28c75640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6f2b0 0x55bc28c75640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 -- 172.21.15.42:0/3048980030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f010e810 con 0x55bc28c6fbd0 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f5cac640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c787b0 0x55bc28c760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f54ab640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6f2b0 0x55bc28c75640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11f4caa640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75b80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f11e0000c00 tx=0x7f11e0003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bd4cb154f7630b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11d8ff9640 1 -- 172.21.15.42:0/3048980030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0003ac0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11d8ff9640 1 -- 172.21.15.42:0/3048980030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f11e0003c20 con 0x55bc28c6fbd0 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.501+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc288d67e0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11d8ff9640 1 -- 172.21.15.42:0/3048980030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e0003de0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 10 client.16185 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc28bd5610 con 0x55bc28c6fbd0 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11d8ff9640 1 -- 172.21.15.42:0/3048980030 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bc288d67e0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11d8ff9640 1 -- 172.21.15.42:0/3048980030 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f11e0005360 con 0x55bc28c6fbd0 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 10 client.16185 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 20 client.16185 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28c88670 0x55bc28c8aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 4 client.16185 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc28913720 con 0x55bc28c88670 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.502+0000 7f11f54ab640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28c88670 0x55bc28c8aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.503+0000 7f11d8ff9640 10 client.16185 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:45.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.503+0000 7f11f54ab640 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28c88670 0x55bc28c8aa90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5947a6cd7c9bbc server_cookie=eedd037ea4b08537 in_seq=0 out_seq=0 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.503+0000 7f11d8ff9640 1 -- 172.21.15.42:0/3048980030 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc28913720 con 0x55bc28c88670 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.503+0000 7f11d8ff9640 10 client.16185 handle_command_reply: tid=0 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.503+0000 7f11fa3b6740 1 client.16185 shutdown 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 msgr2=0x55bc28c75b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75b80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f11e0000c00 tx=0x7f11e0003920 comp rx=0 tx=0).stop 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 shutdown_connections 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28c88670 0x55bc28c8aa90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c787b0 0x55bc28c760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75b80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11d8ff9640 0 client.16185 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3048980030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6f2b0 0x55bc28c75640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 >> 172.21.15.42:0/3048980030 conn(0x55bc28c69990 msgr2=0x55bc28c6b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 shutdown_connections 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3048980030 wait complete. 2024-03-12T04:17:45.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.504+0000 7f11fa3b6740 20 client.16185 trim_cache size 0 max 16384 2024-03-12T04:17:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 Processor -- start 2024-03-12T04:17:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 -- start start 2024-03-12T04:17:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c77d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c787b0 0x55bc28c74620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28c65120 con 0x55bc28c787b0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28c65320 con 0x55bc28c6fbd0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28c6ed20 con 0x55bc28c6f2b0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c787b0 0x55bc28c74620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c787b0 0x55bc28c74620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54652/0 (socket says 172.21.15.42:54652) 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 -- 172.21.15.42:0/3961434997 learned_addr learned my addr 172.21.15.42:0/3961434997 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f54ab640 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c77d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 -- 172.21.15.42:0/3961434997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 msgr2=0x55bc28c77d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c77d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 -- 172.21.15.42:0/3961434997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6fbd0 msgr2=0x55bc28c75ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f5cac640 1 -- 172.21.15.42:0/3961434997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc28b71ac0 con 0x55bc28c787b0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.519+0000 7f11f54ab640 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c77d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11f5cac640 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c787b0 0x55bc28c74620 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f11e809a380 tx=0x7f11e809a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6319fd3bf21a5398 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11daffd640 1 -- 172.21.15.42:0/3961434997 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80a2e20 con 0x55bc28c787b0 2024-03-12T04:17:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11daffd640 1 -- 172.21.15.42:0/3961434997 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11e8098720 con 0x55bc28c787b0 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11daffd640 1 -- 172.21.15.42:0/3961434997 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8096cf0 con 0x55bc28c787b0 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3961434997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c787b0 msgr2=0x55bc28c74620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c787b0 0x55bc28c74620 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f11e809a380 tx=0x7f11e809a580 comp rx=0 tx=0).stop 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3961434997 shutdown_connections 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c787b0 0x55bc28c74620 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c6fbd0 0x55bc28c75ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/3961434997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c77d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.520+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3961434997 >> 172.21.15.42:0/3961434997 conn(0x55bc28c69990 msgr2=0x55bc28c73f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3961434997 shutdown_connections 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 -- 172.21.15.42:0/3961434997 wait complete. 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 -- start start 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 Processor -- start 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c7f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28c7ea10 con 0x55bc28c6fbd0 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28c7eb80 con 0x55bc28c787b0 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc28cdeba0 con 0x55bc28c6f2b0 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11fa3b6740 10 client.0 _ll_register_callbacks cb 0x7f11e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11f4caa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11f4caa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c7ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54668/0 (socket says 172.21.15.42:54668) 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.521+0000 7f11f4caa640 1 -- 172.21.15.42:0/2139000061 learned_addr learned my addr 172.21.15.42:0/2139000061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f5cac640 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f54ab640 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c7f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f4caa640 1 -- 172.21.15.42:0/2139000061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 msgr2=0x55bc28c7f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f4caa640 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c7f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f4caa640 1 -- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 msgr2=0x55bc28c7e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f4caa640 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f4caa640 1 -- 172.21.15.42:0/2139000061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f01a46e0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11f4caa640 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c7ddd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11e004c4a0 tx=0x7f11e0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f81f59ffc5c9ba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11d8ff9640 1 -- 172.21.15.42:0/2139000061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e002c970 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11d8ff9640 1 -- 172.21.15.42:0/2139000061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f11e0000e40 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11e00044d0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11d8ff9640 1 -- 172.21.15.42:0/2139000061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e002cad0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11fa3b6740 10 client.16191 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc28b976a0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.522+0000 7f11d8ff9640 1 -- 172.21.15.42:0/2139000061 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11e00044d0 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11d8ff9640 1 -- 172.21.15.42:0/2139000061 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f11e004c970 con 0x55bc28c6fbd0 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11fa3b6740 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11fa3b6740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11fa3b6740 20 client.16191 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28cdfeb0 0x55bc28ce22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11fa3b6740 4 client.16191 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bc28962890 con 0x55bc28cdfeb0 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11f54ab640 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28cdfeb0 0x55bc28ce22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11f54ab640 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28cdfeb0 0x55bc28ce22d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c599094a6fd738 server_cookie=2f8d300ffe68c8b3 in_seq=0 out_seq=0 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.523+0000 7f11d8ff9640 10 client.16191 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11d8ff9640 1 -- 172.21.15.42:0/2139000061 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bc28962890 con 0x55bc28cdfeb0 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11d8ff9640 10 client.16191 handle_command_reply: tid=0 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 client.16191 shutdown 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 msgr2=0x55bc28c7ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c7ddd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11e004c4a0 tx=0x7f11e0005970 comp rx=0 tx=0).stop 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 shutdown_connections 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc28cdfeb0 0x55bc28ce22d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc28c787b0 0x55bc28c7e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc28c6fbd0 0x55bc28c7ddd0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11d8ff9640 0 client.16191 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 --2- 172.21.15.42:0/2139000061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc28c6f2b0 0x55bc28c7f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 >> 172.21.15.42:0/2139000061 conn(0x55bc28c69990 msgr2=0x55bc28c6ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 shutdown_connections 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 1 -- 172.21.15.42:0/2139000061 wait complete. 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11fa3b6740 20 client.16191 trim_cache size 0 max 16384 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:17:45.533 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:17:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f11dc0a1ba0 msgr2=0x7f11dc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11f770f640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f11dc0a1ba0 0x7f11dc0a4060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00abe0 tx=0x7f11ec00a5c0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f019bfd0 msgr2=0x7f11f019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.524+0000 7f11f770f640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f019bfd0 0x7f11f019c480 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f11e800eaa0 tx=0x7f11e800ef60 comp rx=0 tx=0).stop 2024-03-12T04:17:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 shutdown_connections 2024-03-12T04:17:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f11dc0a1ba0 0x7f11dc0a4060 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11f019bfd0 0x7f11f019c480 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f0106430 0x7f11f0197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 --2- 172.21.15.42:0/1466000560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f0101360 0x7f11f0197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 >> 172.21.15.42:0/1466000560 conn(0x7f11f00fcc20 msgr2=0x7f11f00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 shutdown_connections 2024-03-12T04:17:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.525+0000 7f11f770f640 1 -- 172.21.15.42:0/1466000560 wait complete. 2024-03-12T04:17:45.538 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:17:45.538 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 -- 172.21.15.42:0/3768942021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 msgr2=0x7f994010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 --2- 172.21.15.42:0/3768942021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f994010cf50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9938009cd0 tx=0x7f993802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 -- 172.21.15.42:0/3768942021 shutdown_connections 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 --2- 172.21.15.42:0/3768942021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f994010cf50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 --2- 172.21.15.42:0/3768942021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9940100c60 0x7f9940104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 --2- 172.21.15.42:0/3768942021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99401002b0 0x7f9940100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 -- 172.21.15.42:0/3768942021 >> 172.21.15.42:0/3768942021 conn(0x7f9940077120 msgr2=0x7f9940077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 -- 172.21.15.42:0/3768942021 shutdown_connections 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.638+0000 7f994718a640 1 -- 172.21.15.42:0/3768942021 wait complete. 2024-03-12T04:17:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 Processor -- start 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 -- start start 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99401002b0 0x7f99401965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f9940196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f994019ae80 0x7f994019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994019b870 con 0x7f9940105380 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994019b9e0 con 0x7f994019ae80 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f994718a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994019bb50 con 0x7f99401002b0 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f993ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f9940196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f993ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f9940196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54690/0 (socket says 172.21.15.42:54690) 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f993ffff640 1 -- 172.21.15.42:0/2243778585 learned_addr learned my addr 172.21.15.42:0/2243778585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f9944f26640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99401002b0 0x7f99401965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f9945727640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f994019ae80 0x7f994019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.639+0000 7f993ffff640 1 -- 172.21.15.42:0/2243778585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99401002b0 msgr2=0x7f99401965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993ffff640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99401002b0 0x7f99401965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993ffff640 1 -- 172.21.15.42:0/2243778585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f994019ae80 msgr2=0x7f994019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993ffff640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f994019ae80 0x7f994019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993ffff640 1 -- 172.21.15.42:0/2243778585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938009900 con 0x7f9940105380 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f9944f26640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99401002b0 0x7f99401965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993ffff640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f9940196b00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f992c005fd0 tx=0x7f992c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993dffb640 1 -- 172.21.15.42:0/2243778585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c009c30 con 0x7f9940105380 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993dffb640 1 -- 172.21.15.42:0/2243778585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f992c0044d0 con 0x7f9940105380 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401a3130 con 0x7f9940105380 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f993dffb640 1 -- 172.21.15.42:0/2243778585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c003e50 con 0x7f9940105380 2024-03-12T04:17:45.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.640+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401a3630 con 0x7f9940105380 2024-03-12T04:17:45.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.641+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99400683c0 con 0x7f9940105380 2024-03-12T04:17:45.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.642+0000 7f993dffb640 1 -- 172.21.15.42:0/2243778585 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f992c004000 con 0x7f9940105380 2024-03-12T04:17:45.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.642+0000 7f993dffb640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f99300a1c20 0x7f99300a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.643+0000 7f9944f26640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f99300a1c20 0x7f99300a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.643+0000 7f993dffb640 1 -- 172.21.15.42:0/2243778585 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f992c0cd410 con 0x7f9940105380 2024-03-12T04:17:45.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.643+0000 7f9944f26640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f99300a1c20 0x7f99300a40e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99340058c0 tx=0x7f9934005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.647+0000 7f993dffb640 1 -- 172.21.15.42:0/2243778585 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f992c096330 con 0x7f9940105380 2024-03-12T04:17:45.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.866+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9940197910 con 0x7f9940105380 2024-03-12T04:17:45.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.866+0000 7f993dffb640 1 -- 172.21.15.42:0/2243778585 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f992c0999f0 con 0x7f9940105380 2024-03-12T04:17:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:17:45.869 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:17:45.870 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:17:45.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f99300a1c20 msgr2=0x7f99300a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f99300a1c20 0x7f99300a40e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99340058c0 tx=0x7f9934005850 comp rx=0 tx=0).stop 2024-03-12T04:17:45.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 msgr2=0x7f9940196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f9940196b00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f992c005fd0 tx=0x7f992c00dde0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 shutdown_connections 2024-03-12T04:17:45.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f99300a1c20 0x7f99300a40e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f994019ae80 0x7f994019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9940105380 0x7f9940196b00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 --2- 172.21.15.42:0/2243778585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99401002b0 0x7f99401965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.868+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 >> 172.21.15.42:0/2243778585 conn(0x7f9940077120 msgr2=0x7f99400ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.869+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 shutdown_connections 2024-03-12T04:17:45.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.869+0000 7f994718a640 1 -- 172.21.15.42:0/2243778585 wait complete. 2024-03-12T04:17:45.882 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:45.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:17:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 -- 172.21.15.42:0/2567231450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b8106450 msgr2=0x7fa5b810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/2567231450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b8106450 0x7fa5b810e020 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009cd0 tx=0x7fa5b002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 -- 172.21.15.42:0/2567231450 shutdown_connections 2024-03-12T04:17:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/2567231450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b8106450 0x7fa5b810e020 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/2567231450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b8101d30 0x7fa5b8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/2567231450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5b8101380 0x7fa5b8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 -- 172.21.15.42:0/2567231450 >> 172.21.15.42:0/2567231450 conn(0x7fa5b80fcc20 msgr2=0x7fa5b80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 -- 172.21.15.42:0/2567231450 shutdown_connections 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.984+0000 7fa5bea6f640 1 -- 172.21.15.42:0/2567231450 wait complete. 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 Processor -- start 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 -- start start 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b8101380 0x7fa5b8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5b8106450 0x7fa5b8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b819bff0 0x7fa5b819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b819c9e0 con 0x7fa5b819bff0 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b819cb50 con 0x7fa5b8106450 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bea6f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b819ccc0 con 0x7fa5b8101380 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b819bff0 0x7fa5b819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b819bff0 0x7fa5b819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54708/0 (socket says 172.21.15.42:54708) 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 -- 172.21.15.42:0/1272153213 learned_addr learned my addr 172.21.15.42:0/1272153213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b8101380 0x7fa5b8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5b8106450 0x7fa5b8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 -- 172.21.15.42:0/1272153213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b8101380 msgr2=0x7fa5b8197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b8101380 0x7fa5b8197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 -- 172.21.15.42:0/1272153213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5b8106450 msgr2=0x7fa5b8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5b8106450 0x7fa5b8197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5bd00c640 1 -- 172.21.15.42:0/1272153213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0009900 con 0x7fa5b819bff0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.985+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5b8106450 0x7fa5b8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.986+0000 7fa5bd00c640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b819bff0 0x7fa5b819c4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0037040 tx=0x7fa5b0035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.986+0000 7fa5b5ffb640 1 -- 172.21.15.42:0/1272153213 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00329e0 con 0x7fa5b819bff0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.986+0000 7fa5b5ffb640 1 -- 172.21.15.42:0/1272153213 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5b0003e20 con 0x7fa5b819bff0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.986+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b81a42a0 con 0x7fa5b819bff0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.986+0000 7fa5b5ffb640 1 -- 172.21.15.42:0/1272153213 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b003f440 con 0x7fa5b819bff0 2024-03-12T04:17:45.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.986+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b81a46c0 con 0x7fa5b819bff0 2024-03-12T04:17:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5b5ffb640 1 -- 172.21.15.42:0/1272153213 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa5b003b040 con 0x7fa5b819bff0 2024-03-12T04:17:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 Processor -- start 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 -- start start 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b72360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b81f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26b2d960 con 0x557b26b71f80 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26ae1310 con 0x557b26b728a0 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5c1643740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26ade840 con 0x557b26b7fb60 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5b5ffb640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa5a80a1ba0 0x7fa5a80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b72360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b72360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54716/0 (socket says 172.21.15.42:54716) 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5b7fff640 1 -- 172.21.15.42:0/3559554395 learned_addr learned my addr 172.21.15.42:0/3559554395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.989+0000 7fa5bd00c640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa5a80a1ba0 0x7fa5a80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5b5ffb640 1 -- 172.21.15.42:0/1272153213 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa5b00f03d0 con 0x7fa5b819bff0 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5b7fff640 1 -- 172.21.15.42:0/3559554395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 msgr2=0x557b26b81f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b81f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5b7fff640 1 -- 172.21.15.42:0/3559554395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 msgr2=0x557b26b7f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5b7fff640 1 -- 172.21.15.42:0/3559554395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b26a9c060 con 0x557b26b71f80 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b72360 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4005940 tx=0x7fa5a4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90a7193552720ba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5bd00c640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa5a80a1ba0 0x7fa5a80a4060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00edd30 tx=0x7fa5b0032000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3559554395 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4005220 con 0x557b26b71f80 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3559554395 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5a4004c50 con 0x557b26b71f80 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.990+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3559554395 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4012800 con 0x557b26b71f80 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 -- 172.21.15.42:0/3559554395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 msgr2=0x557b26b72360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b72360 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4005940 tx=0x7fa5a4010740 comp rx=0 tx=0).stop 2024-03-12T04:17:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 -- 172.21.15.42:0/3559554395 shutdown_connections 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b81f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3559554395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b72360 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 -- 172.21.15.42:0/3559554395 >> 172.21.15.42:0/3559554395 conn(0x557b26b6c980 msgr2=0x557b26b6cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 -- 172.21.15.42:0/3559554395 shutdown_connections 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 -- 172.21.15.42:0/3559554395 wait complete. 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 -- start start 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.991+0000 7fa5c1643740 1 Processor -- start 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b7cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b7d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26ada020 con 0x557b26b71f80 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b267ad430 con 0x557b26b728a0 2024-03-12T04:17:46.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26769e00 con 0x557b26b7fb60 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 10 client.0 _ll_register_callbacks cb 0x557b26a81cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b7cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b7cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54726/0 (socket says 172.21.15.42:54726) 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 -- 172.21.15.42:0/528048953 learned_addr learned my addr 172.21.15.42:0/528048953 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b7d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 -- 172.21.15.42:0/528048953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 msgr2=0x557b26b7d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b7d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 -- 172.21.15.42:0/528048953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 msgr2=0x557b26b7d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 -- 172.21.15.42:0/528048953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a400c940 con 0x557b26b71f80 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b7cb20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40124c0 tx=0x7fa5a400c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2348bb416ad90b47 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa597fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa597fff640 1 -- 172.21.15.42:0/528048953 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4012970 con 0x557b26b71f80 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa597fff640 1 -- 172.21.15.42:0/528048953 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa5a4012ad0 con 0x557b26b71f80 2024-03-12T04:17:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b267d97e0 con 0x557b26b71f80 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa597fff640 1 -- 172.21.15.42:0/528048953 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a4012c90 con 0x557b26b71f80 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 10 client.16209 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.992+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557b26ad86a0 con 0x557b26b71f80 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa597fff640 1 -- 172.21.15.42:0/528048953 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557b267d97e0 con 0x557b26b71f80 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa597fff640 1 -- 172.21.15.42:0/528048953 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5a401a400 con 0x557b26b71f80 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5c1643740 10 client.16209 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5c1643740 10 client.16209 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5c1643740 20 client.16209 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5c1643740 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26b8feb0 0x557b26b922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5c1643740 4 client.16209 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b26816720 con 0x557b26b8feb0 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26b8feb0 0x557b26b922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26b8feb0 0x557b26b922d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b45217b021e5ea6 server_cookie=e60fb9b60c978ebb in_seq=0 out_seq=0 2024-03-12T04:17:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.993+0000 7fa597fff640 10 client.16209 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa597fff640 1 -- 172.21.15.42:0/528048953 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b26816720 con 0x557b26b8feb0 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa597fff640 10 client.16209 handle_command_reply: tid=0 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 client.16209 shutdown 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 msgr2=0x557b26b7cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b7cb20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40124c0 tx=0x7fa5a400c540 comp rx=0 tx=0).stop 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 shutdown_connections 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b7d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b728a0 0x557b26b7d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26b8feb0 0x557b26b922d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 --2- 172.21.15.42:0/528048953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b71f80 0x557b26b7cb20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa597fff640 0 client.16209 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 >> 172.21.15.42:0/528048953 conn(0x557b26b6c980 msgr2=0x557b26b6e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 shutdown_connections 2024-03-12T04:17:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 1 -- 172.21.15.42:0/528048953 wait complete. 2024-03-12T04:17:46.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:45.994+0000 7fa5c1643740 20 client.16209 trim_cache size 0 max 16384 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 Processor -- start 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 -- start start 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b71f80 0x557b26b7f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b79d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26a5f130 con 0x557b26b728a0 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26b685c0 con 0x557b26b71f80 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5c1643740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26b8fc80 con 0x557b26b7fb60 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b79d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b79d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54740/0 (socket says 172.21.15.42:54740) 2024-03-12T04:17:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 -- 172.21.15.42:0/243295414 learned_addr learned my addr 172.21.15.42:0/243295414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 -- 172.21.15.42:0/243295414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 msgr2=0x557b26b7a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/243295414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 -- 172.21.15.42:0/243295414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b71f80 msgr2=0x557b26b7f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/243295414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b71f80 0x557b26b7f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 -- 172.21.15.42:0/243295414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b26a74c20 con 0x557b26b728a0 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/243295414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b79d80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x557b26b7c380 tx=0x7fa5ac00a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ed15cf96b4d2dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.006+0000 7fa5a27fc640 1 -- 172.21.15.42:0/243295414 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac017040 con 0x557b26b728a0 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5a27fc640 1 -- 172.21.15.42:0/243295414 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5ac0044d0 con 0x557b26b728a0 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5a27fc640 1 -- 172.21.15.42:0/243295414 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac019510 con 0x557b26b728a0 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 -- 172.21.15.42:0/243295414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 msgr2=0x557b26b79d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 --2- 172.21.15.42:0/243295414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b79d80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x557b26b7c380 tx=0x7fa5ac00a4b0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 -- 172.21.15.42:0/243295414 shutdown_connections 2024-03-12T04:17:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 --2- 172.21.15.42:0/243295414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b7fb60 0x557b26b7a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 --2- 172.21.15.42:0/243295414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b79d80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 --2- 172.21.15.42:0/243295414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b71f80 0x557b26b7f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 -- 172.21.15.42:0/243295414 >> 172.21.15.42:0/243295414 conn(0x557b26b6c980 msgr2=0x557b26b77da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 -- 172.21.15.42:0/243295414 shutdown_connections 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 -- 172.21.15.42:0/243295414 wait complete. 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 -- start start 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.007+0000 7fa5c1643740 1 Processor -- start 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b71f80 0x557b26b860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b86620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5c1643740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b7fb60 0x557b26bdf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5c1643740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26bdfa50 con 0x557b26b728a0 2024-03-12T04:17:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5c1643740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26bdfbc0 con 0x557b26b7fb60 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5c1643740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b26bdfd80 con 0x557b26b71f80 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5c1643740 10 client.0 _ll_register_callbacks cb 0x7fa5a400c440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b86620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b86620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54748/0 (socket says 172.21.15.42:54748) 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 -- 172.21.15.42:0/3996842479 learned_addr learned my addr 172.21.15.42:0/3996842479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b71f80 0x557b26b860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 -- 172.21.15.42:0/3996842479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b71f80 msgr2=0x557b26b860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b71f80 0x557b26b860e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 -- 172.21.15.42:0/3996842479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b7fb60 msgr2=0x557b26bdf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b7fb60 0x557b26bdf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 -- 172.21.15.42:0/3996842479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac00a790 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa5bc80b640 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b86620 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0057a0 tx=0x7fa5ac013340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39c83718283d8bc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.008+0000 7fa597fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa597fff640 1 -- 172.21.15.42:0/3996842479 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac013b30 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa597fff640 1 -- 172.21.15.42:0/3996842479 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa5ac013c90 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5a4004c50 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa597fff640 1 -- 172.21.15.42:0/3996842479 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5ac005030 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 10 client.16215 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557b26a9a8c0 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa597fff640 1 -- 172.21.15.42:0/3996842479 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5a4004c50 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa597fff640 1 -- 172.21.15.42:0/3996842479 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5ac004730 con 0x557b26b728a0 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 10 client.16215 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 10 client.16215 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:17:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 20 client.16215 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26be8800 0x557b26beac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 4 client.16215 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557b26865890 con 0x557b26be8800 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.009+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26be8800 0x557b26beac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa597fff640 10 client.16215 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5b7fff640 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26be8800 0x557b26beac20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd4e40ff43e457ba server_cookie=e7369035c194cf44 in_seq=0 out_seq=0 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa597fff640 1 -- 172.21.15.42:0/3996842479 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557b26865890 con 0x557b26be8800 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa597fff640 10 client.16215 handle_command_reply: tid=0 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 client.16215 shutdown 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 msgr2=0x557b26b86620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b86620 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0057a0 tx=0x7fa5ac013340 comp rx=0 tx=0).stop 2024-03-12T04:17:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 shutdown_connections 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b26b7fb60 0x557b26bdf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557b26be8800 0x557b26beac20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557b26b728a0 0x557b26b86620 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 --2- 172.21.15.42:0/3996842479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b26b71f80 0x557b26b860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa597fff640 0 client.16215 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 >> 172.21.15.42:0/3996842479 conn(0x557b26b6c980 msgr2=0x557b26b7cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 shutdown_connections 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 1 -- 172.21.15.42:0/3996842479 wait complete. 2024-03-12T04:17:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.010+0000 7fa5c1643740 20 client.16215 trim_cache size 0 max 16384 2024-03-12T04:17:46.022 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa5a80a1ba0 msgr2=0x7fa5a80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa5a80a1ba0 0x7fa5a80a4060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00edd30 tx=0x7fa5b0032000 comp rx=0 tx=0).stop 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b819bff0 msgr2=0x7fa5b819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b819bff0 0x7fa5b819c4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0037040 tx=0x7fa5b0035d10 comp rx=0 tx=0).stop 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 shutdown_connections 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa5a80a1ba0 0x7fa5a80a4060 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b819bff0 0x7fa5b819c4a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5b8106450 0x7fa5b8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 --2- 172.21.15.42:0/1272153213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b8101380 0x7fa5b8197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 >> 172.21.15.42:0/1272153213 conn(0x7fa5b80fcc20 msgr2=0x7fa5b80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 shutdown_connections 2024-03-12T04:17:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.011+0000 7fa5bea6f640 1 -- 172.21.15.42:0/1272153213 wait complete. 2024-03-12T04:17:46.025 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:46.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:17:46.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 -- 172.21.15.42:0/2761404663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc100c30 msgr2=0x7f5adc104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/2761404663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc100c30 0x7f5adc104c70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009930 tx=0x7f5acc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 -- 172.21.15.42:0/2761404663 shutdown_connections 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/2761404663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc105350 0x7f5adc10cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/2761404663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc100c30 0x7f5adc104c70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/2761404663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5adc100280 0x7f5adc100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 -- 172.21.15.42:0/2761404663 >> 172.21.15.42:0/2761404663 conn(0x7f5adc077150 msgr2=0x7f5adc077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 -- 172.21.15.42:0/2761404663 shutdown_connections 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.127+0000 7f5ae2f02640 1 -- 172.21.15.42:0/2761404663 wait complete. 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 Processor -- start 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 -- start start 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc100280 0x7f5adc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5adc105350 0x7f5adc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc1975c0 0x7f5adc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc197fb0 con 0x7f5adc1975c0 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc198120 con 0x7f5adc100280 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae2f02640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc1982e0 con 0x7f5adc105350 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc1975c0 0x7f5adc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc1975c0 0x7f5adc197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54766/0 (socket says 172.21.15.42:54766) 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 -- 172.21.15.42:0/1638320251 learned_addr learned my addr 172.21.15.42:0/1638320251 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5adc105350 0x7f5adc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc100280 0x7f5adc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 -- 172.21.15.42:0/1638320251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5adc105350 msgr2=0x7f5adc19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5adc105350 0x7f5adc19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 -- 172.21.15.42:0/1638320251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc100280 msgr2=0x7f5adc19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc100280 0x7f5adc19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.128+0000 7f5ae149f640 1 -- 172.21.15.42:0/1638320251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc009560 con 0x7f5adc1975c0 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.129+0000 7f5ae149f640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc1975c0 0x7f5adc197a70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009fd0 tx=0x7f5ad400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.129+0000 7f5ad1ffb640 1 -- 172.21.15.42:0/1638320251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400bea0 con 0x7f5adc1975c0 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.129+0000 7f5ad1ffb640 1 -- 172.21.15.42:0/1638320251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ad4009940 con 0x7f5adc1975c0 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.129+0000 7f5ad1ffb640 1 -- 172.21.15.42:0/1638320251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40106c0 con 0x7f5adc1975c0 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.129+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc1a42d0 con 0x7f5adc1975c0 2024-03-12T04:17:46.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.129+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc1a4760 con 0x7f5adc1975c0 2024-03-12T04:17:46.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.131+0000 7f5ad1ffb640 1 -- 172.21.15.42:0/1638320251 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f5ad4003970 con 0x7f5adc1975c0 2024-03-12T04:17:46.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 Processor -- start 2024-03-12T04:17:46.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 -- start start 2024-03-12T04:17:46.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ad1ffb640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5ac40a1ba0 0x7f5ac40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f0880d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880db70 0x562f08816210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f08818b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f087c8960 con 0x562f0880d250 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0877c310 con 0x562f0880db70 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f08779840 con 0x562f08816750 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5ac40a1ba0 0x7f5ac40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ad1ffb640 1 -- 172.21.15.42:0/1638320251 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5ad40cd3f0 con 0x7f5adc1975c0 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f0880d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f0880d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54776/0 (socket says 172.21.15.42:54776) 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/2028026804 learned_addr learned my addr 172.21.15.42:0/2028026804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880db70 0x562f08816210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/2028026804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 msgr2=0x562f08818b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f08818b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/2028026804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880db70 msgr2=0x562f08816210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880db70 0x562f08816210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.132+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/2028026804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f08737060 con 0x562f0880d250 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5ac40a1ba0 0x7f5ac40a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc004b30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f0880d630 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b760 tx=0x7f5ad800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1be176c270798e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ac2ffd640 1 -- 172.21.15.42:0/2028026804 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8003dd0 con 0x562f0880d250 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ac2ffd640 1 -- 172.21.15.42:0/2028026804 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ad800be70 con 0x562f0880d250 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ac2ffd640 1 -- 172.21.15.42:0/2028026804 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8004e80 con 0x562f0880d250 2024-03-12T04:17:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2028026804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 msgr2=0x562f0880d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f0880d630 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b760 tx=0x7f5ad800bc20 comp rx=0 tx=0).stop 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2028026804 shutdown_connections 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f08818b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880db70 0x562f08816210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2028026804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f0880d630 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.133+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2028026804 >> 172.21.15.42:0/2028026804 conn(0x562f08807980 msgr2=0x562f08807d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2028026804 shutdown_connections 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2028026804 wait complete. 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 -- start start 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 Processor -- start 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f088137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f0880db70 0x562f08813d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f08816750 0x562f08814270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f08775020 con 0x562f0880d250 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f08448430 con 0x562f08816750 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f08404e00 con 0x562f0880db70 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae5bb1740 10 client.0 _ll_register_callbacks cb 0x562f0871ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae0c9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f088137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae0c9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f088137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54780/0 (socket says 172.21.15.42:54780) 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.134+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/215887130 learned_addr learned my addr 172.21.15.42:0/215887130 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae149f640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f08816750 0x562f08814270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f0880db70 0x562f08813d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f0880db70 msgr2=0x562f08813d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f0880db70 0x562f08813d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/215887130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f08816750 msgr2=0x562f08814270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f08816750 0x562f08814270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae0c9e640 1 -- 172.21.15.42:0/215887130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad8004370 con 0x562f0880d250 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae149f640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f08816750 0x562f08814270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f088137f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8006010 tx=0x7f5ad8003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4527fe897505cfc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5abbfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5abbfff640 1 -- 172.21.15.42:0/215887130 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8005070 con 0x562f0880d250 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5abbfff640 1 -- 172.21.15.42:0/215887130 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5ad80051d0 con 0x562f0880d250 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f084747e0 con 0x562f0880d250 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5abbfff640 1 -- 172.21.15.42:0/215887130 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad8005390 con 0x562f0880d250 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae5bb1740 10 client.16227 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562f087736a0 con 0x562f0880d250 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.135+0000 7f5abbfff640 1 -- 172.21.15.42:0/215887130 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x562f084747e0 con 0x562f0880d250 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5abbfff640 1 -- 172.21.15.42:0/215887130 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ad8019420 con 0x562f0880d250 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ae5bb1740 10 client.16227 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ae5bb1740 10 client.16227 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ae5bb1740 20 client.16227 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08828f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ae5bb1740 4 client.16227 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f084b1720 con 0x562f08826b10 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08828f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5abbfff640 10 client.16227 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.136+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08828f00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f76a5f4af386bc9a server_cookie=27c54fddf6e85c44 in_seq=0 out_seq=0 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5abbfff640 1 -- 172.21.15.42:0/215887130 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f084b1720 con 0x562f08826b10 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5abbfff640 10 client.16227 handle_command_reply: tid=0 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 client.16227 shutdown 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 msgr2=0x562f088137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f088137f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8006010 tx=0x7f5ad8003de0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 shutdown_connections 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08828f00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f08816750 0x562f08814270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f0880db70 0x562f08813d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5abbfff640 0 client.16227 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/215887130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880d250 0x562f088137f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 >> 172.21.15.42:0/215887130 conn(0x562f08807980 msgr2=0x562f088096d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 shutdown_connections 2024-03-12T04:17:46.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/215887130 wait complete. 2024-03-12T04:17:46.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.137+0000 7f5ae5bb1740 20 client.16227 trim_cache size 0 max 16384 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 Processor -- start 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 -- start start 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f08815f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f08814090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f08810a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f086fa130 con 0x562f0880db70 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f088236c0 con 0x562f0880d250 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0882e0f0 con 0x562f08816750 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f08814090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f08814090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54782/0 (socket says 172.21.15.42:54782) 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 -- 172.21.15.42:0/3577290514 learned_addr learned my addr 172.21.15.42:0/3577290514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f08815f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 -- 172.21.15.42:0/3577290514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 msgr2=0x562f08810a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f08810a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 -- 172.21.15.42:0/3577290514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 msgr2=0x562f08815f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f08815f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.149+0000 7f5ad3fff640 1 -- 172.21.15.42:0/3577290514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f0870fc20 con 0x562f0880db70 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f08814090 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005ab0 tx=0x7f5acc046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6054889b1a5878dc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ac27fc640 1 -- 172.21.15.42:0/3577290514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc053ea0 con 0x562f0880db70 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ac27fc640 1 -- 172.21.15.42:0/3577290514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5acc02de90 con 0x562f0880db70 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ac27fc640 1 -- 172.21.15.42:0/3577290514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc003b50 con 0x562f0880db70 2024-03-12T04:17:46.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/3577290514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 msgr2=0x562f08814090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f08814090 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005ab0 tx=0x7f5acc046000 comp rx=0 tx=0).stop 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/3577290514 shutdown_connections 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f08810a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f08814090 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/3577290514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f08815f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/3577290514 >> 172.21.15.42:0/3577290514 conn(0x562f08807980 msgr2=0x562f088120c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/3577290514 shutdown_connections 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.150+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/3577290514 wait complete. 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 -- start start 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 Processor -- start 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f0881db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f0881bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f0881c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0881ccf0 con 0x562f0880db70 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0887cff0 con 0x562f0880d250 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0887d130 con 0x562f08816750 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae5bb1740 10 client.0 _ll_register_callbacks cb 0x7f5ad8004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f0881bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f0881bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54784/0 (socket says 172.21.15.42:54784) 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ad3fff640 1 -- 172.21.15.42:0/2371997651 learned_addr learned my addr 172.21.15.42:0/2371997651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f0881db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.151+0000 7f5ae149f640 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f0881c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ad3fff640 1 -- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 msgr2=0x562f0881c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f0881c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ad3fff640 1 -- 172.21.15.42:0/2371997651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 msgr2=0x562f0881db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f0881db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ad3fff640 1 -- 172.21.15.42:0/2371997651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc049050 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ad3fff640 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f0881bee0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5acc02c890 tx=0x7f5acc02ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129808b12c68c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5abbfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5abbfff640 1 -- 172.21.15.42:0/2371997651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc003ad0 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5abbfff640 1 -- 172.21.15.42:0/2371997651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5acc003c30 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ad800be70 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5abbfff640 1 -- 172.21.15.42:0/2371997651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5acc0036b0 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 10 client.16233 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562f087358c0 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5abbfff640 1 -- 172.21.15.42:0/2371997651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ad800be70 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5abbfff640 1 -- 172.21.15.42:0/2371997651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5acc03c070 con 0x562f0880db70 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 10 client.16233 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 10 client.16233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 20 client.16233 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08880610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 4 client.16233 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.152+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562f08500890 con 0x562f08826b10 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08880610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae0c9e640 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08880610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18f1214519ba482e server_cookie=9e6095b6348a1841 in_seq=0 out_seq=0 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5abbfff640 10 client.16233 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5abbfff640 1 -- 172.21.15.42:0/2371997651 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562f08500890 con 0x562f08826b10 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5abbfff640 10 client.16233 handle_command_reply: tid=0 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 client.16233 shutdown 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 msgr2=0x562f0881bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f0881bee0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5acc02c890 tx=0x7f5acc02ccc0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 shutdown_connections 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x562f08826b10 0x562f08880610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f08816750 0x562f0881c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f0880db70 0x562f0881bee0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5abbfff640 0 client.16233 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 --2- 172.21.15.42:0/2371997651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0880d250 0x562f0881db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 >> 172.21.15.42:0/2371997651 conn(0x562f08807980 msgr2=0x562f088062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 shutdown_connections 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.153+0000 7f5ae5bb1740 1 -- 172.21.15.42:0/2371997651 wait complete. 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae5bb1740 20 client.16233 trim_cache size 0 max 16384 2024-03-12T04:17:46.164 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5ac40a1ba0 msgr2=0x7f5ac40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5ac40a1ba0 0x7f5ac40a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc004b30 comp rx=0 tx=0).stop 2024-03-12T04:17:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc1975c0 msgr2=0x7f5adc197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc1975c0 0x7f5adc197a70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009fd0 tx=0x7f5ad400b260 comp rx=0 tx=0).stop 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 shutdown_connections 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5ac40a1ba0 0x7f5ac40a4060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5adc1975c0 0x7f5adc197a70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5adc105350 0x7f5adc19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 --2- 172.21.15.42:0/1638320251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc100280 0x7f5adc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 >> 172.21.15.42:0/1638320251 conn(0x7f5adc077150 msgr2=0x7f5adc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 shutdown_connections 2024-03-12T04:17:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.154+0000 7f5ae2f02640 1 -- 172.21.15.42:0/1638320251 wait complete. 2024-03-12T04:17:46.167 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:46.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:17:46.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 -- 172.21.15.42:0/3647816958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84740ffa40 msgr2=0x7f84740ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 --2- 172.21.15.42:0/3647816958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84740ffa40 0x7f84740ffec0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f845c009930 tx=0x7f845c02bcd0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 -- 172.21.15.42:0/3647816958 shutdown_connections 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 --2- 172.21.15.42:0/3647816958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8474100400 0x7f847410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 --2- 172.21.15.42:0/3647816958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84740ffa40 0x7f84740ffec0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 --2- 172.21.15.42:0/3647816958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8474107380 0x7f8474107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 -- 172.21.15.42:0/3647816958 >> 172.21.15.42:0/3647816958 conn(0x7f84740753c0 msgr2=0x7f84740757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 -- 172.21.15.42:0/3647816958 shutdown_connections 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.270+0000 7f8479464640 1 -- 172.21.15.42:0/3647816958 wait complete. 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 Processor -- start 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 -- start start 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84740ffa40 0x7f8474197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8474100400 0x7f8474197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8474107380 0x7f847419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847419c5f0 con 0x7f8474107380 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847419c760 con 0x7f8474100400 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8479464640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847419c920 con 0x7f84740ffa40 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8474107380 0x7f847419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8474107380 0x7f847419bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54808/0 (socket says 172.21.15.42:54808) 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 -- 172.21.15.42:0/3845782706 learned_addr learned my addr 172.21.15.42:0/3845782706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f846bfff640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8474100400 0x7f8474197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 -- 172.21.15.42:0/3845782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84740ffa40 msgr2=0x7f8474197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8472ffd640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84740ffa40 0x7f8474197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84740ffa40 0x7f8474197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 -- 172.21.15.42:0/3845782706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8474100400 msgr2=0x7f8474197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8474100400 0x7f8474197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f84737fe640 1 -- 172.21.15.42:0/3845782706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c009560 con 0x7f8474107380 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f846bfff640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8474100400 0x7f8474197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.271+0000 7f8472ffd640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84740ffa40 0x7f8474197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.272+0000 7f84737fe640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8474107380 0x7f847419bff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f846c00eaa0 tx=0x7f846c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.272+0000 7f8470ff9640 1 -- 172.21.15.42:0/3845782706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c00cdb0 con 0x7f8474107380 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.272+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847419cc00 con 0x7f8474107380 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.272+0000 7f8470ff9640 1 -- 172.21.15.42:0/3845782706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f846c005020 con 0x7f8474107380 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.272+0000 7f8470ff9640 1 -- 172.21.15.42:0/3845782706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0116d0 con 0x7f8474107380 2024-03-12T04:17:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.272+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84741a4480 con 0x7f8474107380 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.274+0000 7f8470ff9640 1 -- 172.21.15.42:0/3845782706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f846c005190 con 0x7f8474107380 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.274+0000 7f847c033740 1 Processor -- start 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.274+0000 7f8470ff9640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f84600a1ba0 0x7f84600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.274+0000 7f847c033740 1 -- start start 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.274+0000 7f8470ff9640 1 -- 172.21.15.42:0/3845782706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f846c0cd9b0 con 0x7f8474107380 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f846bfff640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f84600a1ba0 0x7f84600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f847c033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7250 0x555d921d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f847c033740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7b70 0x555d921e0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f847c033740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f847c033740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92192960 con 0x555d921e0750 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f847c033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92146310 con 0x555d921d7250 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f847c033740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92143840 con 0x555d921d7b70 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921e2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54812/0 (socket says 172.21.15.42:54812) 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 -- 172.21.15.42:0/3581473841 learned_addr learned my addr 172.21.15.42:0/3581473841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f846bfff640 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7b70 0x555d921e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f8472ffd640 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7250 0x555d921d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 -- 172.21.15.42:0/3581473841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7b70 msgr2=0x555d921e0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7b70 0x555d921e0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 -- 172.21.15.42:0/3581473841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7250 msgr2=0x555d921d7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7250 0x555d921d7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 -- 172.21.15.42:0/3581473841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d92101060 con 0x555d921e0750 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f846bfff640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f84600a1ba0 0x7f84600a4060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c0047c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84737fe640 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921e2b40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f846c0a28a0 tx=0x7f846c09a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6417668c683ef761 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.275+0000 7f84697fa640 1 -- 172.21.15.42:0/3581473841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c09e780 con 0x555d921e0750 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f84697fa640 1 -- 172.21.15.42:0/3581473841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f846c0a2dd0 con 0x555d921e0750 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f84697fa640 1 -- 172.21.15.42:0/3581473841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c096ca0 con 0x555d921e0750 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 -- 172.21.15.42:0/3581473841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 msgr2=0x555d921e2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921e2b40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f846c0a28a0 tx=0x7f846c09a260 comp rx=0 tx=0).stop 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 -- 172.21.15.42:0/3581473841 shutdown_connections 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921e2b40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7b70 0x555d921e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 --2- 172.21.15.42:0/3581473841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7250 0x555d921d7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 -- 172.21.15.42:0/3581473841 >> 172.21.15.42:0/3581473841 conn(0x555d921d1980 msgr2=0x555d921d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 -- 172.21.15.42:0/3581473841 shutdown_connections 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.276+0000 7f847c033740 1 -- 172.21.15.42:0/3581473841 wait complete. 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 -- start start 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 Processor -- start 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7250 0x555d921dd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921e0750 0x555d921de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9213f020 con 0x555d921d7250 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d91e12430 con 0x555d921d7b70 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d91dcee00 con 0x555d921e0750 2024-03-12T04:17:46.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f847c033740 10 client.0 _ll_register_callbacks cb 0x555d920e6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7250 0x555d921dd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7250 0x555d921dd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54824/0 (socket says 172.21.15.42:54824) 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 -- 172.21.15.42:0/2684847760 learned_addr learned my addr 172.21.15.42:0/2684847760 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f84737fe640 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921e0750 0x555d921de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f846bfff640 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 -- 172.21.15.42:0/2684847760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921e0750 msgr2=0x555d921de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921e0750 0x555d921de1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 -- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 msgr2=0x555d921ddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.277+0000 7f8472ffd640 1 -- 172.21.15.42:0/2684847760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84741a4480 con 0x555d921d7250 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f8472ffd640 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7250 0x555d921dd740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8464011480 tx=0x7f8464011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f1cb5cb2f5c875 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f845a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f845a7fc640 1 -- 172.21.15.42:0/2684847760 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464012980 con 0x555d921d7250 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f845a7fc640 1 -- 172.21.15.42:0/2684847760 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8464012ae0 con 0x555d921d7250 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d91e3e7e0 con 0x555d921d7250 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f845a7fc640 1 -- 172.21.15.42:0/2684847760 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8464012ca0 con 0x555d921d7250 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 10 client.16245 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555d9213d6a0 con 0x555d921d7250 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f845a7fc640 1 -- 172.21.15.42:0/2684847760 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x555d91e3e7e0 con 0x555d921d7250 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f845a7fc640 1 -- 172.21.15.42:0/2684847760 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f846401a3f0 con 0x555d921d7250 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 10 client.16245 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 20 client.16245 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d921f0b50 0x555d921f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 4 client.16245 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.278+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d91e7b720 con 0x555d921f0b50 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.279+0000 7f846bfff640 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d921f0b50 0x555d921f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.279+0000 7f845a7fc640 10 client.16245 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.279+0000 7f846bfff640 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d921f0b50 0x555d921f2f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6def4ec93c7f5a1c server_cookie=86f0289be635e70d in_seq=0 out_seq=0 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f845a7fc640 1 -- 172.21.15.42:0/2684847760 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d91e7b720 con 0x555d921f0b50 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f845a7fc640 10 client.16245 handle_command_reply: tid=0 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 client.16245 shutdown 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7250 msgr2=0x555d921dd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7250 0x555d921dd740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8464011480 tx=0x7f8464011940 comp rx=0 tx=0).stop 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 shutdown_connections 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d921f0b50 0x555d921f2f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921e0750 0x555d921de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 --2- 172.21.15.42:0/2684847760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7250 0x555d921dd740 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 >> 172.21.15.42:0/2684847760 conn(0x555d921d1980 msgr2=0x555d921d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f845a7fc640 0 client.16245 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 shutdown_connections 2024-03-12T04:17:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 1 -- 172.21.15.42:0/2684847760 wait complete. 2024-03-12T04:17:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.280+0000 7f847c033740 20 client.16245 trim_cache size 0 max 16384 2024-03-12T04:17:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 Processor -- start 2024-03-12T04:17:46.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 -- start start 2024-03-12T04:17:46.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d920c4130 con 0x555d921e0750 2024-03-12T04:17:46.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d921cd2e0 con 0x555d921d7b70 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.292+0000 7f847c033740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d921dbcb0 con 0x555d921d7250 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921da9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54826/0 (socket says 172.21.15.42:54826) 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 -- 172.21.15.42:0/1443429810 learned_addr learned my addr 172.21.15.42:0/1443429810 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f846bfff640 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f8472ffd640 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 -- 172.21.15.42:0/1443429810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 msgr2=0x555d921dfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921dfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 -- 172.21.15.42:0/1443429810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 msgr2=0x555d921ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 -- 172.21.15.42:0/1443429810 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d920d9c20 con 0x555d921e0750 2024-03-12T04:17:46.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f8472ffd640 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f84737fe640 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921da9e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f846c096a90 tx=0x7f846c005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b52160e6706368e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f8468ff9640 1 -- 172.21.15.42:0/1443429810 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c09e780 con 0x555d921e0750 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f8468ff9640 1 -- 172.21.15.42:0/1443429810 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f846c0054d0 con 0x555d921e0750 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.293+0000 7f8468ff9640 1 -- 172.21.15.42:0/1443429810 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0cb550 con 0x555d921e0750 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 -- 172.21.15.42:0/1443429810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 msgr2=0x555d921da9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921da9e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f846c096a90 tx=0x7f846c005440 comp rx=0 tx=0).stop 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 -- 172.21.15.42:0/1443429810 shutdown_connections 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921e0750 0x555d921da9e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921d7b70 0x555d921ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 --2- 172.21.15.42:0/1443429810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 -- 172.21.15.42:0/1443429810 >> 172.21.15.42:0/1443429810 conn(0x555d921d1980 msgr2=0x555d921f78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 -- 172.21.15.42:0/1443429810 shutdown_connections 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 -- 172.21.15.42:0/1443429810 wait complete. 2024-03-12T04:17:46.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 -- start start 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.294+0000 7f847c033740 1 Processor -- start 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7b70 0x555d921e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921e0750 0x555d921e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92247230 con 0x555d921d7b70 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d922473a0 con 0x555d921e0750 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92247560 con 0x555d921d7250 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 10 client.0 _ll_register_callbacks cb 0x7f846c00cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7b70 0x555d921e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7b70 0x555d921e6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54832/0 (socket says 172.21.15.42:54832) 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 -- 172.21.15.42:0/2827990643 learned_addr learned my addr 172.21.15.42:0/2827990643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f84737fe640 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921e0750 0x555d921e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f8472ffd640 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 -- 172.21.15.42:0/2827990643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 msgr2=0x555d921e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 -- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921e0750 msgr2=0x555d921e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921e0750 0x555d921e6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 -- 172.21.15.42:0/2827990643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d92101060 con 0x555d921d7b70 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f846bfff640 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7b70 0x555d921e6770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f845c036040 tx=0x7f845c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f121956fe5ef2c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f845a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f845a7fc640 1 -- 172.21.15.42:0/2827990643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c04f4e0 con 0x555d921d7b70 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f845a7fc640 1 -- 172.21.15.42:0/2827990643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f845c02de40 con 0x555d921d7b70 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.295+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f846c0a2dd0 con 0x555d921d7b70 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f845a7fc640 1 -- 172.21.15.42:0/2827990643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f845c02cc60 con 0x555d921d7b70 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 10 client.16251 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555d920ff8c0 con 0x555d921d7b70 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f845a7fc640 1 -- 172.21.15.42:0/2827990643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f846c0a2dd0 con 0x555d921d7b70 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f845a7fc640 1 -- 172.21.15.42:0/2827990643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f845c03c040 con 0x555d921d7b70 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 10 client.16251 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 10 client.16251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 20 client.16251 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d92248c70 0x555d9224b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 4 client.16251 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555d91eca890 con 0x555d92248c70 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.296+0000 7f8472ffd640 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d92248c70 0x555d9224b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f845a7fc640 10 client.16251 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f8472ffd640 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d92248c70 0x555d9224b090 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab1a7fb08502f738 server_cookie=fcd7d9360a8f4119 in_seq=0 out_seq=0 2024-03-12T04:17:46.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f845a7fc640 1 -- 172.21.15.42:0/2827990643 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555d91eca890 con 0x555d92248c70 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f845a7fc640 10 client.16251 handle_command_reply: tid=0 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 client.16251 shutdown 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7b70 msgr2=0x555d921e6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7b70 0x555d921e6770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f845c036040 tx=0x7f845c02bcd0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 shutdown_connections 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x555d92248c70 0x555d9224b090 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d921e0750 0x555d921e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f845a7fc640 0 client.16251 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d921d7b70 0x555d921e6770 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 --2- 172.21.15.42:0/2827990643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d921d7250 0x555d921e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 >> 172.21.15.42:0/2827990643 conn(0x555d921d1980 msgr2=0x555d921d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 shutdown_connections 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 1 -- 172.21.15.42:0/2827990643 wait complete. 2024-03-12T04:17:46.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.297+0000 7f847c033740 20 client.16251 trim_cache size 0 max 16384 2024-03-12T04:17:46.308 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f84600a1ba0 msgr2=0x7f84600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f84600a1ba0 0x7f84600a4060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c0047c0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8474107380 msgr2=0x7f847419bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8474107380 0x7f847419bff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f846c00eaa0 tx=0x7f846c00ef60 comp rx=0 tx=0).stop 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 shutdown_connections 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f84600a1ba0 0x7f84600a4060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8474107380 0x7f847419bff0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8474100400 0x7f8474197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 --2- 172.21.15.42:0/3845782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84740ffa40 0x7f8474197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 >> 172.21.15.42:0/3845782706 conn(0x7f84740753c0 msgr2=0x7f84740fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 shutdown_connections 2024-03-12T04:17:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.298+0000 7f8479464640 1 -- 172.21.15.42:0/3845782706 wait complete. 2024-03-12T04:17:46.311 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:46.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:17:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/1041736673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380685f0 msgr2=0x7f2a380689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/1041736673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380685f0 0x7f2a380689d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009930 tx=0x7f2a3002bc30 comp rx=0 tx=0).stop 2024-03-12T04:17:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/1041736673 shutdown_connections 2024-03-12T04:17:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/1041736673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381064c0 0x7f2a3810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/1041736673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a38068fa0 0x7f2a38105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/1041736673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380685f0 0x7f2a380689d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/1041736673 >> 172.21.15.42:0/1041736673 conn(0x7f2a38077180 msgr2=0x7f2a38077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/1041736673 shutdown_connections 2024-03-12T04:17:46.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.413+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/1041736673 wait complete. 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 Processor -- start 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 -- start start 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a380685f0 0x7f2a381976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381064c0 0x7f2a38197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3819bfb0 0x7f2a3819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819c9a0 con 0x7f2a3819bfb0 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819cb10 con 0x7f2a381064c0 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a3cb8a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819cc80 con 0x7f2a380685f0 2024-03-12T04:17:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.414+0000 7f2a36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3819bfb0 0x7f2a3819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3819bfb0 0x7f2a3819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54862/0 (socket says 172.21.15.42:54862) 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2481008733 learned_addr learned my addr 172.21.15.42:0/2481008733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381064c0 0x7f2a38197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a380685f0 0x7f2a381976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2481008733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a380685f0 msgr2=0x7f2a381976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a380685f0 0x7f2a381976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2481008733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381064c0 msgr2=0x7f2a38197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381064c0 0x7f2a38197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2481008733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30009560 con 0x7f2a3819bfb0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381064c0 0x7f2a38197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3819bfb0 0x7f2a3819c460 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800eb10 tx=0x7f2a2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a2f7fe640 1 -- 172.21.15.42:0/2481008733 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28004b20 con 0x7f2a3819bfb0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a2f7fe640 1 -- 172.21.15.42:0/2481008733 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a28004c80 con 0x7f2a3819bfb0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a381a4260 con 0x7f2a3819bfb0 2024-03-12T04:17:46.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a2f7fe640 1 -- 172.21.15.42:0/2481008733 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28010690 con 0x7f2a3819bfb0 2024-03-12T04:17:46.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.415+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381a46c0 con 0x7f2a3819bfb0 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.417+0000 7f2a2f7fe640 1 -- 172.21.15.42:0/2481008733 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f2a28004df0 con 0x7f2a3819bfb0 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.417+0000 7f2a3f836740 1 Processor -- start 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3f836740 1 -- start start 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b23630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a2f7fe640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23b70 0x5652e3b2c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b2c750 0x5652e3b2eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3f836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3ade960 con 0x5652e3b2c750 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3a92310 con 0x5652e3b23b70 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3f836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3a8f840 con 0x5652e3b23250 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b2c750 0x5652e3b2eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b2c750 0x5652e3b2eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54876/0 (socket says 172.21.15.42:54876) 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2561309113 learned_addr learned my addr 172.21.15.42:0/2561309113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a2f7fe640 1 -- 172.21.15.42:0/2481008733 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2a28014070 con 0x7f2a3819bfb0 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b23630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23b70 0x5652e3b2c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2561309113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 msgr2=0x5652e3b23630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b23630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2561309113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23b70 msgr2=0x5652e3b2c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23b70 0x5652e3b2c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a36d9d640 1 -- 172.21.15.42:0/2561309113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e3a4d060 con 0x5652e3b2c750 2024-03-12T04:17:46.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.418+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b23630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b2c750 0x5652e3b2eb40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a28097520 tx=0x7f2a2801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05adade58469670 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006010 tx=0x7f2a30000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a2cff9640 1 -- 172.21.15.42:0/2561309113 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2809b250 con 0x5652e3b2c750 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a2cff9640 1 -- 172.21.15.42:0/2561309113 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a2809ab50 con 0x5652e3b2c750 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a2cff9640 1 -- 172.21.15.42:0/2561309113 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280971c0 con 0x5652e3b2c750 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3f836740 1 -- 172.21.15.42:0/2561309113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b2c750 msgr2=0x5652e3b2eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b2c750 0x5652e3b2eb40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a28097520 tx=0x7f2a2801d000 comp rx=0 tx=0).stop 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3f836740 1 -- 172.21.15.42:0/2561309113 shutdown_connections 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b2c750 0x5652e3b2eb40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23b70 0x5652e3b2c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2561309113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b23630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.419+0000 7f2a3f836740 1 -- 172.21.15.42:0/2561309113 >> 172.21.15.42:0/2561309113 conn(0x5652e3b1d980 msgr2=0x5652e3b1dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 -- 172.21.15.42:0/2561309113 shutdown_connections 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 -- 172.21.15.42:0/2561309113 wait complete. 2024-03-12T04:17:46.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 -- start start 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 Processor -- start 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b29740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b29c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b2c750 0x5652e3b2a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3a8b020 con 0x5652e3b23b70 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e375e430 con 0x5652e3b2c750 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e371ae00 con 0x5652e3b23250 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3f836740 10 client.0 _ll_register_callbacks cb 0x5652e3a32cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b29c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b29c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54878/0 (socket says 172.21.15.42:54878) 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2128081696 learned_addr learned my addr 172.21.15.42:0/2128081696 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b29740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.420+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b2c750 0x5652e3b2a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 msgr2=0x5652e3b29740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b29740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2128081696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b2c750 msgr2=0x5652e3b2a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b2c750 0x5652e3b2a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2128081696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381a46c0 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b29c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2a200058c0 tx=0x7f2a20011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2c433817336f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a1dffb640 1 -- 172.21.15.42:0/2128081696 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20010df0 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a1dffb640 1 -- 172.21.15.42:0/2128081696 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2a20005020 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652e378a7e0 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a1dffb640 1 -- 172.21.15.42:0/2128081696 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a200051e0 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 10 client.16263 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5652e3a896a0 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a1dffb640 1 -- 172.21.15.42:0/2128081696 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5652e378a7e0 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a1dffb640 1 -- 172.21.15.42:0/2128081696 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2a2000e4c0 con 0x5652e3b23b70 2024-03-12T04:17:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 10 client.16263 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 20 client.16263 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b3cc20 0x5652e3b3f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 4 client.16263 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.421+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652e37c7720 con 0x5652e3b3cc20 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.422+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b3cc20 0x5652e3b3f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.422+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b3cc20 0x5652e3b3f040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=692aa8a6baa3a7bf server_cookie=b775b029c12d3384 in_seq=0 out_seq=0 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.422+0000 7f2a1dffb640 10 client.16263 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a1dffb640 1 -- 172.21.15.42:0/2128081696 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652e37c7720 con 0x5652e3b3cc20 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a1dffb640 10 client.16263 handle_command_reply: tid=0 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 client.16263 shutdown 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 msgr2=0x5652e3b29c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b29c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2a200058c0 tx=0x7f2a20011ef0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 shutdown_connections 2024-03-12T04:17:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b3cc20 0x5652e3b3f040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b2c750 0x5652e3b2a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b29c80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a1dffb640 0 client.16263 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2128081696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b23250 0x5652e3b29740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 >> 172.21.15.42:0/2128081696 conn(0x5652e3b1d980 msgr2=0x5652e3b1f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 shutdown_connections 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 1 -- 172.21.15.42:0/2128081696 wait complete. 2024-03-12T04:17:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.423+0000 7f2a3f836740 20 client.16263 trim_cache size 0 max 16384 2024-03-12T04:17:46.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 Processor -- start 2024-03-12T04:17:46.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 -- start start 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b29fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b1ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 0x5652e3b33600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3a10130 con 0x5652e3b23b70 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3b0fa70 con 0x5652e3b23250 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a3f836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3b1d7f0 con 0x5652e3b2c750 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.435+0000 7f2a35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b1ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b1ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54880/0 (socket says 172.21.15.42:54880) 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2688164650 learned_addr learned my addr 172.21.15.42:0/2688164650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b29fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2688164650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 msgr2=0x5652e3b33600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 0x5652e3b33600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2688164650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 msgr2=0x5652e3b29fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b29fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 -- 172.21.15.42:0/2688164650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e3a25c20 con 0x5652e3b23b70 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a3659c640 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b29fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b1ca00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2a20005d90 tx=0x7f2a20005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177b466edead82d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a1ffff640 1 -- 172.21.15.42:0/2688164650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000b530 con 0x5652e3b23b70 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a1ffff640 1 -- 172.21.15.42:0/2688164650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a200044d0 con 0x5652e3b23b70 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.436+0000 7f2a1ffff640 1 -- 172.21.15.42:0/2688164650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20010ee0 con 0x5652e3b23b70 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- 172.21.15.42:0/2688164650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 msgr2=0x5652e3b1ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b1ca00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2a20005d90 tx=0x7f2a20005ec0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- 172.21.15.42:0/2688164650 shutdown_connections 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 0x5652e3b33600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b1ca00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 --2- 172.21.15.42:0/2688164650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b29fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- 172.21.15.42:0/2688164650 >> 172.21.15.42:0/2688164650 conn(0x5652e3b1d980 msgr2=0x5652e3b1c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- 172.21.15.42:0/2688164650 shutdown_connections 2024-03-12T04:17:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- 172.21.15.42:0/2688164650 wait complete. 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- start start 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 Processor -- start 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b32350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b32890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 0x5652e3b93130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3b330d0 con 0x5652e3b23b70 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3b93670 con 0x5652e3b23250 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e3b937b0 con 0x5652e3b2c750 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a3f836740 10 client.0 _ll_register_callbacks cb 0x7f2a28004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b32890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b32890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54894/0 (socket says 172.21.15.42:54894) 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.437+0000 7f2a35d9b640 1 -- 172.21.15.42:0/402248331 learned_addr learned my addr 172.21.15.42:0/402248331 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a3659c640 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b32350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a35d9b640 1 -- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 msgr2=0x5652e3b93130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a36d9d640 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 0x5652e3b93130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 0x5652e3b93130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a35d9b640 1 -- 172.21.15.42:0/402248331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 msgr2=0x5652e3b32350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b32350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a35d9b640 1 -- 172.21.15.42:0/402248331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652e3a896a0 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a35d9b640 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b32890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000e870 tx=0x7f2a20010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00b33671e8e5922 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a1dffb640 1 -- 172.21.15.42:0/402248331 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000eae0 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a1dffb640 1 -- 172.21.15.42:0/402248331 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2a20000e40 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a2809ab50 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a1dffb640 1 -- 172.21.15.42:0/402248331 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a2000ec40 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a3f836740 10 client.16269 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5652e3a4b8c0 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a1dffb640 1 -- 172.21.15.42:0/402248331 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a2809ab50 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.438+0000 7f2a1dffb640 1 -- 172.21.15.42:0/402248331 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2a20032040 con 0x5652e3b23b70 2024-03-12T04:17:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 10 client.16269 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 20 client.16269 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b94b60 0x5652e3b96f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 4 client.16269 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5652e3816890 con 0x5652e3b94b60 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3659c640 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b94b60 0x5652e3b96f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a1dffb640 10 client.16269 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3659c640 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b94b60 0x5652e3b96f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec06ac0ba3faded server_cookie=90a2c51c19ecadff in_seq=0 out_seq=0 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a1dffb640 1 -- 172.21.15.42:0/402248331 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5652e3816890 con 0x5652e3b94b60 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a1dffb640 10 client.16269 handle_command_reply: tid=0 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 1 client.16269 shutdown 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 msgr2=0x5652e3b32890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.439+0000 7f2a3f836740 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b32890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000e870 tx=0x7f2a20010ea0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 shutdown_connections 2024-03-12T04:17:46.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5652e3b94b60 0x5652e3b96f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652e3b2c750 0x5652e3b93130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652e3b23b70 0x5652e3b32890 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a1dffb640 0 client.16269 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 --2- 172.21.15.42:0/402248331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652e3b23250 0x5652e3b32350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 >> 172.21.15.42:0/402248331 conn(0x5652e3b1d980 msgr2=0x5652e3b1cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 shutdown_connections 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 1 -- 172.21.15.42:0/402248331 wait complete. 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3f836740 20 client.16269 trim_cache size 0 max 16384 2024-03-12T04:17:46.449 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:46.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 msgr2=0x7f2a240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006010 tx=0x7f2a30000bc0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3819bfb0 msgr2=0x7f2a3819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3819bfb0 0x7f2a3819c460 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800eb10 tx=0x7f2a2800c2d0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 shutdown_connections 2024-03-12T04:17:46.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3819bfb0 0x7f2a3819c460 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381064c0 0x7f2a38197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 --2- 172.21.15.42:0/2481008733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a380685f0 0x7f2a381976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 >> 172.21.15.42:0/2481008733 conn(0x7f2a38077180 msgr2=0x7f2a380fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 shutdown_connections 2024-03-12T04:17:46.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.440+0000 7f2a3cb8a640 1 -- 172.21.15.42:0/2481008733 wait complete. 2024-03-12T04:17:46.452 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:46 smithi042 ceph-mon[32260]: pgmap v80: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-03-12T04:17:46.452 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:46 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2243778585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:17:46.453 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:17:46.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:17:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 -- 172.21.15.42:0/862470242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c105370 msgr2=0x7f964c10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 --2- 172.21.15.42:0/862470242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c105370 0x7f964c10cf40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9644009cd0 tx=0x7f964402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 -- 172.21.15.42:0/862470242 shutdown_connections 2024-03-12T04:17:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 --2- 172.21.15.42:0/862470242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c105370 0x7f964c10cf40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 --2- 172.21.15.42:0/862470242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964c100c50 0x7f964c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 --2- 172.21.15.42:0/862470242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f964c1002a0 0x7f964c100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 -- 172.21.15.42:0/862470242 >> 172.21.15.42:0/862470242 conn(0x7f964c076400 msgr2=0x7f964c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 -- 172.21.15.42:0/862470242 shutdown_connections 2024-03-12T04:17:46.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.555+0000 7f96509d7640 1 -- 172.21.15.42:0/862470242 wait complete. 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 Processor -- start 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 -- start start 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c1002a0 0x7f964c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f964c105370 0x7f964c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964c19ae90 0x7f964c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c19b880 con 0x7f964c1002a0 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c19b9f0 con 0x7f964c105370 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f96509d7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c19bb60 con 0x7f964c19ae90 2024-03-12T04:17:46.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f964a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c1002a0 0x7f964c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f964a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c1002a0 0x7f964c1965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54916/0 (socket says 172.21.15.42:54916) 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f964a59c640 1 -- 172.21.15.42:0/718189319 learned_addr learned my addr 172.21.15.42:0/718189319 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f9649d9b640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f964c105370 0x7f964c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f964ad9d640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964c19ae90 0x7f964c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.556+0000 7f964a59c640 1 -- 172.21.15.42:0/718189319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964c19ae90 msgr2=0x7f964c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f964a59c640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964c19ae90 0x7f964c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f964a59c640 1 -- 172.21.15.42:0/718189319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f964c105370 msgr2=0x7f964c196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f964a59c640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f964c105370 0x7f964c196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f964a59c640 1 -- 172.21.15.42:0/718189319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9644009900 con 0x7f964c1002a0 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f9649d9b640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f964c105370 0x7f964c196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f964ad9d640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964c19ae90 0x7f964c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f964a59c640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c1002a0 0x7f964c1965d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f963c00e9f0 tx=0x7f963c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f963b7fe640 1 -- 172.21.15.42:0/718189319 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c009700 con 0x7f964c1002a0 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f963b7fe640 1 -- 172.21.15.42:0/718189319 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f963c00cd80 con 0x7f964c1002a0 2024-03-12T04:17:46.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c1a3140 con 0x7f964c1002a0 2024-03-12T04:17:46.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f963b7fe640 1 -- 172.21.15.42:0/718189319 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c018840 con 0x7f964c1002a0 2024-03-12T04:17:46.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.557+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c1a3640 con 0x7f964c1002a0 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.559+0000 7f963b7fe640 1 -- 172.21.15.42:0/718189319 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f963c0189a0 con 0x7f964c1002a0 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.559+0000 7f96535ac740 1 Processor -- start 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.559+0000 7f96535ac740 1 -- start start 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f96535ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59eca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f96535ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f96535ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59ed7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f96535ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59e85960 con 0x55bc59ed5960 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f96535ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59e39310 con 0x55bc59eca060 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f96535ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59e36840 con 0x55bc59ecaa10 2024-03-12T04:17:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f963b7fe640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f96340a1c20 0x7f96340a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59ed7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59ed7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54918/0 (socket says 172.21.15.42:54918) 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 -- 172.21.15.42:0/2724997063 learned_addr learned my addr 172.21.15.42:0/2724997063 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f964ad9d640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f96340a1c20 0x7f96340a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f963b7fe640 1 -- 172.21.15.42:0/718189319 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f963c014070 con 0x7f964c1002a0 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f964a59c640 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f964ad9d640 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59eca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 -- 172.21.15.42:0/2724997063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 msgr2=0x55bc59ed5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 -- 172.21.15.42:0/2724997063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 msgr2=0x55bc59eca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59eca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 -- 172.21.15.42:0/2724997063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc59df4060 con 0x55bc59ed5960 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f964ad9d640 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59eca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f9649d9b640 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59ed7d50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9640009c30 tx=0x7f96400117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c747fd9b733765 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.560+0000 7f964ad9d640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f96340a1c20 0x7f96340a40e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9644004af0 tx=0x7f9644004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f9638ff9640 1 -- 172.21.15.42:0/2724997063 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640004020 con 0x55bc59ed5960 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f9638ff9640 1 -- 172.21.15.42:0/2724997063 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96400035f0 con 0x55bc59ed5960 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f9638ff9640 1 -- 172.21.15.42:0/2724997063 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640012670 con 0x55bc59ed5960 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 -- 172.21.15.42:0/2724997063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 msgr2=0x55bc59ed7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59ed7d50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9640009c30 tx=0x7f96400117b0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 -- 172.21.15.42:0/2724997063 shutdown_connections 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59ed7d50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 --2- 172.21.15.42:0/2724997063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59eca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 -- 172.21.15.42:0/2724997063 >> 172.21.15.42:0/2724997063 conn(0x55bc59ec4980 msgr2=0x55bc59ec4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 -- 172.21.15.42:0/2724997063 shutdown_connections 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.561+0000 7f96535ac740 1 -- 172.21.15.42:0/2724997063 wait complete. 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 -- start start 2024-03-12T04:17:46.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 Processor -- start 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ed5960 0x55bc59ed3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59e32020 con 0x55bc59eca060 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59b05430 con 0x55bc59ed5960 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59ac1e00 con 0x55bc59ecaa10 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f96535ac740 10 client.0 _ll_register_callbacks cb 0x55bc59dd9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54926/0 (socket says 172.21.15.42:54926) 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 -- 172.21.15.42:0/3749963801 learned_addr learned my addr 172.21.15.42:0/3749963801 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f9649d9b640 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964ad9d640 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ed5960 0x55bc59ed3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 -- 172.21.15.42:0/3749963801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 msgr2=0x55bc59ed2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 -- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ed5960 msgr2=0x55bc59ed3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ed5960 0x55bc59ed3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.562+0000 7f964a59c640 1 -- 172.21.15.42:0/3749963801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9640009860 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f964a59c640 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed28f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f963c096c80 tx=0x7f963c06cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ae41f0501e2d72 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f9631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f9631ffb640 1 -- 172.21.15.42:0/3749963801 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c09a820 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f9631ffb640 1 -- 172.21.15.42:0/3749963801 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f963c09e970 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc59b317e0 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f9631ffb640 1 -- 172.21.15.42:0/3749963801 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f963c0a2fc0 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 10 client.16281 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc59e306a0 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f9631ffb640 1 -- 172.21.15.42:0/3749963801 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bc59b317e0 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f9631ffb640 1 -- 172.21.15.42:0/3749963801 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f963c096570 con 0x55bc59eca060 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 10 client.16281 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 20 client.16281 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59ee5c30 0x55bc59ee8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 4 client.16281 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.563+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc59b6e720 con 0x55bc59ee5c30 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.564+0000 7f9649d9b640 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59ee5c30 0x55bc59ee8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.564+0000 7f9649d9b640 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59ee5c30 0x55bc59ee8020 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5113d53295ff9e9 server_cookie=b0e4a868244ab5f in_seq=0 out_seq=0 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.564+0000 7f9631ffb640 10 client.16281 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f9631ffb640 1 -- 172.21.15.42:0/3749963801 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc59b6e720 con 0x55bc59ee5c30 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f9631ffb640 10 client.16281 handle_command_reply: tid=0 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 client.16281 shutdown 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 msgr2=0x55bc59ed28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed28f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f963c096c80 tx=0x7f963c06cae0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 shutdown_connections 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59ee5c30 0x55bc59ee8020 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ed5960 0x55bc59ed3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f9631ffb640 0 client.16281 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59ed2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 --2- 172.21.15.42:0/3749963801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed28f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 >> 172.21.15.42:0/3749963801 conn(0x55bc59ec4980 msgr2=0x55bc59ec66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 shutdown_connections 2024-03-12T04:17:46.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 1 -- 172.21.15.42:0/3749963801 wait complete. 2024-03-12T04:17:46.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.565+0000 7f96535ac740 20 client.16281 trim_cache size 0 max 16384 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 Processor -- start 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 -- start start 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ecaa10 0x55bc59ec64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ed5960 0x55bc59edc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59db7130 con 0x55bc59eca060 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59ec95b0 con 0x55bc59ecaa10 2024-03-12T04:17:46.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f96535ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59ec5980 con 0x55bc59ed5960 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54936/0 (socket says 172.21.15.42:54936) 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 -- 172.21.15.42:0/3833428412 learned_addr learned my addr 172.21.15.42:0/3833428412 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964ad9d640 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ed5960 0x55bc59edc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f9649d9b640 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ecaa10 0x55bc59ec64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 -- 172.21.15.42:0/3833428412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ed5960 msgr2=0x55bc59edc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ed5960 0x55bc59edc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 -- 172.21.15.42:0/3833428412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ecaa10 msgr2=0x55bc59ec64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ecaa10 0x55bc59ec64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 -- 172.21.15.42:0/3833428412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc59dccc20 con 0x55bc59eca060 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f9649d9b640 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ecaa10 0x55bc59ec64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.577+0000 7f964a59c640 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed3190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f963c098ad0 tx=0x7f963c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273972c0bc5458ad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f9633fff640 1 -- 172.21.15.42:0/3833428412 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c09a5f0 con 0x55bc59eca060 2024-03-12T04:17:46.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f9633fff640 1 -- 172.21.15.42:0/3833428412 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f963c09e970 con 0x55bc59eca060 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f9633fff640 1 -- 172.21.15.42:0/3833428412 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0ba100 con 0x55bc59eca060 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 -- 172.21.15.42:0/3833428412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 msgr2=0x55bc59ed3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed3190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f963c098ad0 tx=0x7f963c00c7d0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 -- 172.21.15.42:0/3833428412 shutdown_connections 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ed5960 0x55bc59edc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59ecaa10 0x55bc59ec64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 --2- 172.21.15.42:0/3833428412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59eca060 0x55bc59ed3190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 -- 172.21.15.42:0/3833428412 >> 172.21.15.42:0/3833428412 conn(0x55bc59ec4980 msgr2=0x55bc59ec5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 -- 172.21.15.42:0/3833428412 shutdown_connections 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 -- 172.21.15.42:0/3833428412 wait complete. 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.578+0000 7f96535ac740 1 -- start start 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 Processor -- start 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59edbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59edc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59f36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59f36640 con 0x55bc59ed5960 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59f367b0 con 0x55bc59eca060 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc59f36970 con 0x55bc59ecaa10 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 10 client.0 _ll_register_callbacks cb 0x7f96400095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59f36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59f36100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54952/0 (socket says 172.21.15.42:54952) 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 -- 172.21.15.42:0/201946386 learned_addr learned my addr 172.21.15.42:0/201946386 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f9649d9b640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59edc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964a59c640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59edbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 -- 172.21.15.42:0/201946386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 msgr2=0x55bc59edc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59edc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 -- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 msgr2=0x55bc59edbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59edbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 -- 172.21.15.42:0/201946386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc59e306a0 con 0x55bc59ed5960 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964a59c640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59edbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:46.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f964ad9d640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59f36100 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f96440348c0 tx=0x7f964402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c06a60c20f6c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f9631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f9631ffb640 1 -- 172.21.15.42:0/201946386 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644003d80 con 0x55bc59ed5960 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f9631ffb640 1 -- 172.21.15.42:0/201946386 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9644047040 con 0x55bc59ed5960 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.579+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96400035f0 con 0x55bc59ed5960 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f9631ffb640 1 -- 172.21.15.42:0/201946386 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9644004040 con 0x55bc59ed5960 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 10 client.16287 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc59df28c0 con 0x55bc59ed5960 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f9631ffb640 1 -- 172.21.15.42:0/201946386 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96400035f0 con 0x55bc59ed5960 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f9631ffb640 1 -- 172.21.15.42:0/201946386 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9644060770 con 0x55bc59ed5960 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 10 client.16287 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 20 client.16287 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59f3e1e0 0x55bc59f40600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 4 client.16287 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:17:46.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc59bbd890 con 0x55bc59f3e1e0 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.580+0000 7f964a59c640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59f3e1e0 0x55bc59f40600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f964a59c640 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59f3e1e0 0x55bc59f40600 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee23d9ecaee4ca45 server_cookie=a80b75ab70897fa3 in_seq=0 out_seq=0 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f9631ffb640 10 client.16287 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f9631ffb640 1 -- 172.21.15.42:0/201946386 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc59bbd890 con 0x55bc59f3e1e0 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f9631ffb640 10 client.16287 handle_command_reply: tid=0 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 client.16287 shutdown 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 msgr2=0x55bc59f36100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59f36100 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f96440348c0 tx=0x7f964402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 shutdown_connections 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bc59f3e1e0 0x55bc59f40600 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc59ed5960 0x55bc59f36100 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc59ecaa10 0x55bc59edc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f9631ffb640 0 client.16287 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 --2- 172.21.15.42:0/201946386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc59eca060 0x55bc59edbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 >> 172.21.15.42:0/201946386 conn(0x55bc59ec4980 msgr2=0x55bc59ec3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 shutdown_connections 2024-03-12T04:17:46.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 1 -- 172.21.15.42:0/201946386 wait complete. 2024-03-12T04:17:46.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96535ac740 20 client.16287 trim_cache size 0 max 16384 2024-03-12T04:17:46.591 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f96340a1c20 msgr2=0x7f96340a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f96340a1c20 0x7f96340a40e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9644004af0 tx=0x7f9644004a40 comp rx=0 tx=0).stop 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c1002a0 msgr2=0x7f964c1965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c1002a0 0x7f964c1965d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f963c00e9f0 tx=0x7f963c00eeb0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 shutdown_connections 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f96340a1c20 0x7f96340a40e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964c19ae90 0x7f964c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f964c105370 0x7f964c196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 --2- 172.21.15.42:0/718189319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f964c1002a0 0x7f964c1965d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.581+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 >> 172.21.15.42:0/718189319 conn(0x7f964c076400 msgr2=0x7f964c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.582+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 shutdown_connections 2024-03-12T04:17:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:46.582+0000 7f96509d7640 1 -- 172.21.15.42:0/718189319 wait complete. 2024-03-12T04:17:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:46 smithi119 ceph-mon[36119]: pgmap v80: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-03-12T04:17:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:46 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2243778585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:17:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:46 smithi136 ceph-mon[35571]: pgmap v80: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-03-12T04:17:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:46 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2243778585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:17:47.595 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:17:47.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:17:47.595 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:17:47.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.712+0000 7f8747a64640 1 -- 172.21.15.42:0/539798829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101d40 msgr2=0x7f8740105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.712+0000 7f8747a64640 1 --2- 172.21.15.42:0/539798829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101d40 0x7f8740105d80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f873c009930 tx=0x7f873c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:17:47.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.712+0000 7f8747a64640 1 -- 172.21.15.42:0/539798829 shutdown_connections 2024-03-12T04:17:47.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.712+0000 7f8747a64640 1 --2- 172.21.15.42:0/539798829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8740106460 0x7f874010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.712+0000 7f8747a64640 1 --2- 172.21.15.42:0/539798829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101d40 0x7f8740105d80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.712+0000 7f8747a64640 1 --2- 172.21.15.42:0/539798829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8740101390 0x7f8740101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.712+0000 7f8747a64640 1 -- 172.21.15.42:0/539798829 >> 172.21.15.42:0/539798829 conn(0x7f87400fcbb0 msgr2=0x7f87400fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 -- 172.21.15.42:0/539798829 shutdown_connections 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 -- 172.21.15.42:0/539798829 wait complete. 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 Processor -- start 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 -- start start 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101390 0x7f8740197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8740106460 0x7f8740197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f874019bfd0 0x7f874019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874019c9c0 con 0x7f8740101390 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874019cb30 con 0x7f8740106460 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8747a64640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874019cca0 con 0x7f874019bfd0 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.713+0000 7f8745800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101390 0x7f8740197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101390 0x7f8740197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54972/0 (socket says 172.21.15.42:54972) 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 -- 172.21.15.42:0/754714504 learned_addr learned my addr 172.21.15.42:0/754714504 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8744fff640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8740106460 0x7f8740197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8746001640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f874019bfd0 0x7f874019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 -- 172.21.15.42:0/754714504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f874019bfd0 msgr2=0x7f874019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f874019bfd0 0x7f874019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 -- 172.21.15.42:0/754714504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8740106460 msgr2=0x7f8740197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8740106460 0x7f8740197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 -- 172.21.15.42:0/754714504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c009560 con 0x7f8740101390 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8745800640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101390 0x7f8740197710 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f873000d770 tx=0x7f873000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f87367fc640 1 -- 172.21.15.42:0/754714504 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730009c30 con 0x7f8740101390 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f87367fc640 1 -- 172.21.15.42:0/754714504 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f873000de80 con 0x7f8740101390 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87401a4280 con 0x7f8740101390 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f87367fc640 1 -- 172.21.15.42:0/754714504 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730004e00 con 0x7f8740101390 2024-03-12T04:17:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.714+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87401a4700 con 0x7f8740101390 2024-03-12T04:17:47.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f87367fc640 1 -- 172.21.15.42:0/754714504 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8730004f60 con 0x7f8740101390 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 Processor -- start 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 -- start start 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f87367fc640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f872c0a1c20 0x7f872c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e25760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362d93f90 con 0x564362e25380 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362d914c0 con 0x564362e2e8b0 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.717+0000 7f874a638740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362d8cca0 con 0x564362e25d30 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8746001640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f872c0a1c20 0x7f872c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e25760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f87367fc640 1 -- 172.21.15.42:0/754714504 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f87300cd1a0 con 0x7f8740101390 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e25760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54974/0 (socket says 172.21.15.42:54974) 2024-03-12T04:17:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 -- 172.21.15.42:0/3394054388 learned_addr learned my addr 172.21.15.42:0/3394054388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8744fff640 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8746001640 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 -- 172.21.15.42:0/3394054388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 msgr2=0x564362e2e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 -- 172.21.15.42:0/3394054388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 msgr2=0x564362e30d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e30d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 -- 172.21.15.42:0/3394054388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564362d4f0b0 con 0x564362e25380 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8746001640 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8746001640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f872c0a1c20 0x7f872c0a40e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8738008df0 tx=0x7f87380117b0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f8745800640 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e25760 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f87300a2090 tx=0x7f8730099840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7f9ce7762f34d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f872b7fe640 1 -- 172.21.15.42:0/3394054388 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873009df70 con 0x564362e25380 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f872b7fe640 1 -- 172.21.15.42:0/3394054388 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87300a25c0 con 0x564362e25380 2024-03-12T04:17:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.718+0000 7f872b7fe640 1 -- 172.21.15.42:0/3394054388 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730098200 con 0x564362e25380 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 -- 172.21.15.42:0/3394054388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 msgr2=0x564362e25760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e25760 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f87300a2090 tx=0x7f8730099840 comp rx=0 tx=0).stop 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 -- 172.21.15.42:0/3394054388 shutdown_connections 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 --2- 172.21.15.42:0/3394054388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e25760 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 -- 172.21.15.42:0/3394054388 >> 172.21.15.42:0/3394054388 conn(0x564362e1f840 msgr2=0x564362e1fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 -- 172.21.15.42:0/3394054388 shutdown_connections 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.719+0000 7f874a638740 1 -- 172.21.15.42:0/3394054388 wait complete. 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 -- start start 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 Processor -- start 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e25380 0x564362e2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e2e8b0 0x564362e2c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362d31f40 con 0x564362e2e8b0 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362a604b0 con 0x564362e25380 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362d12670 con 0x564362e25d30 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f874a638740 10 client.0 _ll_register_callbacks cb 0x564362d7ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e2e8b0 0x564362e2c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e2e8b0 0x564362e2c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54990/0 (socket says 172.21.15.42:54990) 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 -- 172.21.15.42:0/3943108756 learned_addr learned my addr 172.21.15.42:0/3943108756 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8745800640 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e25380 0x564362e2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8744fff640 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 -- 172.21.15.42:0/3943108756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 msgr2=0x564362e2bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 -- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e25380 msgr2=0x564362e2b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e25380 0x564362e2b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.720+0000 7f8746001640 1 -- 172.21.15.42:0/3943108756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87401a4700 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f8746001640 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e2e8b0 0x564362e2c310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8738008bd0 tx=0x7f8738004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d075ad8e7b2d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f8728ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f8728ff9640 1 -- 172.21.15.42:0/3943108756 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738027410 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f8728ff9640 1 -- 172.21.15.42:0/3943108756 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8738018040 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564362d66f40 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f8728ff9640 1 -- 172.21.15.42:0/3943108756 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8738027570 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 10 client.16299 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564362d8b320 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f8728ff9640 1 -- 172.21.15.42:0/3943108756 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564362d66f40 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f8728ff9640 1 -- 172.21.15.42:0/3943108756 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f873800b050 con 0x564362e2e8b0 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 10 client.16299 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:47.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 20 client.16299 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e3ed90 0x564362e411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 4 client.16299 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.721+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564362d83310 con 0x564362e3ed90 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.722+0000 7f8745800640 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e3ed90 0x564362e411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.722+0000 7f8728ff9640 10 client.16299 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.722+0000 7f8745800640 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e3ed90 0x564362e411b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9b30623aa1e51c server_cookie=8511136eb3e37d7a in_seq=0 out_seq=0 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f8728ff9640 1 -- 172.21.15.42:0/3943108756 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564362d83310 con 0x564362e3ed90 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f8728ff9640 10 client.16299 handle_command_reply: tid=0 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 client.16299 shutdown 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e2e8b0 msgr2=0x564362e2c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e2e8b0 0x564362e2c310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8738008bd0 tx=0x7f8738004ab0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 shutdown_connections 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e3ed90 0x564362e411b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e2e8b0 0x564362e2c310 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e2bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f8728ff9640 0 client.16299 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 --2- 172.21.15.42:0/3943108756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e25380 0x564362e2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 >> 172.21.15.42:0/3943108756 conn(0x564362e1f840 msgr2=0x564362e217a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 shutdown_connections 2024-03-12T04:17:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 1 -- 172.21.15.42:0/3943108756 wait complete. 2024-03-12T04:17:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.723+0000 7f874a638740 20 client.16299 trim_cache size 0 max 16384 2024-03-12T04:17:47.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 Processor -- start 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 -- start start 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e2dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e26340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e2c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362e3b930 con 0x564362e25380 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362e3eb90 con 0x564362e2e8b0 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f874a638740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362e463c0 con 0x564362e25d30 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f8745800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e2dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f8745800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e2dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54996/0 (socket says 172.21.15.42:54996) 2024-03-12T04:17:47.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.739+0000 7f8745800640 1 -- 172.21.15.42:0/4209439019 learned_addr learned my addr 172.21.15.42:0/4209439019 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f8746001640 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e2c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f8745800640 1 -- 172.21.15.42:0/4209439019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 msgr2=0x564362e26340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f8745800640 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e26340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f8745800640 1 -- 172.21.15.42:0/4209439019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 msgr2=0x564362e2c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f8745800640 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e2c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f8745800640 1 -- 172.21.15.42:0/4209439019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564362d27c00 con 0x564362e25380 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f8745800640 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e2dfc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f87300caf40 tx=0x7f8730015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584176b582ac9217 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f872affd640 1 -- 172.21.15.42:0/4209439019 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873009df70 con 0x564362e25380 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f872affd640 1 -- 172.21.15.42:0/4209439019 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8730099e20 con 0x564362e25380 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f872affd640 1 -- 172.21.15.42:0/4209439019 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300cabc0 con 0x564362e25380 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f874a638740 1 -- 172.21.15.42:0/4209439019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 msgr2=0x564362e2dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.740+0000 7f874a638740 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e2dfc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f87300caf40 tx=0x7f8730015040 comp rx=0 tx=0).stop 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- 172.21.15.42:0/4209439019 shutdown_connections 2024-03-12T04:17:47.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e2c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25d30 0x564362e26340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 --2- 172.21.15.42:0/4209439019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25380 0x564362e2dfc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- 172.21.15.42:0/4209439019 >> 172.21.15.42:0/4209439019 conn(0x564362e1f840 msgr2=0x564362e2a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- 172.21.15.42:0/4209439019 shutdown_connections 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- 172.21.15.42:0/4209439019 wait complete. 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- start start 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 Processor -- start 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25380 0x564362e357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25d30 0x564362e35d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e33d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362e34460 con 0x564362e25d30 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362e345d0 con 0x564362e2e8b0 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564362e34790 con 0x564362e25380 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f874a638740 10 client.0 _ll_register_callbacks cb 0x7f87300046e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.741+0000 7f8744fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25d30 0x564362e35d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25d30 0x564362e35d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55004/0 (socket says 172.21.15.42:55004) 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 -- 172.21.15.42:0/164395799 learned_addr learned my addr 172.21.15.42:0/164395799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8745800640 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25380 0x564362e357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8746001640 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e33d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 -- 172.21.15.42:0/164395799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25380 msgr2=0x564362e357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25380 0x564362e357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 -- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 msgr2=0x564362e33d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e33d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 -- 172.21.15.42:0/164395799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564362d4f0b0 con 0x564362e25d30 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8744fff640 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25d30 0x564362e35d10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f873c005ab0 tx=0x7f873c02cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd224459b4e49f36 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8728ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8728ff9640 1 -- 172.21.15.42:0/164395799 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c0042d0 con 0x564362e25d30 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8728ff9640 1 -- 172.21.15.42:0/164395799 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f873c004430 con 0x564362e25d30 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564362db2900 con 0x564362e25d30 2024-03-12T04:17:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8728ff9640 1 -- 172.21.15.42:0/164395799 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f873c0035f0 con 0x564362e25d30 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f874a638740 10 client.16305 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564362d4d910 con 0x564362e25d30 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.742+0000 7f8728ff9640 1 -- 172.21.15.42:0/164395799 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f873c055020 con 0x564362e25d30 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f8728ff9640 1 -- 172.21.15.42:0/164395799 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f873c03c040 con 0x564362e25d30 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f874a638740 10 client.16305 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f874a638740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f874a638740 20 client.16305 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f874a638740 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e96680 0x564362e98aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f874a638740 4 client.16305 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564362a8c860 con 0x564362e96680 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f8745800640 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e96680 0x564362e98aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f8728ff9640 10 client.16305 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.743+0000 7f8745800640 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e96680 0x564362e98aa0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fdd0e3e54307be7 server_cookie=4f93d4502661a759 in_seq=0 out_seq=0 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.744+0000 7f8728ff9640 1 -- 172.21.15.42:0/164395799 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564362a8c860 con 0x564362e96680 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.744+0000 7f8728ff9640 10 client.16305 handle_command_reply: tid=0 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.744+0000 7f874a638740 1 client.16305 shutdown 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25d30 msgr2=0x564362e35d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25d30 0x564362e35d10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f873c005ab0 tx=0x7f873c02cc20 comp rx=0 tx=0).stop 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 shutdown_connections 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564362e96680 0x564362e98aa0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564362e2e8b0 0x564362e33d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564362e25d30 0x564362e35d10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 --2- 172.21.15.42:0/164395799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564362e25380 0x564362e357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f8728ff9640 0 client.16305 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 >> 172.21.15.42:0/164395799 conn(0x564362e1f840 msgr2=0x564362e20e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 shutdown_connections 2024-03-12T04:17:47.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 1 -- 172.21.15.42:0/164395799 wait complete. 2024-03-12T04:17:47.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f874a638740 20 client.16305 trim_cache size 0 max 16384 2024-03-12T04:17:47.758 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:17:47.758 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:17:47.758 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "0882f582-780b-4792-89f2-7af054517d10", 2024-03-12T04:17:47.758 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:17:47.758 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f872c0a1c20 msgr2=0x7f872c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f8747a64640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f872c0a1c20 0x7f872c0a40e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8738008df0 tx=0x7f87380117b0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101390 msgr2=0x7f8740197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.745+0000 7f8747a64640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101390 0x7f8740197710 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f873000d770 tx=0x7f873000dc30 comp rx=0 tx=0).stop 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 shutdown_connections 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f872c0a1c20 0x7f872c0a40e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f874019bfd0 0x7f874019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8740106460 0x7f8740197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 --2- 172.21.15.42:0/754714504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8740101390 0x7f8740197710 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 >> 172.21.15.42:0/754714504 conn(0x7f87400fcbb0 msgr2=0x7f87400fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 shutdown_connections 2024-03-12T04:17:47.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.746+0000 7f8747a64640 1 -- 172.21.15.42:0/754714504 wait complete. 2024-03-12T04:17:47.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:17:47.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:17:47.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 -- 172.21.15.42:0/1287130784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 msgr2=0x7f57c01064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/1287130784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c01064f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009900 tx=0x7f57b002bc30 comp rx=0 tx=0).stop 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 -- 172.21.15.42:0/1287130784 shutdown_connections 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/1287130784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c0100c90 0x7f57c010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/1287130784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c01002d0 0x7f57c0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/1287130784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c01064f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 -- 172.21.15.42:0/1287130784 >> 172.21.15.42:0/1287130784 conn(0x7f57c0076e20 msgr2=0x7f57c0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 -- 172.21.15.42:0/1287130784 shutdown_connections 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.863+0000 7f57c5b3a640 1 -- 172.21.15.42:0/1287130784 wait complete. 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 Processor -- start 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 -- start start 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c01002d0 0x7f57c010d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c0100c90 0x7f57c010dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c010e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01a5a70 con 0x7f57c0106110 2024-03-12T04:17:47.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01a5be0 con 0x7f57c01002d0 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57c5b3a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01a5da0 con 0x7f57c0100c90 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c010e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c010e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55022/0 (socket says 172.21.15.42:55022) 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 -- 172.21.15.42:0/515943856 learned_addr learned my addr 172.21.15.42:0/515943856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57beffd640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c0100c90 0x7f57c010dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c01002d0 0x7f57c010d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 -- 172.21.15.42:0/515943856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c0100c90 msgr2=0x7f57c010dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c0100c90 0x7f57c010dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 -- 172.21.15.42:0/515943856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c01002d0 msgr2=0x7f57c010d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c01002d0 0x7f57c010d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.864+0000 7f57bffff640 1 -- 172.21.15.42:0/515943856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0009560 con 0x7f57c0106110 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c01002d0 0x7f57c010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:47.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57beffd640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c0100c90 0x7f57c010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:17:47.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57bffff640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c010e2e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f57b800cad0 tx=0x7f57b800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57bcff9640 1 -- 172.21.15.42:0/515943856 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800ae20 con 0x7f57c0106110 2024-03-12T04:17:47.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57bcff9640 1 -- 172.21.15.42:0/515943856 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57b8005020 con 0x7f57c0106110 2024-03-12T04:17:47.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c01a6080 con 0x7f57c0106110 2024-03-12T04:17:47.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57bcff9640 1 -- 172.21.15.42:0/515943856 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800f690 con 0x7f57c0106110 2024-03-12T04:17:47.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.865+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c01a6540 con 0x7f57c0106110 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.867+0000 7f57bcff9640 1 -- 172.21.15.42:0/515943856 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f57b8005190 con 0x7f57c0106110 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.867+0000 7f57c87e4740 1 Processor -- start 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.867+0000 7f57c87e4740 1 -- start start 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bcff9640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57a80a1ba0 0x7f57a80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cbbd0 0x5648243d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x5648243d6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57c87e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482433a280 con 0x5648243cb2b0 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57c87e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648243377b0 con 0x5648243cbbd0 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57c87e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564824332f90 con 0x5648243d47b0 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57beffd640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57a80a1ba0 0x7f57a80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bcff9640 1 -- 172.21.15.42:0/515943856 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f57b80cd8c0 con 0x7f57c0106110 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55026/0 (socket says 172.21.15.42:55026) 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 -- 172.21.15.42:0/4164476152 learned_addr learned my addr 172.21.15.42:0/4164476152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57beffd640 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cbbd0 0x5648243d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bffff640 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x5648243d6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 -- 172.21.15.42:0/4164476152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 msgr2=0x5648243d6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x5648243d6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 -- 172.21.15.42:0/4164476152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cbbd0 msgr2=0x5648243d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cbbd0 0x5648243d4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 -- 172.21.15.42:0/4164476152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648242f1570 con 0x5648243cb2b0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57beffd640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57a80a1ba0 0x7f57a80a4060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57c010f2f0 tx=0x7f57b4013820 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.868+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243cb690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009900 tx=0x7f57b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f4da922ca23d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57a5ffb640 1 -- 172.21.15.42:0/4164476152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b002c770 con 0x5648243cb2b0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57a5ffb640 1 -- 172.21.15.42:0/4164476152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57b002de90 con 0x5648243cb2b0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57a5ffb640 1 -- 172.21.15.42:0/4164476152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b00039e0 con 0x5648243cb2b0 2024-03-12T04:17:47.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 -- 172.21.15.42:0/4164476152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 msgr2=0x5648243cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243cb690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009900 tx=0x7f57b00046c0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 -- 172.21.15.42:0/4164476152 shutdown_connections 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x5648243d6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cbbd0 0x5648243d4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 --2- 172.21.15.42:0/4164476152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243cb690 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 -- 172.21.15.42:0/4164476152 >> 172.21.15.42:0/4164476152 conn(0x5648243c5990 msgr2=0x5648243c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 -- 172.21.15.42:0/4164476152 shutdown_connections 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.869+0000 7f57c87e4740 1 -- 172.21.15.42:0/4164476152 wait complete. 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 -- start start 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 Processor -- start 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243d47b0 0x5648243d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564824006430 con 0x5648243cb2b0 2024-03-12T04:17:47.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823fc2e00 con 0x5648243d47b0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648242b7fd0 con 0x5648243cbbd0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57c87e4740 10 client.0 _ll_register_callbacks cb 0x56482430dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243d41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55040/0 (socket says 172.21.15.42:55040) 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 -- 172.21.15.42:0/679152261 learned_addr learned my addr 172.21.15.42:0/679152261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57beffd640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bffff640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243d47b0 0x5648243d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 -- 172.21.15.42:0/679152261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 msgr2=0x5648243d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 -- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243d47b0 msgr2=0x5648243d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243d47b0 0x5648243d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bf7fe640 1 -- 172.21.15.42:0/679152261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c0069810 con 0x5648243cb2b0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.870+0000 7f57bffff640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243d47b0 0x5648243d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243d41f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f57b0004ff0 tx=0x7f57b0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b02ff85579f8a73 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f579effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f579effd640 1 -- 172.21.15.42:0/679152261 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0005550 con 0x5648243cb2b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648240327e0 con 0x5648243cb2b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f579effd640 1 -- 172.21.15.42:0/679152261 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f57b0034440 con 0x5648243cb2b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f579effd640 1 -- 172.21.15.42:0/679152261 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b0004ab0 con 0x5648243cb2b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 10 client.16317 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564824331610 con 0x5648243cb2b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f579effd640 1 -- 172.21.15.42:0/679152261 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5648240327e0 con 0x5648243cb2b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f579effd640 1 -- 172.21.15.42:0/679152261 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57b003c040 con 0x5648243cb2b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 10 client.16317 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 20 client.16317 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5648243e44b0 0x5648243e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 4 client.16317 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.871+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56482406f720 con 0x5648243e44b0 2024-03-12T04:17:47.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.872+0000 7f57beffd640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5648243e44b0 0x5648243e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.872+0000 7f57beffd640 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5648243e44b0 0x5648243e68d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e9358e75d3cd08 server_cookie=49b5218c7e87ea51 in_seq=0 out_seq=0 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.872+0000 7f579effd640 10 client.16317 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f579effd640 1 -- 172.21.15.42:0/679152261 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56482406f720 con 0x5648243e44b0 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f579effd640 10 client.16317 handle_command_reply: tid=0 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 client.16317 shutdown 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 msgr2=0x5648243d41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243d41f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f57b0004ff0 tx=0x7f57b0005020 comp rx=0 tx=0).stop 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 shutdown_connections 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5648243e44b0 0x5648243e68d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243d47b0 0x5648243d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f579effd640 0 client.16317 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 --2- 172.21.15.42:0/679152261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cb2b0 0x5648243d41f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 >> 172.21.15.42:0/679152261 conn(0x5648243c5990 msgr2=0x5648243c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 shutdown_connections 2024-03-12T04:17:47.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 1 -- 172.21.15.42:0/679152261 wait complete. 2024-03-12T04:17:47.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.873+0000 7f57c87e4740 20 client.16317 trim_cache size 0 max 16384 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 Processor -- start 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 -- start start 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243d47b0 0x5648243d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648243c6c10 con 0x5648243d47b0 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648243c10f0 con 0x5648243cb2b0 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57c87e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648243cadb0 con 0x5648243cbbd0 2024-03-12T04:17:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57bffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243d47b0 0x5648243d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57bffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243d47b0 0x5648243d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55054/0 (socket says 172.21.15.42:55054) 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57bffff640 1 -- 172.21.15.42:0/2559676818 learned_addr learned my addr 172.21.15.42:0/2559676818 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.888+0000 7f57beffd640 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57bffff640 1 -- 172.21.15.42:0/2559676818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 msgr2=0x5648243d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57bffff640 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57bffff640 1 -- 172.21.15.42:0/2559676818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 msgr2=0x5648243d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57bffff640 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57bffff640 1 -- 172.21.15.42:0/2559676818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648242cdac0 con 0x5648243d47b0 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57beffd640 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57bffff640 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243d47b0 0x5648243d0e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f57b8096940 tx=0x7f57b809a020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c0dca1c2a7194f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57a57fa640 1 -- 172.21.15.42:0/2559676818 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800f7f0 con 0x5648243d47b0 2024-03-12T04:17:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57a57fa640 1 -- 172.21.15.42:0/2559676818 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57b8004a40 con 0x5648243d47b0 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57a57fa640 1 -- 172.21.15.42:0/2559676818 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80a2c80 con 0x5648243d47b0 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 -- 172.21.15.42:0/2559676818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243d47b0 msgr2=0x5648243d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243d47b0 0x5648243d0e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f57b8096940 tx=0x7f57b809a020 comp rx=0 tx=0).stop 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 -- 172.21.15.42:0/2559676818 shutdown_connections 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243d47b0 0x5648243d0e10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243cbbd0 0x5648243d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 --2- 172.21.15.42:0/2559676818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 -- 172.21.15.42:0/2559676818 >> 172.21.15.42:0/2559676818 conn(0x5648243c5990 msgr2=0x5648243cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 -- 172.21.15.42:0/2559676818 shutdown_connections 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.889+0000 7f57c87e4740 1 -- 172.21.15.42:0/2559676818 wait complete. 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 -- start start 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 Processor -- start 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cbbd0 0x5648243d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x56482443a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648243d9400 con 0x5648243cbbd0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482443af60 con 0x5648243cb2b0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482443b120 con 0x5648243d47b0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57c87e4740 10 client.0 _ll_register_callbacks cb 0x7f57b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57beffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cbbd0 0x5648243d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57beffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cbbd0 0x5648243d8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55068/0 (socket says 172.21.15.42:55068) 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57beffd640 1 -- 172.21.15.42:0/1054006826 learned_addr learned my addr 172.21.15.42:0/1054006826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.890+0000 7f57bffff640 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x56482443a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57beffd640 1 -- 172.21.15.42:0/1054006826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 msgr2=0x56482443a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57beffd640 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x56482443a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57beffd640 1 -- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 msgr2=0x5648243d8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57beffd640 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57beffd640 1 -- 172.21.15.42:0/1054006826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c01a6540 con 0x5648243cbbd0 2024-03-12T04:17:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57beffd640 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cbbd0 0x5648243d8c60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f57b40042a0 tx=0x7f57b401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84746a0fdc23be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f579effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f579effd640 1 -- 172.21.15.42:0/1054006826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4025040 con 0x5648243cbbd0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f579effd640 1 -- 172.21.15.42:0/1054006826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f57b4028be0 con 0x5648243cbbd0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57b002de90 con 0x5648243cbbd0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f579effd640 1 -- 172.21.15.42:0/1054006826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b4028da0 con 0x5648243cbbd0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 10 client.16323 fetch_fsmap learned FSMap version 27 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648242f36a0 con 0x5648243cbbd0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f579effd640 1 -- 172.21.15.42:0/1054006826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57b002de90 con 0x5648243cbbd0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f579effd640 1 -- 172.21.15.42:0/1054006826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57b402e020 con 0x5648243cbbd0 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 10 client.16323 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 20 client.16323 populate_metadata read hostname 'smithi042' 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56482443c910 0x56482443ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 4 client.16323 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.891+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5648240be890 con 0x56482443c910 2024-03-12T04:17:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56482443c910 0x56482443ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f579effd640 10 client.16323 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f57bf7fe640 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56482443c910 0x56482443ed30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2fdd6dde2571f16 server_cookie=d1a44d6dad20d1ae in_seq=0 out_seq=0 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f579effd640 1 -- 172.21.15.42:0/1054006826 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5648240be890 con 0x56482443c910 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f579effd640 10 client.16323 handle_command_reply: tid=0 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f57c87e4740 1 client.16323 shutdown 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cbbd0 msgr2=0x5648243d8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.892+0000 7f57c87e4740 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cbbd0 0x5648243d8c60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f57b40042a0 tx=0x7f57b401b000 comp rx=0 tx=0).stop 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 shutdown_connections 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56482443c910 0x56482443ed30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648243d47b0 0x56482443a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648243cbbd0 0x5648243d8c60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f579effd640 0 client.16323 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 --2- 172.21.15.42:0/1054006826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648243cb2b0 0x5648243d8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 >> 172.21.15.42:0/1054006826 conn(0x5648243c5990 msgr2=0x5648243d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 shutdown_connections 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 1 -- 172.21.15.42:0/1054006826 wait complete. 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c87e4740 20 client.16323 trim_cache size 0 max 16384 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: "0882f582-780b-4792-89f2-7af054517d10": { 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "0882f582-780b-4792-89f2-7af054517d10", 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:17:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:17:47.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:17:47.903 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:17:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57a80a1ba0 msgr2=0x7f57a80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57a80a1ba0 0x7f57a80a4060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57c010f2f0 tx=0x7f57b4013820 comp rx=0 tx=0).stop 2024-03-12T04:17:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 msgr2=0x7f57c010e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c010e2e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f57b800cad0 tx=0x7f57b800cf90 comp rx=0 tx=0).stop 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 shutdown_connections 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57a80a1ba0 0x7f57a80a4060 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57c0106110 0x7f57c010e2e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c0100c90 0x7f57c010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 --2- 172.21.15.42:0/515943856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c01002d0 0x7f57c010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 >> 172.21.15.42:0/515943856 conn(0x7f57c0076e20 msgr2=0x7f57c00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 shutdown_connections 2024-03-12T04:17:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:17:47.893+0000 7f57c5b3a640 1 -- 172.21.15.42:0/515943856 wait complete. 2024-03-12T04:17:47.906 INFO:tasks.cephfs.filesystem:scrub status for tag:0882f582-780b-4792-89f2-7af054517d10 - {'path': '/', 'tag': '0882f582-780b-4792-89f2-7af054517d10', 'options': 'recursive,force'} 2024-03-12T04:17:48.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:48 smithi042 ceph-mon[32260]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-03-12T04:17:48.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:48 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:17:48.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:48 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:48.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:48 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:17:48.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:48 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:48.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:48 smithi119 ceph-mon[36119]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-03-12T04:17:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:48 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:17:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:48 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:48 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:17:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:48 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:48 smithi136 ceph-mon[35571]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-03-12T04:17:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:48 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:17:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:48 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:48 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:17:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:48 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:17:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:49 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:17:49.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:49 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:17:49.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:49 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:17:50.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:50 smithi042 ceph-mon[32260]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-12T04:17:50.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:50 smithi119 ceph-mon[36119]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-12T04:17:50.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:50 smithi136 ceph-mon[35571]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-12T04:17:52.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:52 smithi042 ceph-mon[32260]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-12T04:17:52.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:52 smithi119 ceph-mon[36119]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-12T04:17:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:52 smithi136 ceph-mon[35571]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-12T04:17:54.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:54 smithi042 ceph-mon[32260]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-12T04:17:54.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:54 smithi119 ceph-mon[36119]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-12T04:17:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:54 smithi136 ceph-mon[35571]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-12T04:17:56.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:56 smithi042 ceph-mon[32260]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-03-12T04:17:56.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:56 smithi119 ceph-mon[36119]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-03-12T04:17:56.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:56 smithi136 ceph-mon[35571]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-03-12T04:17:58.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:17:58 smithi042 ceph-mon[32260]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-12T04:17:58.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:17:58 smithi119 ceph-mon[36119]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-12T04:17:58.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:17:58 smithi136 ceph-mon[35571]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-12T04:18:00.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:00 smithi042 ceph-mon[32260]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:00.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:00 smithi119 ceph-mon[36119]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:00.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:00 smithi136 ceph-mon[35571]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:02.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:02 smithi042 ceph-mon[32260]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:02.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:02 smithi119 ceph-mon[36119]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:02.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:02 smithi136 ceph-mon[35571]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:04.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:04 smithi042 ceph-mon[32260]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:04.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:04 smithi119 ceph-mon[36119]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:04.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:04 smithi136 ceph-mon[35571]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:06.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:06 smithi119 ceph-mon[36119]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:06.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:06 smithi136 ceph-mon[35571]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:06.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:06 smithi042 ceph-mon[32260]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:08.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:08 smithi119 ceph-mon[36119]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:08 smithi136 ceph-mon[35571]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:08 smithi042 ceph-mon[32260]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:10.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:10 smithi119 ceph-mon[36119]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:10.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:10 smithi136 ceph-mon[35571]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:10.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:10 smithi042 ceph-mon[32260]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:12.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:12 smithi042 ceph-mon[32260]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:12.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:12 smithi119 ceph-mon[36119]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:12 smithi136 ceph-mon[35571]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:18:14.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:14 smithi119 ceph-mon[36119]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:14 smithi136 ceph-mon[35571]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:14 smithi042 ceph-mon[32260]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:18:16.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:16 smithi119 ceph-mon[36119]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:18:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:16 smithi136 ceph-mon[35571]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:18:16.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:16 smithi042 ceph-mon[32260]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:18:17.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:18:17.908 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:18:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 -- 172.21.15.42:0/2598788869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 msgr2=0x7fc97c1064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 --2- 172.21.15.42:0/2598788869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c1064f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc964009900 tx=0x7fc96402bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 -- 172.21.15.42:0/2598788869 shutdown_connections 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 --2- 172.21.15.42:0/2598788869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc97c100c90 0x7fc97c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 --2- 172.21.15.42:0/2598788869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc97c1002d0 0x7fc97c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 --2- 172.21.15.42:0/2598788869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c1064f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 -- 172.21.15.42:0/2598788869 >> 172.21.15.42:0/2598788869 conn(0x7fc97c076e20 msgr2=0x7fc97c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 -- 172.21.15.42:0/2598788869 shutdown_connections 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.023+0000 7fc981ac3640 1 -- 172.21.15.42:0/2598788869 wait complete. 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 Processor -- start 2024-03-12T04:18:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 -- start start 2024-03-12T04:18:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc97c1002d0 0x7fc97c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc97c100c90 0x7fc97c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c19c520 con 0x7fc97c106110 2024-03-12T04:18:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c19c690 con 0x7fc97c1002d0 2024-03-12T04:18:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc981ac3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c19c850 con 0x7fc97c100c90 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c19bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41184/0 (socket says 172.21.15.42:41184) 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.024+0000 7fc97bfff640 1 -- 172.21.15.42:0/212893986 learned_addr learned my addr 172.21.15.42:0/212893986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc97c1002d0 0x7fc97c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97affd640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc97c100c90 0x7fc97c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97bfff640 1 -- 172.21.15.42:0/212893986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc97c100c90 msgr2=0x7fc97c197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97bfff640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc97c100c90 0x7fc97c197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97bfff640 1 -- 172.21.15.42:0/212893986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc97c1002d0 msgr2=0x7fc97c197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97bfff640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc97c1002d0 0x7fc97c197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97bfff640 1 -- 172.21.15.42:0/212893986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964009560 con 0x7fc97c106110 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc97c1002d0 0x7fc97c197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc97bfff640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c19bf20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc97400cce0 tx=0x7fc97400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc978ff9640 1 -- 172.21.15.42:0/212893986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400f450 con 0x7fc97c106110 2024-03-12T04:18:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc978ff9640 1 -- 172.21.15.42:0/212893986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc97400fa60 con 0x7fc97c106110 2024-03-12T04:18:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c19cb30 con 0x7fc97c106110 2024-03-12T04:18:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc978ff9640 1 -- 172.21.15.42:0/212893986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974018b10 con 0x7fc97c106110 2024-03-12T04:18:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.025+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c1a43b0 con 0x7fc97c106110 2024-03-12T04:18:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc978ff9640 1 -- 172.21.15.42:0/212893986 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fc97400fbd0 con 0x7fc97c106110 2024-03-12T04:18:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 Processor -- start 2024-03-12T04:18:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 -- start start 2024-03-12T04:18:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc978ff9640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc9680a1ba0 0x7fc9680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014924d280 con 0x5601492e77b0 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014924a7b0 con 0x5601492de2b0 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc984775740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560149245f90 con 0x5601492debd0 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc97affd640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc9680a1ba0 0x7fc9680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc978ff9640 1 -- 172.21.15.42:0/212893986 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc9740cdd00 con 0x7fc97c106110 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41196/0 (socket says 172.21.15.42:41196) 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc97bfff640 1 -- 172.21.15.42:0/646362263 learned_addr learned my addr 172.21.15.42:0/646362263 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.028+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97affd640 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97bfff640 1 -- 172.21.15.42:0/646362263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 msgr2=0x5601492e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97bfff640 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97bfff640 1 -- 172.21.15.42:0/646362263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 msgr2=0x5601492de690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97bfff640 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492de690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97bfff640 1 -- 172.21.15.42:0/646362263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560149204570 con 0x5601492e77b0 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97affd640 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97bfff640 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e9ba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc97409a620 tx=0x7fc974096bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d45dd69d6a45a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc97affd640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc9680a1ba0 0x7fc9680a4060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5601492ddd80 tx=0x7fc96c009340 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc971ffb640 1 -- 172.21.15.42:0/646362263 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97409a280 con 0x5601492e77b0 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc971ffb640 1 -- 172.21.15.42:0/646362263 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc97400d750 con 0x5601492e77b0 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.029+0000 7fc971ffb640 1 -- 172.21.15.42:0/646362263 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974096f90 con 0x5601492e77b0 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 -- 172.21.15.42:0/646362263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 msgr2=0x5601492e9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e9ba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc97409a620 tx=0x7fc974096bc0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 -- 172.21.15.42:0/646362263 shutdown_connections 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e9ba0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 --2- 172.21.15.42:0/646362263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 -- 172.21.15.42:0/646362263 >> 172.21.15.42:0/646362263 conn(0x5601492d8990 msgr2=0x5601492d8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 -- 172.21.15.42:0/646362263 shutdown_connections 2024-03-12T04:18:18.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 -- 172.21.15.42:0/646362263 wait complete. 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 -- start start 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.030+0000 7fc984775740 1 Processor -- start 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc984775740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492e7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc984775740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc984775740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc984775740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560148f19430 con 0x5601492e77b0 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc984775740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560148ed5e00 con 0x5601492de2b0 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc984775740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601491cafd0 con 0x5601492debd0 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc984775740 10 client.0 _ll_register_callbacks cb 0x560149220fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41206/0 (socket says 172.21.15.42:41206) 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 -- 172.21.15.42:0/3125005055 learned_addr learned my addr 172.21.15.42:0/3125005055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97affd640 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 -- 172.21.15.42:0/3125005055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 msgr2=0x5601492e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 -- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 msgr2=0x5601492e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492e7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 -- 172.21.15.42:0/3125005055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c1a43b0 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc97bfff640 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e4cd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc97409a8b0 tx=0x7fc97409eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b132b9da31aa06 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.031+0000 7fc95affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc95affd640 1 -- 172.21.15.42:0/3125005055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9740cb8c0 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc95affd640 1 -- 172.21.15.42:0/3125005055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc97409b020 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560148f457e0 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc95affd640 1 -- 172.21.15.42:0/3125005055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc97409a920 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 10 client.16335 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560149244610 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc95affd640 1 -- 172.21.15.42:0/3125005055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x560148f457e0 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc95affd640 1 -- 172.21.15.42:0/3125005055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc974016020 con 0x5601492e77b0 2024-03-12T04:18:18.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 10 client.16335 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 20 client.16335 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601492f73e0 0x5601492f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 4 client.16335 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560148f82720 con 0x5601492f73e0 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.032+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601492f73e0 0x5601492f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.033+0000 7fc95affd640 10 client.16335 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.033+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601492f73e0 0x5601492f97d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445dfd5f5bd4272f server_cookie=4390222249228fba in_seq=0 out_seq=0 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.033+0000 7fc95affd640 1 -- 172.21.15.42:0/3125005055 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560148f82720 con 0x5601492f73e0 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.033+0000 7fc95affd640 10 client.16335 handle_command_reply: tid=0 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.033+0000 7fc984775740 1 client.16335 shutdown 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 msgr2=0x5601492e4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e4cd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc97409a8b0 tx=0x7fc97409eae0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 shutdown_connections 2024-03-12T04:18:18.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601492f73e0 0x5601492f97d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492e77b0 0x5601492e4cd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492debd0 0x5601492e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 --2- 172.21.15.42:0/3125005055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc95affd640 0 client.16335 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 >> 172.21.15.42:0/3125005055 conn(0x5601492d8990 msgr2=0x5601492da730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 shutdown_connections 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 1 -- 172.21.15.42:0/3125005055 wait complete. 2024-03-12T04:18:18.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.034+0000 7fc984775740 20 client.16335 trim_cache size 0 max 16384 2024-03-12T04:18:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 Processor -- start 2024-03-12T04:18:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 -- start start 2024-03-12T04:18:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492ee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601492f44b0 con 0x5601492debd0 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601492d40f0 con 0x5601492de2b0 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc984775740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601492f3570 con 0x5601492e77b0 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc97affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc97affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492eeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41208/0 (socket says 172.21.15.42:41208) 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.049+0000 7fc97affd640 1 -- 172.21.15.42:0/4007006975 learned_addr learned my addr 172.21.15.42:0/4007006975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97affd640 1 -- 172.21.15.42:0/4007006975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 msgr2=0x5601492e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97affd640 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97affd640 1 -- 172.21.15.42:0/4007006975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 msgr2=0x5601492ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97affd640 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97affd640 1 -- 172.21.15.42:0/4007006975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601491e0ac0 con 0x5601492debd0 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc97affd640 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492eeba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0fe770 tx=0x7fc96c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90142f4bfa3d6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc9717fa640 1 -- 172.21.15.42:0/4007006975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c01e050 con 0x5601492debd0 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc9717fa640 1 -- 172.21.15.42:0/4007006975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc96c01f040 con 0x5601492debd0 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc9717fa640 1 -- 172.21.15.42:0/4007006975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c023400 con 0x5601492debd0 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc984775740 1 -- 172.21.15.42:0/4007006975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 msgr2=0x5601492eeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.050+0000 7fc984775740 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492eeba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0fe770 tx=0x7fc96c015000 comp rx=0 tx=0).stop 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- 172.21.15.42:0/4007006975 shutdown_connections 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492eeba0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 --2- 172.21.15.42:0/4007006975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- 172.21.15.42:0/4007006975 >> 172.21.15.42:0/4007006975 conn(0x5601492d8990 msgr2=0x5601492fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- 172.21.15.42:0/4007006975 shutdown_connections 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- 172.21.15.42:0/4007006975 wait complete. 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- start start 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 Processor -- start 2024-03-12T04:18:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492ed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601492edba0 con 0x5601492debd0 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601492edd10 con 0x5601492de2b0 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014934d8c0 con 0x5601492e77b0 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc984775740 10 client.0 _ll_register_callbacks cb 0x7fc974018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc97affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc97affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492ecea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41210/0 (socket says 172.21.15.42:41210) 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc97affd640 1 -- 172.21.15.42:0/104240190 learned_addr learned my addr 172.21.15.42:0/104240190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.051+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97bfff640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492ed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97affd640 1 -- 172.21.15.42:0/104240190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 msgr2=0x5601492ed4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97affd640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492ed4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97affd640 1 -- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 msgr2=0x5601492eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97affd640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97affd640 1 -- 172.21.15.42:0/104240190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c00f9a0 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97bfff640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492ed4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc97affd640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492ecea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006e10 tx=0x7fc96c0176b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a3c7aeee86a083 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc95affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc95affd640 1 -- 172.21.15.42:0/104240190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c023600 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc95affd640 1 -- 172.21.15.42:0/104240190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc96c023760 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc97400d750 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc95affd640 1 -- 172.21.15.42:0/104240190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc96c01e020 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc984775740 10 client.16341 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.052+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5601492066a0 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc95affd640 1 -- 172.21.15.42:0/104240190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc97400d750 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc95affd640 1 -- 172.21.15.42:0/104240190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc96c03f880 con 0x5601492debd0 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc984775740 10 client.16341 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc984775740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc984775740 20 client.16341 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc984775740 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56014934efd0 0x5601493513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc984775740 4 client.16341 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560148fd1890 con 0x56014934efd0 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56014934efd0 0x5601493513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc95affd640 10 client.16341 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.053+0000 7fc97b7fe640 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56014934efd0 0x5601493513f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e70c6756228d586 server_cookie=76e62d68a7c79d8f in_seq=0 out_seq=0 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc95affd640 1 -- 172.21.15.42:0/104240190 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560148fd1890 con 0x56014934efd0 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc95affd640 10 client.16341 handle_command_reply: tid=0 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 client.16341 shutdown 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 msgr2=0x5601492ecea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492ecea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006e10 tx=0x7fc96c0176b0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 shutdown_connections 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56014934efd0 0x5601493513f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601492e77b0 0x5601492ed4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc95affd640 0 client.16341 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601492debd0 0x5601492ecea0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 --2- 172.21.15.42:0/104240190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601492de2b0 0x5601492eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 >> 172.21.15.42:0/104240190 conn(0x5601492d8990 msgr2=0x5601492e3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 shutdown_connections 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 1 -- 172.21.15.42:0/104240190 wait complete. 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc984775740 20 client.16341 trim_cache size 0 max 16384 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:18:18.063 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:18:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc9680a1ba0 msgr2=0x7fc9680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc9680a1ba0 0x7fc9680a4060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5601492ddd80 tx=0x7fc96c009340 comp rx=0 tx=0).stop 2024-03-12T04:18:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 msgr2=0x7fc97c19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c19bf20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc97400cce0 tx=0x7fc97400a620 comp rx=0 tx=0).stop 2024-03-12T04:18:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 shutdown_connections 2024-03-12T04:18:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc9680a1ba0 0x7fc9680a4060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97c106110 0x7fc97c19bf20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc97c100c90 0x7fc97c197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 --2- 172.21.15.42:0/212893986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc97c1002d0 0x7fc97c197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.054+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 >> 172.21.15.42:0/212893986 conn(0x7fc97c076e20 msgr2=0x7fc97c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.055+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 shutdown_connections 2024-03-12T04:18:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.055+0000 7fc981ac3640 1 -- 172.21.15.42:0/212893986 wait complete. 2024-03-12T04:18:18.068 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:18:18.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:18:18.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 -- 172.21.15.42:0/1434383925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01002f0 msgr2=0x7f23b01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 --2- 172.21.15.42:0/1434383925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01002f0 0x7f23b01006d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009930 tx=0x7f23a002bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 -- 172.21.15.42:0/1434383925 shutdown_connections 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 --2- 172.21.15.42:0/1434383925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b01053c0 0x7f23b010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 --2- 172.21.15.42:0/1434383925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b0100ca0 0x7f23b0104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 --2- 172.21.15.42:0/1434383925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01002f0 0x7f23b01006d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 -- 172.21.15.42:0/1434383925 >> 172.21.15.42:0/1434383925 conn(0x7f23b0077180 msgr2=0x7f23b0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 -- 172.21.15.42:0/1434383925 shutdown_connections 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.168+0000 7f23b6081640 1 -- 172.21.15.42:0/1434383925 wait complete. 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 Processor -- start 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 -- start start 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b01002f0 0x7f23b01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01053c0 0x7f23b0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b01990c0 0x7f23b01a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b01a3d10 con 0x7f23b01053c0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b01a3e80 con 0x7f23b01002f0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23b6081640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b01a4040 con 0x7f23b01990c0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23aeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01053c0 0x7f23b0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23aeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01053c0 0x7f23b0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41228/0 (socket says 172.21.15.42:41228) 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.169+0000 7f23aeffd640 1 -- 172.21.15.42:0/3150190032 learned_addr learned my addr 172.21.15.42:0/3150190032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23af7fe640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b01002f0 0x7f23b01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23affff640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b01990c0 0x7f23b01a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23aeffd640 1 -- 172.21.15.42:0/3150190032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b01990c0 msgr2=0x7f23b01a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23aeffd640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b01990c0 0x7f23b01a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23aeffd640 1 -- 172.21.15.42:0/3150190032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b01002f0 msgr2=0x7f23b01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23aeffd640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b01002f0 0x7f23b01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23aeffd640 1 -- 172.21.15.42:0/3150190032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0009560 con 0x7f23b01053c0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23aeffd640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01053c0 0x7f23b0198b30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f23a400d740 tx=0x7f23a400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23acff9640 1 -- 172.21.15.42:0/3150190032 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4009c30 con 0x7f23b01053c0 2024-03-12T04:18:18.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23acff9640 1 -- 172.21.15.42:0/3150190032 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23a400de50 con 0x7f23b01053c0 2024-03-12T04:18:18.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b01a4320 con 0x7f23b01053c0 2024-03-12T04:18:18.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23acff9640 1 -- 172.21.15.42:0/3150190032 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4004e40 con 0x7f23b01053c0 2024-03-12T04:18:18.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.170+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b01a47e0 con 0x7f23b01053c0 2024-03-12T04:18:18.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.171+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b0067f70 con 0x7f23b01053c0 2024-03-12T04:18:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.172+0000 7f23acff9640 1 -- 172.21.15.42:0/3150190032 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f23a4004fa0 con 0x7f23b01053c0 2024-03-12T04:18:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.173+0000 7f23acff9640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f23980a1ba0 0x7f23980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.173+0000 7f23af7fe640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f23980a1ba0 0x7f23980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.173+0000 7f23acff9640 1 -- 172.21.15.42:0/3150190032 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f23a40cd290 con 0x7f23b01053c0 2024-03-12T04:18:18.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.173+0000 7f23af7fe640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f23980a1ba0 0x7f23980a4060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23a0006010 tx=0x7f23a0000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.177+0000 7f23acff9640 1 -- 172.21.15.42:0/3150190032 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23a4099c80 con 0x7f23b01053c0 2024-03-12T04:18:18.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.396+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23b01085f0 con 0x7f23b01053c0 2024-03-12T04:18:18.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.396+0000 7f23acff9640 1 -- 172.21.15.42:0/3150190032 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f23a4020090 con 0x7f23b01053c0 2024-03-12T04:18:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:18:18.399 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:18:18.400 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:18:18.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f23980a1ba0 msgr2=0x7f23980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f23980a1ba0 0x7f23980a4060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23a0006010 tx=0x7f23a0000bc0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01053c0 msgr2=0x7f23b0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01053c0 0x7f23b0198b30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f23a400d740 tx=0x7f23a400dc00 comp rx=0 tx=0).stop 2024-03-12T04:18:18.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 shutdown_connections 2024-03-12T04:18:18.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f23980a1ba0 0x7f23980a4060 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b01990c0 0x7f23b01a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b01053c0 0x7f23b0198b30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 --2- 172.21.15.42:0/3150190032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b01002f0 0x7f23b01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.398+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 >> 172.21.15.42:0/3150190032 conn(0x7f23b0077180 msgr2=0x7f23b00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.399+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 shutdown_connections 2024-03-12T04:18:18.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.399+0000 7f23b6081640 1 -- 172.21.15.42:0/3150190032 wait complete. 2024-03-12T04:18:18.412 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:18.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 -- 172.21.15.42:0/438011570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 msgr2=0x7f2198100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 --2- 172.21.15.42:0/438011570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198100640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2190009930 tx=0x7f219002bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 -- 172.21.15.42:0/438011570 shutdown_connections 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 --2- 172.21.15.42:0/438011570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2198105330 0x7f219810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 --2- 172.21.15.42:0/438011570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2198100c10 0x7f2198104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 --2- 172.21.15.42:0/438011570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198100640 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 -- 172.21.15.42:0/438011570 >> 172.21.15.42:0/438011570 conn(0x7f2198077180 msgr2=0x7f2198077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 -- 172.21.15.42:0/438011570 shutdown_connections 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.515+0000 7f219c80e640 1 -- 172.21.15.42:0/438011570 wait complete. 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 Processor -- start 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 -- start start 2024-03-12T04:18:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2198105330 0x7f2198197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f219819c010 0x7f219819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219819ca00 con 0x7f2198100260 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219819cb70 con 0x7f2198105330 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219c80e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219819cce0 con 0x7f219819c010 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41252/0 (socket says 172.21.15.42:41252) 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 -- 172.21.15.42:0/4217210581 learned_addr learned my addr 172.21.15.42:0/4217210581 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f2196d9d640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f219819c010 0x7f219819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f2195d9b640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2198105330 0x7f2198197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 -- 172.21.15.42:0/4217210581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f219819c010 msgr2=0x7f219819c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f219819c010 0x7f219819c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 -- 172.21.15.42:0/4217210581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2198105330 msgr2=0x7f2198197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2198105330 0x7f2198197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 -- 172.21.15.42:0/4217210581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2190009560 con 0x7f2198100260 2024-03-12T04:18:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f2195d9b640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2198105330 0x7f2198197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.516+0000 7f219659c640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198197750 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2190009230 tx=0x7f21900040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.517+0000 7f218f7fe640 1 -- 172.21.15.42:0/4217210581 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21900044c0 con 0x7f2198100260 2024-03-12T04:18:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.517+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21981a42c0 con 0x7f2198100260 2024-03-12T04:18:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.517+0000 7f218f7fe640 1 -- 172.21.15.42:0/4217210581 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f219002de90 con 0x7f2198100260 2024-03-12T04:18:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.517+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21981a4740 con 0x7f2198100260 2024-03-12T04:18:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.517+0000 7f218f7fe640 1 -- 172.21.15.42:0/4217210581 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21900053a0 con 0x7f2198100260 2024-03-12T04:18:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.519+0000 7f218f7fe640 1 -- 172.21.15.42:0/4217210581 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f2190005500 con 0x7f2198100260 2024-03-12T04:18:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.519+0000 7f219f3dd740 1 Processor -- start 2024-03-12T04:18:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219f3dd740 1 -- start start 2024-03-12T04:18:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464250 0x558309464630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55830946d750 0x55830946fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219f3dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830941f960 con 0x558309464250 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219f3dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583093d3310 con 0x558309464b70 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219f3dd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583093d0840 con 0x55830946d750 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464250 0x558309464630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464250 0x558309464630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41266/0 (socket says 172.21.15.42:41266) 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 -- 172.21.15.42:0/2982236545 learned_addr learned my addr 172.21.15.42:0/2982236545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219659c640 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f218f7fe640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f21840a1ba0 0x7f21840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2195d9b640 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55830946d750 0x55830946fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 -- 172.21.15.42:0/2982236545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55830946d750 msgr2=0x55830946fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55830946d750 0x55830946fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 -- 172.21.15.42:0/2982236545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 msgr2=0x55830946d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 -- 172.21.15.42:0/2982236545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55830938e060 con 0x558309464250 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f219659c640 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f218f7fe640 1 -- 172.21.15.42:0/4217210581 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f21900ef7a0 con 0x7f2198100260 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.520+0000 7f2196d9d640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f21840a1ba0 0x7f21840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.521+0000 7f2196d9d640 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464250 0x558309464630 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f21880104a0 tx=0x7f2188010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d571fc75098bac server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.521+0000 7f218cff9640 1 -- 172.21.15.42:0/2982236545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188022070 con 0x558309464250 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.521+0000 7f218cff9640 1 -- 172.21.15.42:0/2982236545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2188013700 con 0x558309464250 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.521+0000 7f218cff9640 1 -- 172.21.15.42:0/2982236545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188011bc0 con 0x558309464250 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.521+0000 7f219f3dd740 1 -- 172.21.15.42:0/2982236545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464250 msgr2=0x558309464630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.521+0000 7f2196d9d640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f21840a1ba0 0x7f21840a4060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2188021000 tx=0x7f218801e000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.521+0000 7f219f3dd740 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464250 0x558309464630 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f21880104a0 tx=0x7f2188010960 comp rx=0 tx=0).stop 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- 172.21.15.42:0/2982236545 shutdown_connections 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55830946d750 0x55830946fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 --2- 172.21.15.42:0/2982236545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464250 0x558309464630 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- 172.21.15.42:0/2982236545 >> 172.21.15.42:0/2982236545 conn(0x55830945e980 msgr2=0x55830945ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- 172.21.15.42:0/2982236545 shutdown_connections 2024-03-12T04:18:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- 172.21.15.42:0/2982236545 wait complete. 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- start start 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 Processor -- start 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464b70 0x55830946ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55830946d750 0x55830946b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583093cc020 con 0x558309464b70 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830909f430 con 0x55830946d750 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830905be00 con 0x558309464250 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f219f3dd740 10 client.0 _ll_register_callbacks cb 0x558309373cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f2195d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464b70 0x55830946ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f2195d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464b70 0x55830946ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41278/0 (socket says 172.21.15.42:41278) 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.522+0000 7f2195d9b640 1 -- 172.21.15.42:0/1417106463 learned_addr learned my addr 172.21.15.42:0/1417106463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f2196d9d640 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55830946d750 0x55830946b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f219659c640 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f2195d9b640 1 -- 172.21.15.42:0/1417106463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 msgr2=0x55830946a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f2195d9b640 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f2195d9b640 1 -- 172.21.15.42:0/1417106463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55830946d750 msgr2=0x55830946b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f2195d9b640 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55830946d750 0x55830946b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f2195d9b640 1 -- 172.21.15.42:0/1417106463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2188009c40 con 0x558309464b70 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f2195d9b640 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464b70 0x55830946ac80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f218000bec0 tx=0x7f2180012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b34613d4ae808d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f217dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f217dffb640 1 -- 172.21.15.42:0/1417106463 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180010960 con 0x558309464b70 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f217dffb640 1 -- 172.21.15.42:0/1417106463 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2180010ac0 con 0x558309464b70 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558309108720 con 0x558309464b70 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f217dffb640 1 -- 172.21.15.42:0/1417106463 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2180010c80 con 0x558309464b70 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f219f3dd740 10 client.16359 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5583093ca6a0 con 0x558309464b70 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.523+0000 7f217dffb640 1 -- 172.21.15.42:0/1417106463 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558309108720 con 0x558309464b70 2024-03-12T04:18:18.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f217dffb640 1 -- 172.21.15.42:0/1417106463 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f218001a3e0 con 0x558309464b70 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219f3dd740 10 client.16359 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219f3dd740 10 client.16359 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219f3dd740 20 client.16359 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55830947dc50 0x558309480070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219f3dd740 4 client.16359 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558309157890 con 0x55830947dc50 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219659c640 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55830947dc50 0x558309480070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f217dffb640 10 client.16359 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f219659c640 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55830947dc50 0x558309480070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f8be99576fad0ca server_cookie=cf099a65c2cbf6b2 in_seq=0 out_seq=0 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f217dffb640 1 -- 172.21.15.42:0/1417106463 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558309157890 con 0x55830947dc50 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.524+0000 7f217dffb640 10 client.16359 handle_command_reply: tid=0 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 client.16359 shutdown 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464b70 msgr2=0x55830946ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464b70 0x55830946ac80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f218000bec0 tx=0x7f2180012820 comp rx=0 tx=0).stop 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 shutdown_connections 2024-03-12T04:18:18.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55830946d750 0x55830946b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55830947dc50 0x558309480070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558309464b70 0x55830946ac80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f217dffb640 0 client.16359 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1417106463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 >> 172.21.15.42:0/1417106463 conn(0x55830945e980 msgr2=0x5583094606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 shutdown_connections 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 1 -- 172.21.15.42:0/1417106463 wait complete. 2024-03-12T04:18:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.525+0000 7f219f3dd740 20 client.16359 trim_cache size 0 max 16384 2024-03-12T04:18:18.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.536+0000 7f219f3dd740 1 Processor -- start 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.536+0000 7f219f3dd740 1 -- start start 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x5583094679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f219f3dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558309351130 con 0x55830946d750 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f219f3dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558309463eb0 con 0x558309464b70 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f219f3dd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558309466370 con 0x558309464250 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x5583094679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x5583094679a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41292/0 (socket says 172.21.15.42:41292) 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 -- 172.21.15.42:0/8000913 learned_addr learned my addr 172.21.15.42:0/8000913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f219659c640 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 -- 172.21.15.42:0/8000913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 msgr2=0x55830946ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 -- 172.21.15.42:0/8000913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 msgr2=0x55830946afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2195d9b640 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 -- 172.21.15.42:0/8000913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558309366c20 con 0x55830946d750 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2195d9b640 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f2196d9d640 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x5583094679a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2188011f10 tx=0x7f2188011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33ab9f8bc28ad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f217ffff640 1 -- 172.21.15.42:0/8000913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218802c070 con 0x55830946d750 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f217ffff640 1 -- 172.21.15.42:0/8000913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2188023040 con 0x55830946d750 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.537+0000 7f217ffff640 1 -- 172.21.15.42:0/8000913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188022040 con 0x55830946d750 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- 172.21.15.42:0/8000913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 msgr2=0x5583094679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x5583094679a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2188011f10 tx=0x7f2188011f40 comp rx=0 tx=0).stop 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- 172.21.15.42:0/8000913 shutdown_connections 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x5583094679a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x55830946afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 --2- 172.21.15.42:0/8000913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x55830946ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- 172.21.15.42:0/8000913 >> 172.21.15.42:0/8000913 conn(0x55830945e980 msgr2=0x5583094849d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- 172.21.15.42:0/8000913 shutdown_connections 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- 172.21.15.42:0/8000913 wait complete. 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- start start 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 Processor -- start 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x5583094cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x5583094cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x558309473820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558309473ef0 con 0x55830946d750 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558309474060 con 0x558309464b70 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.538+0000 7f219f3dd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558309474220 con 0x558309464250 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f219f3dd740 10 client.0 _ll_register_callbacks cb 0x7f2188009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x558309473820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x558309473820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41294/0 (socket says 172.21.15.42:41294) 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 -- 172.21.15.42:0/1909469906 learned_addr learned my addr 172.21.15.42:0/1909469906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f219659c640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x5583094cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 -- 172.21.15.42:0/1909469906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 msgr2=0x5583094cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2195d9b640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x5583094cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x5583094cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 -- 172.21.15.42:0/1909469906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 msgr2=0x5583094cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x5583094cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 -- 172.21.15.42:0/1909469906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55830938e060 con 0x55830946d750 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f219659c640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x5583094cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f2196d9d640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x558309473820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2188009910 tx=0x7f218801dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2160653c79ab9ce0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f217dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f217dffb640 1 -- 172.21.15.42:0/1909469906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218802c030 con 0x55830946d750 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f217dffb640 1 -- 172.21.15.42:0/1909469906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2188011640 con 0x55830946d750 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583090cb7e0 con 0x55830946d750 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f217dffb640 1 -- 172.21.15.42:0/1909469906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2188013160 con 0x55830946d750 2024-03-12T04:18:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f219f3dd740 10 client.16365 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.539+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55830938c8c0 con 0x55830946d750 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f217dffb640 1 -- 172.21.15.42:0/1909469906 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2188046080 con 0x55830946d750 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f217dffb640 1 -- 172.21.15.42:0/1909469906 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2188027c30 con 0x55830946d750 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 10 client.16365 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 10 client.16365 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 20 client.16365 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5583094d6020 0x5583094d8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 4 client.16365 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558309077700 con 0x5583094d6020 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219659c640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5583094d6020 0x5583094d8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f217dffb640 10 client.16365 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219659c640 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5583094d6020 0x5583094d8410 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba93174933722451 server_cookie=e1753d5f2a4f13dc in_seq=0 out_seq=0 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f217dffb640 1 -- 172.21.15.42:0/1909469906 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558309077700 con 0x5583094d6020 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f217dffb640 10 client.16365 handle_command_reply: tid=0 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 client.16365 shutdown 2024-03-12T04:18:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 msgr2=0x558309473820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x558309473820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2188009910 tx=0x7f218801dac0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 shutdown_connections 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5583094d6020 0x5583094d8410 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55830946d750 0x558309473820 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558309464b70 0x5583094cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 --2- 172.21.15.42:0/1909469906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558309464250 0x5583094cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f217dffb640 0 client.16365 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 >> 172.21.15.42:0/1909469906 conn(0x55830945e980 msgr2=0x55830945d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 shutdown_connections 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.540+0000 7f219f3dd740 1 -- 172.21.15.42:0/1909469906 wait complete. 2024-03-12T04:18:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219f3dd740 20 client.16365 trim_cache size 0 max 16384 2024-03-12T04:18:18.551 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f21840a1ba0 msgr2=0x7f21840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f21840a1ba0 0x7f21840a4060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2188021000 tx=0x7f218801e000 comp rx=0 tx=0).stop 2024-03-12T04:18:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 msgr2=0x7f2198197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198197750 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2190009230 tx=0x7f21900040c0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 shutdown_connections 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f21840a1ba0 0x7f21840a4060 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f219819c010 0x7f219819c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2198105330 0x7f2198197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 --2- 172.21.15.42:0/4217210581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2198100260 0x7f2198197750 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 >> 172.21.15.42:0/4217210581 conn(0x7f2198077180 msgr2=0x7f21980687c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 shutdown_connections 2024-03-12T04:18:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.541+0000 7f219c80e640 1 -- 172.21.15.42:0/4217210581 wait complete. 2024-03-12T04:18:18.554 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:18.554 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:18:18.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 -- 172.21.15.42:0/2486898276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc10b7c0 msgr2=0x7f74dc10dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 --2- 172.21.15.42:0/2486898276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc10b7c0 0x7f74dc10dc30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009ca0 tx=0x7f74d402c180 comp rx=0 tx=0).stop 2024-03-12T04:18:18.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 -- 172.21.15.42:0/2486898276 shutdown_connections 2024-03-12T04:18:18.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 --2- 172.21.15.42:0/2486898276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc10b7c0 0x7f74dc10dc30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 --2- 172.21.15.42:0/2486898276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74dc102d10 0x7f74dc10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 --2- 172.21.15.42:0/2486898276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74dc1023f0 0x7f74dc1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 -- 172.21.15.42:0/2486898276 >> 172.21.15.42:0/2486898276 conn(0x7f74dc0771e0 msgr2=0x7f74dc0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 -- 172.21.15.42:0/2486898276 shutdown_connections 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.656+0000 7f74e308d640 1 -- 172.21.15.42:0/2486898276 wait complete. 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 Processor -- start 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 -- start start 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74dc1023f0 0x7f74dc1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc102d10 0x7f74dc197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74dc10b7c0 0x7f74dc19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc19c5a0 con 0x7f74dc102d10 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc19c710 con 0x7f74dc10b7c0 2024-03-12T04:18:18.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e308d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc19c8d0 con 0x7f74dc1023f0 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc102d10 0x7f74dc197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc102d10 0x7f74dc197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41312/0 (socket says 172.21.15.42:41312) 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74d3fff640 1 -- 172.21.15.42:0/1223251220 learned_addr learned my addr 172.21.15.42:0/1223251220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e162a640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74dc10b7c0 0x7f74dc19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.657+0000 7f74e0e29640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74dc1023f0 0x7f74dc1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d3fff640 1 -- 172.21.15.42:0/1223251220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74dc1023f0 msgr2=0x7f74dc1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74dc1023f0 0x7f74dc1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d3fff640 1 -- 172.21.15.42:0/1223251220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74dc10b7c0 msgr2=0x7f74dc19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74dc10b7c0 0x7f74dc19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d3fff640 1 -- 172.21.15.42:0/1223251220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4009900 con 0x7f74dc102d10 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc102d10 0x7f74dc197c20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00d950 tx=0x7f74cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d1ffb640 1 -- 172.21.15.42:0/1223251220 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc009c30 con 0x7f74dc102d10 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d1ffb640 1 -- 172.21.15.42:0/1223251220 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74cc009d90 con 0x7f74dc102d10 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc19cbb0 con 0x7f74dc102d10 2024-03-12T04:18:18.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74d1ffb640 1 -- 172.21.15.42:0/1223251220 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00f6b0 con 0x7f74dc102d10 2024-03-12T04:18:18.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.658+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc1a4430 con 0x7f74dc102d10 2024-03-12T04:18:18.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.660+0000 7f74d1ffb640 1 -- 172.21.15.42:0/1223251220 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f74cc010aa0 con 0x7f74dc102d10 2024-03-12T04:18:18.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 Processor -- start 2024-03-12T04:18:18.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74d1ffb640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f74c40a1ba0 0x7f74c40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 -- start start 2024-03-12T04:18:18.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f74c40a1ba0 0x7f74c40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c4c250 0x5648a8c4c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74d1ffb640 1 -- 172.21.15.42:0/1223251220 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f74cc0cd440 con 0x7f74dc102d10 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c55210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c55750 0x5648a8c57b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8c07960 con 0x5648a8c55750 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8bbb310 con 0x5648a8c4cb70 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8bb8840 con 0x5648a8c4c250 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c55750 0x5648a8c57b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c55750 0x5648a8c57b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41326/0 (socket says 172.21.15.42:41326) 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 -- 172.21.15.42:0/393087294 learned_addr learned my addr 172.21.15.42:0/393087294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e0e29640 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c4c250 0x5648a8c4c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74d3fff640 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c55210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f74c40a1ba0 0x7f74c40a4060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74d4032620 tx=0x7f74d4034e30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 -- 172.21.15.42:0/393087294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c4c250 msgr2=0x5648a8c4c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c4c250 0x5648a8c4c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 -- 172.21.15.42:0/393087294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 msgr2=0x5648a8c55210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c55210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.661+0000 7f74e162a640 1 -- 172.21.15.42:0/393087294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a8b76060 con 0x5648a8c55750 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e162a640 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c55750 0x5648a8c57b40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009ca0 tx=0x7f74d4049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721b27cbe138a242 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74c2ffd640 1 -- 172.21.15.42:0/393087294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4003b70 con 0x5648a8c55750 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74c2ffd640 1 -- 172.21.15.42:0/393087294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74d40348d0 con 0x5648a8c55750 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74c2ffd640 1 -- 172.21.15.42:0/393087294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004290 con 0x5648a8c55750 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 -- 172.21.15.42:0/393087294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c55750 msgr2=0x5648a8c57b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c55750 0x5648a8c57b40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009ca0 tx=0x7f74d4049000 comp rx=0 tx=0).stop 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 -- 172.21.15.42:0/393087294 shutdown_connections 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c55750 0x5648a8c57b40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c55210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/393087294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c4c250 0x5648a8c4c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 -- 172.21.15.42:0/393087294 >> 172.21.15.42:0/393087294 conn(0x5648a8c46980 msgr2=0x5648a8c46d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.662+0000 7f74e5c5c740 1 -- 172.21.15.42:0/393087294 shutdown_connections 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 -- 172.21.15.42:0/393087294 wait complete. 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 -- start start 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 Processor -- start 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4c250 0x5648a8c52710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4cb70 0x5648a8c52c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c53190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8bb4020 con 0x5648a8c4cb70 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8887430 con 0x5648a8c4c250 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8843e00 con 0x5648a8c55750 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74e5c5c740 10 client.0 _ll_register_callbacks cb 0x5648a8b5bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4cb70 0x5648a8c52c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4cb70 0x5648a8c52c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41340/0 (socket says 172.21.15.42:41340) 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.663+0000 7f74d3fff640 1 -- 172.21.15.42:0/213811896 learned_addr learned my addr 172.21.15.42:0/213811896 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e0e29640 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4c250 0x5648a8c52710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74d3fff640 1 -- 172.21.15.42:0/213811896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 msgr2=0x5648a8c53190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74d3fff640 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c53190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74d3fff640 1 -- 172.21.15.42:0/213811896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4c250 msgr2=0x5648a8c52710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74d3fff640 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4c250 0x5648a8c52710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74d3fff640 1 -- 172.21.15.42:0/213811896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d4049070 con 0x5648a8c4cb70 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e0e29640 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4c250 0x5648a8c52710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74d3fff640 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4cb70 0x5648a8c52c50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f74cc099d60 tx=0x7f74cc09a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854b6e25a12d5b9c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74bbfff640 1 -- 172.21.15.42:0/213811896 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0a2370 con 0x5648a8c4cb70 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74bbfff640 1 -- 172.21.15.42:0/213811896 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f74cc0a24d0 con 0x5648a8c4cb70 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a88f0720 con 0x5648a8c4cb70 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74bbfff640 1 -- 172.21.15.42:0/213811896 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74cc0b4ea0 con 0x5648a8c4cb70 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 10 client.16377 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648a8bb26a0 con 0x5648a8c4cb70 2024-03-12T04:18:18.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74bbfff640 1 -- 172.21.15.42:0/213811896 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74cc0b8020 con 0x5648a8c4cb70 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74bbfff640 1 -- 172.21.15.42:0/213811896 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f74cc098f20 con 0x5648a8c4cb70 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 10 client.16377 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 10 client.16377 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 20 client.16377 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8c65ae0 0x5648a8c67f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 4 client.16377 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.664+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648a893f890 con 0x5648a8c65ae0 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.665+0000 7f74e0e29640 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8c65ae0 0x5648a8c67f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.665+0000 7f74e0e29640 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8c65ae0 0x5648a8c67f10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f43b88210fe2ce3 server_cookie=641996e7879fc457 in_seq=0 out_seq=0 2024-03-12T04:18:18.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.665+0000 7f74bbfff640 10 client.16377 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74bbfff640 1 -- 172.21.15.42:0/213811896 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648a893f890 con 0x5648a8c65ae0 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74bbfff640 10 client.16377 handle_command_reply: tid=0 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 client.16377 shutdown 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4cb70 msgr2=0x5648a8c52c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4cb70 0x5648a8c52c50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f74cc099d60 tx=0x7f74cc09a1a0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 shutdown_connections 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8c65ae0 0x5648a8c67f10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c53190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4cb70 0x5648a8c52c50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/213811896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4c250 0x5648a8c52710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74bbfff640 0 client.16377 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:18.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 >> 172.21.15.42:0/213811896 conn(0x5648a8c46980 msgr2=0x5648a8c486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 shutdown_connections 2024-03-12T04:18:18.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 1 -- 172.21.15.42:0/213811896 wait complete. 2024-03-12T04:18:18.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.666+0000 7f74e5c5c740 20 client.16377 trim_cache size 0 max 16384 2024-03-12T04:18:18.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 Processor -- start 2024-03-12T04:18:18.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 -- start start 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8c54e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c52fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c4f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8b39130 con 0x5648a8c4c250 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8c47b40 con 0x5648a8c4cb70 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8c42080 con 0x5648a8c55750 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e0e29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8c54e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e0e29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8c54e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41348/0 (socket says 172.21.15.42:41348) 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.678+0000 7f74e0e29640 1 -- 172.21.15.42:0/1644463601 learned_addr learned my addr 172.21.15.42:0/1644463601 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c52fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74e0e29640 1 -- 172.21.15.42:0/1644463601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 msgr2=0x5648a8c4f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74e162a640 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c4f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74e0e29640 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c4f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74e0e29640 1 -- 172.21.15.42:0/1644463601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 msgr2=0x5648a8c52fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74e0e29640 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c52fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74e0e29640 1 -- 172.21.15.42:0/1644463601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a8b4ec20 con 0x5648a8c4c250 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c52fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74e0e29640 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8c54e40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x5648a8c68830 tx=0x7f74d800b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46de88ce5c92c9e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74c27fc640 1 -- 172.21.15.42:0/1644463601 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8018040 con 0x5648a8c4c250 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74c27fc640 1 -- 172.21.15.42:0/1644463601 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74d8014e90 con 0x5648a8c4c250 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.679+0000 7f74c27fc640 1 -- 172.21.15.42:0/1644463601 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80126b0 con 0x5648a8c4c250 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1644463601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 msgr2=0x5648a8c54e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8c54e40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x5648a8c68830 tx=0x7f74d800b4d0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1644463601 shutdown_connections 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c4f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8c52fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1644463601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8c54e40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1644463601 >> 172.21.15.42:0/1644463601 conn(0x5648a8c46980 msgr2=0x5648a8c4da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1644463601 shutdown_connections 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1644463601 wait complete. 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- start start 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 Processor -- start 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8cb6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8cb6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8c5c450 con 0x5648a8c4c250 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8c5c5c0 con 0x5648a8c4cb70 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8c5c780 con 0x5648a8c55750 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e5c5c740 10 client.0 _ll_register_callbacks cb 0x7f74d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e0e29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8cb6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e0e29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8cb6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41350/0 (socket says 172.21.15.42:41350) 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.680+0000 7f74e0e29640 1 -- 172.21.15.42:0/1728281094 learned_addr learned my addr 172.21.15.42:0/1728281094 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e162a640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8cb6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e0e29640 1 -- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 msgr2=0x5648a8c5be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e0e29640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c5be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e0e29640 1 -- 172.21.15.42:0/1728281094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 msgr2=0x5648a8cb6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e0e29640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8cb6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e0e29640 1 -- 172.21.15.42:0/1728281094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648a8b748c0 con 0x5648a8c4c250 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e162a640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8cb6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e0e29640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8cb6a20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f74d8000c00 tx=0x7f74d80142a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3882a1512e501353 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74bbfff640 1 -- 172.21.15.42:0/1728281094 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8014c60 con 0x5648a8c4c250 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74bbfff640 1 -- 172.21.15.42:0/1728281094 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f74d801a2b0 con 0x5648a8c4c250 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a88b37e0 con 0x5648a8c4c250 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74bbfff640 1 -- 172.21.15.42:0/1728281094 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d801a470 con 0x5648a8c4c250 2024-03-12T04:18:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e5c5c740 10 client.16383 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648a8b5b050 con 0x5648a8c4c250 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74bbfff640 1 -- 172.21.15.42:0/1728281094 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5648a88b37e0 con 0x5648a8c4c250 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.681+0000 7f74bbfff640 1 -- 172.21.15.42:0/1728281094 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f74d802c440 con 0x5648a8c4c250 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74e5c5c740 10 client.16383 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74e5c5c740 10 client.16383 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74e5c5c740 20 client.16383 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8cbe8b0 0x5648a8cc0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74e5c5c740 4 client.16383 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5648a885f700 con 0x5648a8cbe8b0 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8cbe8b0 0x5648a8cc0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74d3fff640 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8cbe8b0 0x5648a8cc0cd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2efac535e7685719 server_cookie=b7fef9462945ebce in_seq=0 out_seq=0 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.682+0000 7f74bbfff640 10 client.16383 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74bbfff640 1 -- 172.21.15.42:0/1728281094 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5648a885f700 con 0x5648a8cbe8b0 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74bbfff640 10 client.16383 handle_command_reply: tid=0 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 client.16383 shutdown 2024-03-12T04:18:18.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 msgr2=0x5648a8cb6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8cb6a20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f74d8000c00 tx=0x7f74d80142a0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 shutdown_connections 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5648a8cbe8b0 0x5648a8cc0cd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648a8c55750 0x5648a8c5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648a8c4cb70 0x5648a8cb6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74bbfff640 0 client.16383 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 --2- 172.21.15.42:0/1728281094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5648a8c4c250 0x5648a8cb6a20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 >> 172.21.15.42:0/1728281094 conn(0x5648a8c46980 msgr2=0x5648a8c452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 shutdown_connections 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 1 -- 172.21.15.42:0/1728281094 wait complete. 2024-03-12T04:18:18.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e5c5c740 20 client.16383 trim_cache size 0 max 16384 2024-03-12T04:18:18.693 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:18.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f74c40a1ba0 msgr2=0x7f74c40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f74c40a1ba0 0x7f74c40a4060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74d4032620 tx=0x7f74d4034e30 comp rx=0 tx=0).stop 2024-03-12T04:18:18.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc102d10 msgr2=0x7f74dc197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc102d10 0x7f74dc197c20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00d950 tx=0x7f74cc00de10 comp rx=0 tx=0).stop 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 shutdown_connections 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f74c40a1ba0 0x7f74c40a4060 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74dc10b7c0 0x7f74dc19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.683+0000 7f74e308d640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74dc102d10 0x7f74dc197c20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.684+0000 7f74e308d640 1 --2- 172.21.15.42:0/1223251220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74dc1023f0 0x7f74dc1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.684+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 >> 172.21.15.42:0/1223251220 conn(0x7f74dc0771e0 msgr2=0x7f74dc0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.684+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 shutdown_connections 2024-03-12T04:18:18.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.684+0000 7f74e308d640 1 -- 172.21.15.42:0/1223251220 wait complete. 2024-03-12T04:18:18.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:18 smithi119 ceph-mon[36119]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:18:18.696 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:18 smithi042 ceph-mon[32260]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:18:18.697 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:18.697 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:18:18.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:18 smithi136 ceph-mon[35571]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 -- 172.21.15.42:0/2035595974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd068101380 msgr2=0x7fd068101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 --2- 172.21.15.42:0/2035595974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd068101380 0x7fd068101760 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd058009930 tx=0x7fd05802bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 -- 172.21.15.42:0/2035595974 shutdown_connections 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 --2- 172.21.15.42:0/2035595974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd068106450 0x7fd06810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 --2- 172.21.15.42:0/2035595974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd068101d30 0x7fd068105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 --2- 172.21.15.42:0/2035595974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd068101380 0x7fd068101760 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 -- 172.21.15.42:0/2035595974 >> 172.21.15.42:0/2035595974 conn(0x7fd068078580 msgr2=0x7fd068078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 -- 172.21.15.42:0/2035595974 shutdown_connections 2024-03-12T04:18:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.799+0000 7fd06e423640 1 -- 172.21.15.42:0/2035595974 wait complete. 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 Processor -- start 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 -- start start 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd068101380 0x7fd068197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd068106450 0x7fd068197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06819bff0 0x7fd06819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06819c9e0 con 0x7fd06819bff0 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06819cb50 con 0x7fd068106450 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06e423640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06819ccc0 con 0x7fd068101380 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06819bff0 0x7fd06819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06819bff0 0x7fd06819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41368/0 (socket says 172.21.15.42:41368) 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 -- 172.21.15.42:0/3229697791 learned_addr learned my addr 172.21.15.42:0/3229697791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd067fff640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd068101380 0x7fd068197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd0677fe640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd068106450 0x7fd068197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 -- 172.21.15.42:0/3229697791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd068101380 msgr2=0x7fd068197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd068101380 0x7fd068197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 -- 172.21.15.42:0/3229697791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd068106450 msgr2=0x7fd068197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd068106450 0x7fd068197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd06c9c0640 1 -- 172.21.15.42:0/3229697791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009560 con 0x7fd06819bff0 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.800+0000 7fd0677fe640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd068106450 0x7fd068197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.801+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06819bff0 0x7fd06819c4a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd06000eb10 tx=0x7fd06000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.801+0000 7fd0657fa640 1 -- 172.21.15.42:0/3229697791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060004b20 con 0x7fd06819bff0 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.801+0000 7fd0657fa640 1 -- 172.21.15.42:0/3229697791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd060004c80 con 0x7fd06819bff0 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.801+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0681a42a0 con 0x7fd06819bff0 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.801+0000 7fd0657fa640 1 -- 172.21.15.42:0/3229697791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060010690 con 0x7fd06819bff0 2024-03-12T04:18:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.801+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0681a47a0 con 0x7fd06819bff0 2024-03-12T04:18:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.803+0000 7fd0657fa640 1 -- 172.21.15.42:0/3229697791 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd060005020 con 0x7fd06819bff0 2024-03-12T04:18:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.803+0000 7fd070ff9740 1 Processor -- start 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.803+0000 7fd070ff9740 1 -- start start 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdb3250 0x55e6abdb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3b70 0x55e6abdbc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0657fa640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd0500a1c20 0x7fd0500a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdbc750 0x55e6abdbeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd070ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abd6e960 con 0x55e6abdb3b70 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd070ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abd22310 con 0x55e6abdb3250 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd070ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abd1f840 con 0x55e6abdbc750 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3b70 0x55e6abdbc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd067fff640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd0500a1c20 0x7fd0500a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3b70 0x55e6abdbc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41384/0 (socket says 172.21.15.42:41384) 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 -- 172.21.15.42:0/953674986 learned_addr learned my addr 172.21.15.42:0/953674986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0657fa640 1 -- 172.21.15.42:0/3229697791 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd060014070 con 0x7fd06819bff0 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdbc750 0x55e6abdbeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdb3250 0x55e6abdb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 -- 172.21.15.42:0/953674986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdbc750 msgr2=0x55e6abdbeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdbc750 0x55e6abdbeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 -- 172.21.15.42:0/953674986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdb3250 msgr2=0x55e6abdb3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdb3250 0x55e6abdb3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 -- 172.21.15.42:0/953674986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6abcdd060 con 0x55e6abdb3b70 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd067fff640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd0500a1c20 0x7fd0500a40e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd058006010 tx=0x7fd058000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd0677fe640 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3b70 0x55e6abdbc210 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0124c0 tx=0x7fd05c00b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74b0b48227a8823 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd04e7fc640 1 -- 172.21.15.42:0/953674986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c005020 con 0x55e6abdb3b70 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.804+0000 7fd04e7fc640 1 -- 172.21.15.42:0/953674986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd05c005630 con 0x55e6abdb3b70 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd04e7fc640 1 -- 172.21.15.42:0/953674986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00e660 con 0x55e6abdb3b70 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 -- 172.21.15.42:0/953674986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3b70 msgr2=0x55e6abdbc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3b70 0x55e6abdbc210 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0124c0 tx=0x7fd05c00b5e0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 -- 172.21.15.42:0/953674986 shutdown_connections 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdbc750 0x55e6abdbeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3b70 0x55e6abdbc210 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 --2- 172.21.15.42:0/953674986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdb3250 0x55e6abdb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 -- 172.21.15.42:0/953674986 >> 172.21.15.42:0/953674986 conn(0x55e6abdad980 msgr2=0x55e6abdadd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 -- 172.21.15.42:0/953674986 shutdown_connections 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 -- 172.21.15.42:0/953674986 wait complete. 2024-03-12T04:18:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.805+0000 7fd070ff9740 1 -- start start 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 1 Processor -- start 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdb9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abd1b020 con 0x55e6abdb3250 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab9ee430 con 0x55e6abdbc750 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab9aae00 con 0x55e6abdb3b70 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd070ff9740 10 client.0 _ll_register_callbacks cb 0x55e6abcc2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdb97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41394/0 (socket says 172.21.15.42:41394) 2024-03-12T04:18:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 -- 172.21.15.42:0/3290464360 learned_addr learned my addr 172.21.15.42:0/3290464360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd0677fe640 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdb9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 -- 172.21.15.42:0/3290464360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 msgr2=0x55e6abdb9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdb9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 -- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 msgr2=0x55e6abdba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.806+0000 7fd067fff640 1 -- 172.21.15.42:0/3290464360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c00bcf0 con 0x55e6abdb3250 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd067fff640 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdb97b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd058035000 tx=0x7fd058009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2ca55cdb86c583 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd0477fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd0477fe640 1 -- 172.21.15.42:0/3290464360 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058003a00 con 0x55e6abdb3250 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd0477fe640 1 -- 172.21.15.42:0/3290464360 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd058046050 con 0x55e6abdb3250 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6aba1a7e0 con 0x55e6abdb3250 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd0477fe640 1 -- 172.21.15.42:0/3290464360 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd058003b60 con 0x55e6abdb3250 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 10 client.16395 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6abd196a0 con 0x55e6abdb3250 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd0477fe640 1 -- 172.21.15.42:0/3290464360 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e6aba1a7e0 con 0x55e6abdb3250 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd0477fe640 1 -- 172.21.15.42:0/3290464360 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd05803c040 con 0x55e6abdb3250 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 10 client.16395 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 10 client.16395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 20 client.16395 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abdccbe0 0x55e6abdcf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 4 client.16395 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.807+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6aba57720 con 0x55e6abdccbe0 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.808+0000 7fd0677fe640 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abdccbe0 0x55e6abdcf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.808+0000 7fd0477fe640 10 client.16395 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.808+0000 7fd0677fe640 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abdccbe0 0x55e6abdcf000 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=393258204ba6d8dd server_cookie=b180525f7941715e in_seq=0 out_seq=0 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd0477fe640 1 -- 172.21.15.42:0/3290464360 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6aba57720 con 0x55e6abdccbe0 2024-03-12T04:18:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd0477fe640 10 client.16395 handle_command_reply: tid=0 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 client.16395 shutdown 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 msgr2=0x55e6abdb97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdb97b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd058035000 tx=0x7fd058009260 comp rx=0 tx=0).stop 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 shutdown_connections 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abdccbe0 0x55e6abdcf000 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdb9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 --2- 172.21.15.42:0/3290464360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdb97b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd0477fe640 0 client.16395 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 >> 172.21.15.42:0/3290464360 conn(0x55e6abdad980 msgr2=0x55e6abdaf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 shutdown_connections 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 1 -- 172.21.15.42:0/3290464360 wait complete. 2024-03-12T04:18:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.809+0000 7fd070ff9740 20 client.16395 trim_cache size 0 max 16384 2024-03-12T04:18:18.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 Processor -- start 2024-03-12T04:18:18.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 -- start start 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdaca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abca0130 con 0x55e6abdb3250 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abdae780 con 0x55e6abdbc750 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd070ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abdc96e0 con 0x55e6abdb3b70 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd067fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd067fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41410/0 (socket says 172.21.15.42:41410) 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.821+0000 7fd067fff640 1 -- 172.21.15.42:0/1379238203 learned_addr learned my addr 172.21.15.42:0/1379238203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd0677fe640 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdaca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd067fff640 1 -- 172.21.15.42:0/1379238203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 msgr2=0x55e6abdaca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd067fff640 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdaca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd067fff640 1 -- 172.21.15.42:0/1379238203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 msgr2=0x55e6abdc3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd067fff640 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd067fff640 1 -- 172.21.15.42:0/1379238203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6abcb5c20 con 0x55e6abdb3250 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd0677fe640 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdaca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd067fff640 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdba050 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd058009230 tx=0x7fd05802cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73f98fa83adf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd04dffb640 1 -- 172.21.15.42:0/1379238203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580349b0 con 0x55e6abdb3250 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd04dffb640 1 -- 172.21.15.42:0/1379238203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd05802de90 con 0x55e6abdb3250 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.822+0000 7fd04dffb640 1 -- 172.21.15.42:0/1379238203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058004dc0 con 0x55e6abdb3250 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- 172.21.15.42:0/1379238203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 msgr2=0x55e6abdba050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdba050 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd058009230 tx=0x7fd05802cca0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- 172.21.15.42:0/1379238203 shutdown_connections 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdaca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 --2- 172.21.15.42:0/1379238203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdba050 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- 172.21.15.42:0/1379238203 >> 172.21.15.42:0/1379238203 conn(0x55e6abdad980 msgr2=0x55e6abdac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- 172.21.15.42:0/1379238203 shutdown_connections 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- 172.21.15.42:0/1379238203 wait complete. 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- start start 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 Processor -- start 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdc21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdc2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abe27640 con 0x55e6abdb3250 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abe277b0 con 0x55e6abdbc750 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6abe27970 con 0x55e6abdb3b70 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.823+0000 7fd070ff9740 10 client.0 _ll_register_callbacks cb 0x7fd05c00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdc21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdc21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41422/0 (socket says 172.21.15.42:41422) 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 -- 172.21.15.42:0/2319497462 learned_addr learned my addr 172.21.15.42:0/2319497462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd0677fe640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdc2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 -- 172.21.15.42:0/2319497462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 msgr2=0x55e6abdc2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdc2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 -- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 msgr2=0x55e6abdc2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 -- 172.21.15.42:0/2319497462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6abd196a0 con 0x55e6abdb3250 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd06c9c0640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd067fff640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdc21e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd05802c170 tx=0x7fd058003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d510193ad393e718 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd0477fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd0477fe640 1 -- 172.21.15.42:0/2319497462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058004fc0 con 0x55e6abdb3250 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd0477fe640 1 -- 172.21.15.42:0/2319497462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd058046070 con 0x55e6abdb3250 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd05c005630 con 0x55e6abdb3250 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd0477fe640 1 -- 172.21.15.42:0/2319497462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd058003e60 con 0x55e6abdb3250 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd070ff9740 10 client.16401 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6abcdb8c0 con 0x55e6abdb3250 2024-03-12T04:18:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.824+0000 7fd0477fe640 1 -- 172.21.15.42:0/2319497462 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd05c005630 con 0x55e6abdb3250 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd0477fe640 1 -- 172.21.15.42:0/2319497462 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd058065070 con 0x55e6abdb3250 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd070ff9740 10 client.16401 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd070ff9740 10 client.16401 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd070ff9740 20 client.16401 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd070ff9740 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abe28c40 0x55e6abe2b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd070ff9740 4 client.16401 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6abaa6890 con 0x55e6abe28c40 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd0677fe640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abe28c40 0x55e6abe2b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd0477fe640 10 client.16401 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd0677fe640 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abe28c40 0x55e6abe2b030 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5b0748c10a8c026 server_cookie=a86023455dd93ad8 in_seq=0 out_seq=0 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd0477fe640 1 -- 172.21.15.42:0/2319497462 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6abaa6890 con 0x55e6abe28c40 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd0477fe640 10 client.16401 handle_command_reply: tid=0 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.825+0000 7fd070ff9740 1 client.16401 shutdown 2024-03-12T04:18:18.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 msgr2=0x55e6abdc21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdc21e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd05802c170 tx=0x7fd058003a30 comp rx=0 tx=0).stop 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 shutdown_connections 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55e6abe28c40 0x55e6abe2b030 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6abdbc750 0x55e6abdc2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd0477fe640 0 client.16401 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6abdb3b70 0x55e6abdc2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 --2- 172.21.15.42:0/2319497462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6abdb3250 0x55e6abdc21e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 >> 172.21.15.42:0/2319497462 conn(0x55e6abdad980 msgr2=0x55e6abdaf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 shutdown_connections 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 1 -- 172.21.15.42:0/2319497462 wait complete. 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd070ff9740 20 client.16401 trim_cache size 0 max 16384 2024-03-12T04:18:18.835 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd0500a1c20 msgr2=0x7fd0500a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd0500a1c20 0x7fd0500a40e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd058006010 tx=0x7fd058000bc0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06819bff0 msgr2=0x7fd06819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06819bff0 0x7fd06819c4a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd06000eb10 tx=0x7fd06000c2d0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 shutdown_connections 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd0500a1c20 0x7fd0500a40e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06819bff0 0x7fd06819c4a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd068106450 0x7fd068197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 --2- 172.21.15.42:0/3229697791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd068101380 0x7fd068197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 >> 172.21.15.42:0/3229697791 conn(0x7fd068078580 msgr2=0x7fd0680ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 shutdown_connections 2024-03-12T04:18:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.826+0000 7fd06e423640 1 -- 172.21.15.42:0/3229697791 wait complete. 2024-03-12T04:18:18.840 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:18.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:18:18.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 -- 172.21.15.42:0/2408402167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 msgr2=0x7ff13410cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2408402167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff13410cf40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff124009cd0 tx=0x7ff12402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 -- 172.21.15.42:0/2408402167 shutdown_connections 2024-03-12T04:18:18.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2408402167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff13410cf40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2408402167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff134100c50 0x7ff134104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2408402167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1341002a0 0x7ff134100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 -- 172.21.15.42:0/2408402167 >> 172.21.15.42:0/2408402167 conn(0x7ff134076400 msgr2=0x7ff134076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 -- 172.21.15.42:0/2408402167 shutdown_connections 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.941+0000 7ff138ef4640 1 -- 172.21.15.42:0/2408402167 wait complete. 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 Processor -- start 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 -- start start 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1341002a0 0x7ff1341965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff134196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff13419ae70 0x7ff13419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13419b860 con 0x7ff134105370 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13419b9d0 con 0x7ff1341002a0 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff138ef4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13419bb40 con 0x7ff13419ae70 2024-03-12T04:18:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff134196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff134196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41438/0 (socket says 172.21.15.42:41438) 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.942+0000 7ff131d9b640 1 -- 172.21.15.42:0/2227308597 learned_addr learned my addr 172.21.15.42:0/2227308597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff13259c640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1341002a0 0x7ff1341965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff132d9d640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff13419ae70 0x7ff13419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff131d9b640 1 -- 172.21.15.42:0/2227308597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff13419ae70 msgr2=0x7ff13419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff131d9b640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff13419ae70 0x7ff13419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff131d9b640 1 -- 172.21.15.42:0/2227308597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1341002a0 msgr2=0x7ff1341965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff131d9b640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1341002a0 0x7ff1341965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff131d9b640 1 -- 172.21.15.42:0/2227308597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124009900 con 0x7ff134105370 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff132d9d640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff13419ae70 0x7ff13419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff131d9b640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff134196af0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c970 tx=0x7ff11c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff12b7fe640 1 -- 172.21.15.42:0/2227308597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00cea0 con 0x7ff134105370 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1341a3120 con 0x7ff134105370 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff12b7fe640 1 -- 172.21.15.42:0/2227308597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff11c007bb0 con 0x7ff134105370 2024-03-12T04:18:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff12b7fe640 1 -- 172.21.15.42:0/2227308597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c018660 con 0x7ff134105370 2024-03-12T04:18:18.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.943+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1341a35d0 con 0x7ff134105370 2024-03-12T04:18:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.945+0000 7ff12b7fe640 1 -- 172.21.15.42:0/2227308597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7ff11c007d20 con 0x7ff134105370 2024-03-12T04:18:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 Processor -- start 2024-03-12T04:18:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 -- start start 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff12b7fe640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff1200a1ba0 0x7ff1200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace197840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace150960 con 0x563ace197d80 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace104310 con 0x563ace197460 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13bba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace101840 con 0x563ace1a0960 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff132d9d640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff1200a1ba0 0x7ff1200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff12b7fe640 1 -- 172.21.15.42:0/2227308597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff11c0cd8a0 con 0x7ff134105370 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:55732/0 (socket says 172.21.15.42:55732) 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace1a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41442/0 (socket says 172.21.15.42:41442) 2024-03-12T04:18:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.946+0000 7ff13259c640 1 -- 172.21.15.42:0/3795789861 learned_addr learned my addr 172.21.15.42:0/3795789861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff132d9d640 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff13259c640 1 -- 172.21.15.42:0/3795789861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 msgr2=0x563ace1a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff13259c640 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff13259c640 1 -- 172.21.15.42:0/3795789861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 msgr2=0x563ace1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff13259c640 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff13259c640 1 -- 172.21.15.42:0/3795789861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ace0bf060 con 0x563ace197460 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff131d9b640 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff132d9d640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff1200a1ba0 0x7ff1200a4060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff124037040 tx=0x7ff124034520 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff13259c640 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace197840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00ad00 tx=0x7ff12c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0ca5e9c08f727c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff128ff9640 1 -- 172.21.15.42:0/3795789861 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00a800 con 0x563ace197460 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff128ff9640 1 -- 172.21.15.42:0/3795789861 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff12c018af0 con 0x563ace197460 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.947+0000 7ff128ff9640 1 -- 172.21.15.42:0/3795789861 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c010b60 con 0x563ace197460 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 -- 172.21.15.42:0/3795789861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 msgr2=0x563ace197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace197840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00ad00 tx=0x7ff12c00f790 comp rx=0 tx=0).stop 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 -- 172.21.15.42:0/3795789861 shutdown_connections 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3795789861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace197840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 -- 172.21.15.42:0/3795789861 >> 172.21.15.42:0/3795789861 conn(0x563ace191b90 msgr2=0x563ace191f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 -- 172.21.15.42:0/3795789861 shutdown_connections 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 -- 172.21.15.42:0/3795789861 wait complete. 2024-03-12T04:18:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 -- start start 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.948+0000 7ff13bba5740 1 Processor -- start 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197460 0x563ace19d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace19de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace1a0960 0x563ace19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace0fd020 con 0x563ace197d80 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acddd0430 con 0x563ace1a0960 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acdd8ce00 con 0x563ace197460 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 10 client.0 _ll_register_callbacks cb 0x563ace0a4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41448/0 (socket says 172.21.15.42:41448) 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 -- 172.21.15.42:0/1323511643 learned_addr learned my addr 172.21.15.42:0/1323511643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff132d9d640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace1a0960 0x563ace19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13259c640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197460 0x563ace19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 -- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197460 msgr2=0x563ace19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197460 0x563ace19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 -- 172.21.15.42:0/1323511643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace1a0960 msgr2=0x563ace19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace1a0960 0x563ace19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 -- 172.21.15.42:0/1323511643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c00a960 con 0x563ace197d80 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13259c640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197460 0x563ace19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff131d9b640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace19de60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff11c096e90 tx=0x7ff11c09abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df05c5a2ebd0ff5f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff119ffb640 1 -- 172.21.15.42:0/1323511643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0cb460 con 0x563ace197d80 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff119ffb640 1 -- 172.21.15.42:0/1323511643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff11c09a4c0 con 0x563ace197d80 2024-03-12T04:18:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.949+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563acddfc7e0 con 0x563ace197d80 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff119ffb640 1 -- 172.21.15.42:0/1323511643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff11c09e610 con 0x563ace197d80 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 10 client.16413 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ace0fb6a0 con 0x563ace197d80 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff119ffb640 1 -- 172.21.15.42:0/1323511643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x563acddfc7e0 con 0x563ace197d80 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff119ffb640 1 -- 172.21.15.42:0/1323511643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff11c016020 con 0x563ace197d80 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 10 client.16413 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 10 client.16413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 20 client.16413 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace1b0db0 0x563ace1b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 4 client.16413 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:18.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563acde39720 con 0x563ace1b0db0 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.950+0000 7ff13259c640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace1b0db0 0x563ace1b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.951+0000 7ff13259c640 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace1b0db0 0x563ace1b31e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db0c51151f10e1e5 server_cookie=9d38618ccc39c692 in_seq=0 out_seq=0 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.951+0000 7ff119ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.951+0000 7ff119ffb640 1 -- 172.21.15.42:0/1323511643 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563acde39720 con 0x563ace1b0db0 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.951+0000 7ff119ffb640 10 client.16413 handle_command_reply: tid=0 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.951+0000 7ff13bba5740 1 client.16413 shutdown 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 msgr2=0x563ace19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace19de60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff11c096e90 tx=0x7ff11c09abc0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 shutdown_connections 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace1b0db0 0x563ace1b31e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace1a0960 0x563ace19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff119ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:18.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197d80 0x563ace19de60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1323511643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197460 0x563ace19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 >> 172.21.15.42:0/1323511643 conn(0x563ace191b90 msgr2=0x563ace193050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 shutdown_connections 2024-03-12T04:18:18.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 1 -- 172.21.15.42:0/1323511643 wait complete. 2024-03-12T04:18:18.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.952+0000 7ff13bba5740 20 client.16413 trim_cache size 0 max 16384 2024-03-12T04:18:18.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 Processor -- start 2024-03-12T04:18:18.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 -- start start 2024-03-12T04:18:18.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197460 0x563ace19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197d80 0x563ace1a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace082130 con 0x563ace197460 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace1929a0 con 0x563ace197d80 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13bba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace196c00 con 0x563ace1a0960 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197460 0x563ace19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197460 0x563ace19e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41462/0 (socket says 172.21.15.42:41462) 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 -- 172.21.15.42:0/1132470635 learned_addr learned my addr 172.21.15.42:0/1132470635 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff131d9b640 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197d80 0x563ace1a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff132d9d640 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 -- 172.21.15.42:0/1132470635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 msgr2=0x563ace1a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 -- 172.21.15.42:0/1132470635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197d80 msgr2=0x563ace1a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197d80 0x563ace1a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 -- 172.21.15.42:0/1132470635 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ace097c20 con 0x563ace197460 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff132d9d640 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.964+0000 7ff13259c640 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197460 0x563ace19e1c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00ad00 tx=0x7ff12c00ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b455a6a5f520681 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff11bfff640 1 -- 172.21.15.42:0/1132470635 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0186c0 con 0x563ace197460 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff11bfff640 1 -- 172.21.15.42:0/1132470635 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff12c02c070 con 0x563ace197460 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff11bfff640 1 -- 172.21.15.42:0/1132470635 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c010a60 con 0x563ace197460 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 -- 172.21.15.42:0/1132470635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197460 msgr2=0x563ace19e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197460 0x563ace19e1c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00ad00 tx=0x7ff12c00ac90 comp rx=0 tx=0).stop 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 -- 172.21.15.42:0/1132470635 shutdown_connections 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace1a0960 0x563ace1a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197d80 0x563ace1a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 --2- 172.21.15.42:0/1132470635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace197460 0x563ace19e1c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 -- 172.21.15.42:0/1132470635 >> 172.21.15.42:0/1132470635 conn(0x563ace191b90 msgr2=0x563ace198ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 -- 172.21.15.42:0/1132470635 shutdown_connections 2024-03-12T04:18:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 -- 172.21.15.42:0/1132470635 wait complete. 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.965+0000 7ff13bba5740 1 -- start start 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 Processor -- start 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace1a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197d80 0x563ace1a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace1a0960 0x563ace207310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace1a7440 con 0x563ace1a0960 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace207850 con 0x563ace197460 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ace207990 con 0x563ace197d80 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 10 client.0 _ll_register_callbacks cb 0x7ff12c00a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace1a0960 0x563ace207310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace1a0960 0x563ace207310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41470/0 (socket says 172.21.15.42:41470) 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 -- 172.21.15.42:0/3702112196 learned_addr learned my addr 172.21.15.42:0/3702112196 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13259c640 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace1a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 -- 172.21.15.42:0/3702112196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197d80 msgr2=0x563ace1a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197d80 0x563ace1a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 -- 172.21.15.42:0/3702112196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 msgr2=0x563ace1a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace1a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 -- 172.21.15.42:0/3702112196 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ace0bf060 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13259c640 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace1a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff132d9d640 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace1a0960 0x563ace207310 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff124032800 tx=0x7ff12402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9a5b73756b3cfd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff119ffb640 1 -- 172.21.15.42:0/3702112196 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124003970 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff119ffb640 1 -- 172.21.15.42:0/3702112196 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff124003ad0 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.966+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff12c018af0 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff119ffb640 1 -- 172.21.15.42:0/3702112196 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff124003c90 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 10 client.16419 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ace0bd8c0 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff119ffb640 1 -- 172.21.15.42:0/3702112196 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff12c018af0 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff119ffb640 1 -- 172.21.15.42:0/3702112196 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff12403e070 con 0x563ace1a0960 2024-03-12T04:18:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 10 client.16419 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 10 client.16419 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 20 client.16419 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace209100 0x563ace20b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 4 client.16419 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563acde88890 con 0x563ace209100 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.967+0000 7ff13259c640 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace209100 0x563ace20b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff119ffb640 10 client.16419 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13259c640 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace209100 0x563ace20b520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f11082f121dc249 server_cookie=c6cb5bab86b49e7a in_seq=0 out_seq=0 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff119ffb640 1 -- 172.21.15.42:0/3702112196 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563acde88890 con 0x563ace209100 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff119ffb640 10 client.16419 handle_command_reply: tid=0 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 client.16419 shutdown 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace1a0960 msgr2=0x563ace207310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace1a0960 0x563ace207310 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff124032800 tx=0x7ff12402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 shutdown_connections 2024-03-12T04:18:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x563ace209100 0x563ace20b520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ace1a0960 0x563ace207310 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ace197d80 0x563ace1a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff119ffb640 0 client.16419 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 --2- 172.21.15.42:0/3702112196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ace197460 0x563ace1a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 >> 172.21.15.42:0/3702112196 conn(0x563ace191b90 msgr2=0x563ace18e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 shutdown_connections 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 1 -- 172.21.15.42:0/3702112196 wait complete. 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.968+0000 7ff13bba5740 20 client.16419 trim_cache size 0 max 16384 2024-03-12T04:18:18.977 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff1200a1ba0 msgr2=0x7ff1200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff1200a1ba0 0x7ff1200a4060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff124037040 tx=0x7ff124034520 comp rx=0 tx=0).stop 2024-03-12T04:18:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 msgr2=0x7ff134196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff134196af0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c970 tx=0x7ff11c00cd70 comp rx=0 tx=0).stop 2024-03-12T04:18:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 shutdown_connections 2024-03-12T04:18:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff1200a1ba0 0x7ff1200a4060 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff13419ae70 0x7ff13419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134105370 0x7ff134196af0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 --2- 172.21.15.42:0/2227308597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1341002a0 0x7ff1341965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:18.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 >> 172.21.15.42:0/2227308597 conn(0x7ff134076400 msgr2=0x7ff1340fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:18.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 shutdown_connections 2024-03-12T04:18:18.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:18.969+0000 7ff138ef4640 1 -- 172.21.15.42:0/2227308597 wait complete. 2024-03-12T04:18:18.982 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:18.982 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:18:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 -- 172.21.15.42:0/3583766561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3041002c0 msgr2=0x7fe304105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/3583766561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3041002c0 0x7fe304105df0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009900 tx=0x7fe2f402bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 -- 172.21.15.42:0/3583766561 shutdown_connections 2024-03-12T04:18:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/3583766561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe304106520 0x7fe30410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/3583766561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3041002c0 0x7fe304105df0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/3583766561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3040ff910 0x7fe3040ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 -- 172.21.15.42:0/3583766561 >> 172.21.15.42:0/3583766561 conn(0x7fe304076640 msgr2=0x7fe3040fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 -- 172.21.15.42:0/3583766561 shutdown_connections 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.084+0000 7fe30c1f4640 1 -- 172.21.15.42:0/3583766561 wait complete. 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 Processor -- start 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 -- start start 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3040ff910 0x7fe304100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe304106520 0x7fe304100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe304101300 0x7fe3041a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041a5e60 con 0x7fe304106520 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041a5fd0 con 0x7fe3040ff910 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30c1f4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041a6190 con 0x7fe304101300 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe304106520 0x7fe304100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe304106520 0x7fe304100dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41484/0 (socket says 172.21.15.42:41484) 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 -- 172.21.15.42:0/2842022681 learned_addr learned my addr 172.21.15.42:0/2842022681 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe309f90640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3040ff910 0x7fe304100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30a791640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe304101300 0x7fe3041a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 -- 172.21.15.42:0/2842022681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe304101300 msgr2=0x7fe3041a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe304101300 0x7fe3041a5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 -- 172.21.15.42:0/2842022681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3040ff910 msgr2=0x7fe304100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3040ff910 0x7fe304100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30978f640 1 -- 172.21.15.42:0/2842022681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f4009560 con 0x7fe304106520 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.085+0000 7fe30a791640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe304101300 0x7fe3041a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.086+0000 7fe30978f640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe304106520 0x7fe304100dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4036040 tx=0x7fe2f4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.086+0000 7fe2faffd640 1 -- 172.21.15.42:0/2842022681 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40043d0 con 0x7fe304106520 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.086+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3041a6410 con 0x7fe304106520 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.086+0000 7fe2faffd640 1 -- 172.21.15.42:0/2842022681 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2f402de90 con 0x7fe304106520 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.086+0000 7fe2faffd640 1 -- 172.21.15.42:0/2842022681 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40052e0 con 0x7fe304106520 2024-03-12T04:18:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.086+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3041a6920 con 0x7fe304106520 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe2faffd640 1 -- 172.21.15.42:0/2842022681 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe2f4003cc0 con 0x7fe304106520 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 Processor -- start 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 -- start start 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 0x55dedb7d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb785960 con 0x55dedb7ca250 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb739310 con 0x55dedb7cab70 2024-03-12T04:18:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30edca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb736840 con 0x55dedb7d3750 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30a791640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30a791640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41490/0 (socket says 172.21.15.42:41490) 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30a791640 1 -- 172.21.15.42:0/4271145664 learned_addr learned my addr 172.21.15.42:0/4271145664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe2faffd640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2f00a1bd0 0x7fe2f00a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30a791640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2f00a1bd0 0x7fe2f00a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe309f90640 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe2faffd640 1 -- 172.21.15.42:0/2842022681 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe2f40ef6a0 con 0x7fe304106520 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.089+0000 7fe30978f640 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 0x55dedb7d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe30a791640 1 -- 172.21.15.42:0/4271145664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 msgr2=0x55dedb7d5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe30a791640 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 0x55dedb7d5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe30a791640 1 -- 172.21.15.42:0/4271145664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 msgr2=0x55dedb7d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe30a791640 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe30a791640 1 -- 172.21.15.42:0/4271145664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dedb6f4060 con 0x55dedb7ca250 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe309f90640 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe30a791640 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7ca630 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005c40 tx=0x7fe2fc012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8aba57dd8998 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe2effff640 1 -- 172.21.15.42:0/4271145664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc01fc90 con 0x55dedb7ca250 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe2effff640 1 -- 172.21.15.42:0/4271145664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2fc009480 con 0x55dedb7ca250 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.090+0000 7fe2effff640 1 -- 172.21.15.42:0/4271145664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00fbd0 con 0x55dedb7ca250 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 -- 172.21.15.42:0/4271145664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 msgr2=0x55dedb7ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30a791640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2f00a1bd0 0x7fe2f00a4090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc023000 tx=0x7fe2fc022000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7ca630 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005c40 tx=0x7fe2fc012f80 comp rx=0 tx=0).stop 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 -- 172.21.15.42:0/4271145664 shutdown_connections 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 0x55dedb7d5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 --2- 172.21.15.42:0/4271145664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7ca630 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 -- 172.21.15.42:0/4271145664 >> 172.21.15.42:0/4271145664 conn(0x55dedb7c4980 msgr2=0x55dedb7c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 -- 172.21.15.42:0/4271145664 shutdown_connections 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 -- 172.21.15.42:0/4271145664 wait complete. 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 -- start start 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.091+0000 7fe30edca740 1 Processor -- start 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb7d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb7d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb732020 con 0x55dedb7d3750 2024-03-12T04:18:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb405430 con 0x55dedb7ca250 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb3c1e00 con 0x55dedb7cab70 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 10 client.0 _ll_register_callbacks cb 0x55dedb6d9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41492/0 (socket says 172.21.15.42:41492) 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 -- 172.21.15.42:0/3363427210 learned_addr learned my addr 172.21.15.42:0/3363427210 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe309f90640 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb7d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30978f640 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb7d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 -- 172.21.15.42:0/3363427210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 msgr2=0x55dedb7d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb7d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 -- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 msgr2=0x55dedb7d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb7d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 -- 172.21.15.42:0/3363427210 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc0126d0 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30a791640 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d11e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005c10 tx=0x7fe2fc00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3757f36e00fe63dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe2ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3363427210 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00f5d0 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3363427210 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe2fc004a00 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.092+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dedb4317e0 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3363427210 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2fc00f730 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 10 client.16431 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dedb7306a0 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3363427210 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55dedb4317e0 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3363427210 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2fc005720 con 0x55dedb7d3750 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 10 client.16431 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 20 client.16431 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb7e39f0 0x55dedb7e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 4 client.16431 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dedb46e720 con 0x55dedb7e39f0 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.093+0000 7fe309f90640 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb7e39f0 0x55dedb7e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.094+0000 7fe2ed7fa640 10 client.16431 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.094+0000 7fe309f90640 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb7e39f0 0x55dedb7e5e10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8142041a2530cda5 server_cookie=515079d22f9b06b5 in_seq=0 out_seq=0 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.094+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3363427210 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dedb46e720 con 0x55dedb7e39f0 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.094+0000 7fe2ed7fa640 10 client.16431 handle_command_reply: tid=0 2024-03-12T04:18:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.094+0000 7fe30edca740 1 client.16431 shutdown 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 msgr2=0x55dedb7d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d11e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005c10 tx=0x7fe2fc00fc00 comp rx=0 tx=0).stop 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 shutdown_connections 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb7e39f0 0x55dedb7e5e10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d11e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb7d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 --2- 172.21.15.42:0/3363427210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb7d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe2ed7fa640 0 client.16431 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 >> 172.21.15.42:0/3363427210 conn(0x55dedb7c4980 msgr2=0x55dedb7c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:19.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 shutdown_connections 2024-03-12T04:18:19.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 1 -- 172.21.15.42:0/3363427210 wait complete. 2024-03-12T04:18:19.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.095+0000 7fe30edca740 20 client.16431 trim_cache size 0 max 16384 2024-03-12T04:18:19.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 Processor -- start 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 -- start start 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 0x55dedb7cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb6b7130 con 0x55dedb7ca250 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb7c0080 con 0x55dedb7cab70 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30edca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb7c02b0 con 0x55dedb7d3750 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41502/0 (socket says 172.21.15.42:41502) 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 -- 172.21.15.42:0/3011533213 learned_addr learned my addr 172.21.15.42:0/3011533213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30978f640 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 -- 172.21.15.42:0/3011533213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 msgr2=0x55dedb7cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 0x55dedb7cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 -- 172.21.15.42:0/3011533213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 msgr2=0x55dedb7d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 -- 172.21.15.42:0/3011533213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dedb6ccc20 con 0x55dedb7ca250 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe30978f640 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe309f90640 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7d2e90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55dedb7e6710 tx=0x7fe3000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb22b39b59f34a98 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.107+0000 7fe2ef7fe640 1 -- 172.21.15.42:0/3011533213 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300003d20 con 0x55dedb7ca250 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe2ef7fe640 1 -- 172.21.15.42:0/3011533213 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe300018040 con 0x55dedb7ca250 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe2ef7fe640 1 -- 172.21.15.42:0/3011533213 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000108c0 con 0x55dedb7ca250 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 -- 172.21.15.42:0/3011533213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 msgr2=0x55dedb7d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7d2e90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55dedb7e6710 tx=0x7fe3000046c0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 -- 172.21.15.42:0/3011533213 shutdown_connections 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7d3750 0x55dedb7cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7cab70 0x55dedb7d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 --2- 172.21.15.42:0/3011533213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7ca250 0x55dedb7d2e90 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 -- 172.21.15.42:0/3011533213 >> 172.21.15.42:0/3011533213 conn(0x55dedb7c4980 msgr2=0x55dedb7cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 -- 172.21.15.42:0/3011533213 shutdown_connections 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 -- 172.21.15.42:0/3011533213 wait complete. 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 -- start start 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.108+0000 7fe30edca740 1 Processor -- start 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30edca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb833290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30edca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb8337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30edca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30edca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb7d9f10 con 0x55dedb7d3750 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30edca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb7da080 con 0x55dedb7ca250 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30edca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dedb7da240 con 0x55dedb7cab70 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30edca740 10 client.0 _ll_register_callbacks cb 0x7fe2fc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41516/0 (socket says 172.21.15.42:41516) 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 -- 172.21.15.42:0/3527552765 learned_addr learned my addr 172.21.15.42:0/3527552765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe309f90640 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb833290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30978f640 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb8337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 -- 172.21.15.42:0/3527552765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 msgr2=0x55dedb8337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb8337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 -- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 msgr2=0x55dedb833290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb833290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.109+0000 7fe30a791640 1 -- 172.21.15.42:0/3527552765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe300005c50 con 0x55dedb7d3750 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe30a791640 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d9860 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00e890 tx=0x7fe2fc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72af8e58fc986e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe2ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3527552765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0049c0 con 0x55dedb7d3750 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3527552765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe2fc004b20 con 0x55dedb7d3750 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2fc009480 con 0x55dedb7d3750 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3527552765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2fc004ce0 con 0x55dedb7d3750 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe30edca740 10 client.16437 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dedb6f28c0 con 0x55dedb7d3750 2024-03-12T04:18:19.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.110+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3527552765 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2fc009480 con 0x55dedb7d3750 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3527552765 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2fc040070 con 0x55dedb7d3750 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe30edca740 10 client.16437 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe30edca740 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe30edca740 20 client.16437 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe30edca740 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb83bdc0 0x55dedb83e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe30edca740 4 client.16437 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dedb4bd890 con 0x55dedb83bdc0 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe309f90640 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb83bdc0 0x55dedb83e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe309f90640 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb83bdc0 0x55dedb83e1b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6767f1b414f4d56 server_cookie=d30ebee90df7d355 in_seq=0 out_seq=0 2024-03-12T04:18:19.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.111+0000 7fe2ed7fa640 10 client.16437 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:19.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe2ed7fa640 1 -- 172.21.15.42:0/3527552765 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dedb4bd890 con 0x55dedb83bdc0 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe2ed7fa640 10 client.16437 handle_command_reply: tid=0 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 client.16437 shutdown 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 msgr2=0x55dedb7d9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d9860 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00e890 tx=0x7fe2fc005b00 comp rx=0 tx=0).stop 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 shutdown_connections 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dedb83bdc0 0x55dedb83e1b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dedb7d3750 0x55dedb7d9860 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe2ed7fa640 0 client.16437 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dedb7cab70 0x55dedb8337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 --2- 172.21.15.42:0/3527552765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dedb7ca250 0x55dedb833290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 >> 172.21.15.42:0/3527552765 conn(0x55dedb7c4980 msgr2=0x55dedb7c32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 shutdown_connections 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 1 -- 172.21.15.42:0/3527552765 wait complete. 2024-03-12T04:18:19.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30edca740 20 client.16437 trim_cache size 0 max 16384 2024-03-12T04:18:19.121 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2f00a1bd0 msgr2=0x7fe2f00a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2f00a1bd0 0x7fe2f00a4090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc023000 tx=0x7fe2fc022000 comp rx=0 tx=0).stop 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe304106520 msgr2=0x7fe304100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe304106520 0x7fe304100dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4036040 tx=0x7fe2f4005e20 comp rx=0 tx=0).stop 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 shutdown_connections 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2f00a1bd0 0x7fe2f00a4090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe304101300 0x7fe3041a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.112+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe304106520 0x7fe304100dc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.113+0000 7fe30c1f4640 1 --2- 172.21.15.42:0/2842022681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3040ff910 0x7fe304100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.113+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 >> 172.21.15.42:0/2842022681 conn(0x7fe304076640 msgr2=0x7fe304104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.113+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 shutdown_connections 2024-03-12T04:18:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:19.113+0000 7fe30c1f4640 1 -- 172.21.15.42:0/2842022681 wait complete. 2024-03-12T04:18:19.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:19 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3150190032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:18:19.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:19 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:18:19.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:19 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3150190032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:18:19.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:19 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:18:19.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:19 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3150190032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:18:19.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:19 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:18:20.127 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:18:20.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:18:20.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 -- 172.21.15.42:0/22544706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1480685f0 msgr2=0x7fd1480689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 --2- 172.21.15.42:0/22544706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1480685f0 0x7fd1480689d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd130009930 tx=0x7fd13002bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 -- 172.21.15.42:0/22544706 shutdown_connections 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 --2- 172.21.15.42:0/22544706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1481064c0 0x7fd14810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 --2- 172.21.15.42:0/22544706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd148068fa0 0x7fd148105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 --2- 172.21.15.42:0/22544706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1480685f0 0x7fd1480689d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 -- 172.21.15.42:0/22544706 >> 172.21.15.42:0/22544706 conn(0x7fd148077180 msgr2=0x7fd148077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 -- 172.21.15.42:0/22544706 shutdown_connections 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 -- 172.21.15.42:0/22544706 wait complete. 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 Processor -- start 2024-03-12T04:18:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.235+0000 7fd14e440640 1 -- start start 2024-03-12T04:18:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14e440640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1480685f0 0x7fd1481976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14e440640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1481064c0 0x7fd148197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14e440640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14819bfb0 0x7fd14819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14e440640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14819c9a0 con 0x7fd1481064c0 2024-03-12T04:18:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14e440640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14819cb10 con 0x7fd14819bfb0 2024-03-12T04:18:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14e440640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14819cc80 con 0x7fd1480685f0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1481064c0 0x7fd148197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1481064c0 0x7fd148197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41544/0 (socket says 172.21.15.42:41544) 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 -- 172.21.15.42:0/4163937336 learned_addr learned my addr 172.21.15.42:0/4163937336 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd147fff640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1480685f0 0x7fd1481976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14819bfb0 0x7fd14819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 -- 172.21.15.42:0/4163937336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1480685f0 msgr2=0x7fd1481976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1480685f0 0x7fd1481976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 -- 172.21.15.42:0/4163937336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14819bfb0 msgr2=0x7fd14819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14819bfb0 0x7fd14819c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 -- 172.21.15.42:0/4163937336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130009560 con 0x7fd1481064c0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1477fe640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1481064c0 0x7fd148197c30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c970 tx=0x7fd13800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1457fa640 1 -- 172.21.15.42:0/4163937336 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138007bb0 con 0x7fd1481064c0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1481a4260 con 0x7fd1481064c0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1457fa640 1 -- 172.21.15.42:0/4163937336 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd138007d10 con 0x7fd1481064c0 2024-03-12T04:18:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.236+0000 7fd1457fa640 1 -- 172.21.15.42:0/4163937336 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138004e40 con 0x7fd1481064c0 2024-03-12T04:18:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.237+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1481a4710 con 0x7fd1481064c0 2024-03-12T04:18:20.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.238+0000 7fd1457fa640 1 -- 172.21.15.42:0/4163937336 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd138005000 con 0x7fd1481064c0 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 Processor -- start 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 -- start start 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd1457fa640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd1340a1ba0 0x7fd1340a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x561417423760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x56141742ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417391f90 con 0x561417423380 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141738f4c0 con 0x561417423d30 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd15100d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141738aca0 con 0x56141742c8b0 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd1340a1ba0 0x7fd1340a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd147fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x561417423760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd147fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x561417423760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41554/0 (socket says 172.21.15.42:41554) 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd147fff640 1 -- 172.21.15.42:0/1362817252 learned_addr learned my addr 172.21.15.42:0/1362817252 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd1457fa640 1 -- 172.21.15.42:0/4163937336 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd1380cd480 con 0x7fd1481064c0 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x56141742ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.239+0000 7fd1477fe640 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd147fff640 1 -- 172.21.15.42:0/1362817252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 msgr2=0x56141742ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd147fff640 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x56141742ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd147fff640 1 -- 172.21.15.42:0/1362817252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 msgr2=0x56141742c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd147fff640 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd147fff640 1 -- 172.21.15.42:0/1362817252 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141734d0b0 con 0x561417423380 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd1340a1ba0 0x7fd1340a4060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd140009fd0 tx=0x7fd140013b80 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd147fff640 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x561417423760 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd130036040 tx=0x7fd1300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67a222f1ecfad22 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd13e7fc640 1 -- 172.21.15.42:0/1362817252 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130004970 con 0x561417423380 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd13e7fc640 1 -- 172.21.15.42:0/1362817252 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd13002de90 con 0x561417423380 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.240+0000 7fd13e7fc640 1 -- 172.21.15.42:0/1362817252 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130003850 con 0x561417423380 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- 172.21.15.42:0/1362817252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 msgr2=0x561417423760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x561417423760 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd130036040 tx=0x7fd1300046c0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- 172.21.15.42:0/1362817252 shutdown_connections 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x56141742ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 --2- 172.21.15.42:0/1362817252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x561417423760 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- 172.21.15.42:0/1362817252 >> 172.21.15.42:0/1362817252 conn(0x56141741d840 msgr2=0x56141741dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- 172.21.15.42:0/1362817252 shutdown_connections 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- 172.21.15.42:0/1362817252 wait complete. 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- start start 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 Processor -- start 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561417423380 0x561417429810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x561417429d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56141742c8b0 0x56141742a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141732ff40 con 0x56141742c8b0 2024-03-12T04:18:20.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141705e4b0 con 0x561417423d30 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.241+0000 7fd15100d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417310670 con 0x561417423380 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd15100d740 10 client.0 _ll_register_callbacks cb 0x56141737df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56141742c8b0 0x56141742a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56141742c8b0 0x56141742a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41562/0 (socket says 172.21.15.42:41562) 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 -- 172.21.15.42:0/599575603 learned_addr learned my addr 172.21.15.42:0/599575603 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd147fff640 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561417423380 0x561417429810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd1477fe640 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x561417429d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 -- 172.21.15.42:0/599575603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561417423380 msgr2=0x561417429810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561417423380 0x561417429810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 -- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 msgr2=0x561417429d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x561417429d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 -- 172.21.15.42:0/599575603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14810e880 con 0x56141742c8b0 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56141742c8b0 0x56141742a290 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd14000ff10 tx=0x7fd140004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6045384f3fd7b45 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd1277fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:20.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd1277fe640 1 -- 172.21.15.42:0/599575603 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140003e00 con 0x56141742c8b0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd1277fe640 1 -- 172.21.15.42:0/599575603 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd140004740 con 0x56141742c8b0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561417364f40 con 0x56141742c8b0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd1277fe640 1 -- 172.21.15.42:0/599575603 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd140020430 con 0x56141742c8b0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd15100d740 10 client.16449 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.242+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561417389320 con 0x56141742c8b0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd1277fe640 1 -- 172.21.15.42:0/599575603 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561417364f40 con 0x56141742c8b0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd1277fe640 1 -- 172.21.15.42:0/599575603 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd140032550 con 0x56141742c8b0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd15100d740 10 client.16449 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd15100d740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd15100d740 20 client.16449 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd15100d740 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56141743cd70 0x56141743f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd15100d740 4 client.16449 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561417381310 con 0x56141743cd70 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd147fff640 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56141743cd70 0x56141743f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd1277fe640 10 client.16449 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.243+0000 7fd147fff640 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56141743cd70 0x56141743f190 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3d7d2714d27920 server_cookie=ab63ab030b54b4dc in_seq=0 out_seq=0 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd1277fe640 1 -- 172.21.15.42:0/599575603 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561417381310 con 0x56141743cd70 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd1277fe640 10 client.16449 handle_command_reply: tid=0 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 client.16449 shutdown 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56141742c8b0 msgr2=0x56141742a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56141742c8b0 0x56141742a290 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd14000ff10 tx=0x7fd140004ab0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 shutdown_connections 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56141743cd70 0x56141743f190 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56141742c8b0 0x56141742a290 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x561417429d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 --2- 172.21.15.42:0/599575603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561417423380 0x561417429810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd1277fe640 0 client.16449 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 >> 172.21.15.42:0/599575603 conn(0x56141741d840 msgr2=0x56141741f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 shutdown_connections 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 1 -- 172.21.15.42:0/599575603 wait complete. 2024-03-12T04:18:20.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.244+0000 7fd15100d740 20 client.16449 trim_cache size 0 max 16384 2024-03-12T04:18:20.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 Processor -- start 2024-03-12T04:18:20.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 -- start start 2024-03-12T04:18:20.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x56141742bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x561417426ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417417d20 con 0x561417423380 2024-03-12T04:18:20.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417427d80 con 0x561417423d30 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.259+0000 7fd15100d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614174253f0 con 0x56141742c8b0 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x56141742bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x56141742bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41564/0 (socket says 172.21.15.42:41564) 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 -- 172.21.15.42:0/3359761899 learned_addr learned my addr 172.21.15.42:0/3359761899 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd1477fe640 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 -- 172.21.15.42:0/3359761899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 msgr2=0x561417426ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x561417426ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x561417426ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 -- 172.21.15.42:0/3359761899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 msgr2=0x56141742a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 -- 172.21.15.42:0/3359761899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561417325c00 con 0x561417423380 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd147fff640 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x56141742bf40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd13002f020 tx=0x7fd130005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b655fe32b9e852 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd13dffb640 1 -- 172.21.15.42:0/3359761899 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1300049b0 con 0x561417423380 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd13dffb640 1 -- 172.21.15.42:0/3359761899 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd130050b70 con 0x561417423380 2024-03-12T04:18:20.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.260+0000 7fd13dffb640 1 -- 172.21.15.42:0/3359761899 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130004dc0 con 0x561417423380 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 -- 172.21.15.42:0/3359761899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 msgr2=0x56141742bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x56141742bf40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd13002f020 tx=0x7fd130005fd0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 -- 172.21.15.42:0/3359761899 shutdown_connections 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x561417426ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423d30 0x56141742a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 --2- 172.21.15.42:0/3359761899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423380 0x56141742bf40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 -- 172.21.15.42:0/3359761899 >> 172.21.15.42:0/3359761899 conn(0x56141741d840 msgr2=0x561417428100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 -- 172.21.15.42:0/3359761899 shutdown_connections 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 -- 172.21.15.42:0/3359761899 wait complete. 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 -- start start 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 Processor -- start 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423380 0x561417433d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423d30 0x5614174320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.261+0000 7fd15100d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x5614174326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd15100d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417432dc0 con 0x561417423d30 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd15100d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417432f30 con 0x561417423380 2024-03-12T04:18:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd15100d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614174932d0 con 0x56141742c8b0 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd15100d740 10 client.0 _ll_register_callbacks cb 0x7fd130009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423d30 0x5614174320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423d30 0x5614174320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41576/0 (socket says 172.21.15.42:41576) 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 -- 172.21.15.42:0/1318591840 learned_addr learned my addr 172.21.15.42:0/1318591840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd147fff640 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423380 0x561417433d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd14c9dd640 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x5614174326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 -- 172.21.15.42:0/1318591840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 msgr2=0x5614174326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x5614174326f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 -- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423380 msgr2=0x561417433d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423380 0x561417433d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 -- 172.21.15.42:0/1318591840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56141734d0b0 con 0x561417423d30 2024-03-12T04:18:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1477fe640 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423d30 0x5614174320c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd1380cac20 tx=0x7fd138099ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d4570b37ee91f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1277fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1277fe640 1 -- 172.21.15.42:0/1318591840 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1380a2840 con 0x561417423d30 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1277fe640 1 -- 172.21.15.42:0/1318591840 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd1380052b0 con 0x561417423d30 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614173b0900 con 0x561417423d30 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd1277fe640 1 -- 172.21.15.42:0/1318591840 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd138005560 con 0x561417423d30 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd15100d740 10 client.16455 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.262+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56141734b910 con 0x561417423d30 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd1277fe640 1 -- 172.21.15.42:0/1318591840 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5614173b0900 con 0x561417423d30 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd1277fe640 1 -- 172.21.15.42:0/1318591840 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd138016030 con 0x561417423d30 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd15100d740 10 client.16455 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd15100d740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd15100d740 20 client.16455 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd15100d740 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561417494be0 0x561417497000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd15100d740 4 client.16455 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:18:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56141708a860 con 0x561417494be0 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd147fff640 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561417494be0 0x561417497000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd147fff640 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561417494be0 0x561417497000 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605b07616f18701e server_cookie=e1e65d1d2e407b3c in_seq=0 out_seq=0 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.263+0000 7fd1277fe640 10 client.16455 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd1277fe640 1 -- 172.21.15.42:0/1318591840 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56141708a860 con 0x561417494be0 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd1277fe640 10 client.16455 handle_command_reply: tid=0 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 client.16455 shutdown 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423d30 msgr2=0x5614174320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423d30 0x5614174320c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd1380cac20 tx=0x7fd138099ab0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 shutdown_connections 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561417494be0 0x561417497000 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56141742c8b0 0x5614174326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561417423d30 0x5614174320c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd1277fe640 0 client.16455 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 --2- 172.21.15.42:0/1318591840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561417423380 0x561417433d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 >> 172.21.15.42:0/1318591840 conn(0x56141741d840 msgr2=0x56141741c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 shutdown_connections 2024-03-12T04:18:20.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.264+0000 7fd15100d740 1 -- 172.21.15.42:0/1318591840 wait complete. 2024-03-12T04:18:20.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd15100d740 20 client.16455 trim_cache size 0 max 16384 2024-03-12T04:18:20.273 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:18:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:18:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "132953ab-7816-43fd-90bc-8516fded045d", 2024-03-12T04:18:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:18:20.273 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd1340a1ba0 msgr2=0x7fd1340a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd1340a1ba0 0x7fd1340a4060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd140009fd0 tx=0x7fd140013b80 comp rx=0 tx=0).stop 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1481064c0 msgr2=0x7fd148197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1481064c0 0x7fd148197c30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c970 tx=0x7fd13800ce30 comp rx=0 tx=0).stop 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 shutdown_connections 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd1340a1ba0 0x7fd1340a4060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14819bfb0 0x7fd14819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1481064c0 0x7fd148197c30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 --2- 172.21.15.42:0/4163937336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1480685f0 0x7fd1481976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 >> 172.21.15.42:0/4163937336 conn(0x7fd148077180 msgr2=0x7fd1480fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 shutdown_connections 2024-03-12T04:18:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.265+0000 7fd14e440640 1 -- 172.21.15.42:0/4163937336 wait complete. 2024-03-12T04:18:20.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:18:20.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:18:20.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 -- 172.21.15.42:0/2972386166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d0106430 msgr2=0x7f08d010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 --2- 172.21.15.42:0/2972386166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d0106430 0x7f08d010e000 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009cd0 tx=0x7f08c802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 -- 172.21.15.42:0/2972386166 shutdown_connections 2024-03-12T04:18:20.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 --2- 172.21.15.42:0/2972386166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d0106430 0x7f08d010e000 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 --2- 172.21.15.42:0/2972386166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d0101d10 0x7f08d0105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 --2- 172.21.15.42:0/2972386166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d0101360 0x7f08d0101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 -- 172.21.15.42:0/2972386166 >> 172.21.15.42:0/2972386166 conn(0x7f08d00fcc20 msgr2=0x7f08d00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 -- 172.21.15.42:0/2972386166 shutdown_connections 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.380+0000 7f08d86ff640 1 -- 172.21.15.42:0/2972386166 wait complete. 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 Processor -- start 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 -- start start 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d0101360 0x7f08d0197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d0106430 0x7f08d0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d019bfa0 0x7f08d019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019c990 con 0x7f08d019bfa0 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019cb00 con 0x7f08d0101360 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d86ff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019cc70 con 0x7f08d0106430 2024-03-12T04:18:20.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d6c9c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d019bfa0 0x7f08d019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d6c9c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d019bfa0 0x7f08d019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41600/0 (socket says 172.21.15.42:41600) 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d6c9c640 1 -- 172.21.15.42:0/690480036 learned_addr learned my addr 172.21.15.42:0/690480036 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d649b640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d0101360 0x7f08d0197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.381+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d0106430 0x7f08d0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d6c9c640 1 -- 172.21.15.42:0/690480036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d0106430 msgr2=0x7f08d0197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d0106430 0x7f08d0197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d6c9c640 1 -- 172.21.15.42:0/690480036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d0101360 msgr2=0x7f08d0197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d0101360 0x7f08d0197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d6c9c640 1 -- 172.21.15.42:0/690480036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8009900 con 0x7f08d019bfa0 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d0106430 0x7f08d0197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d649b640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d0101360 0x7f08d0197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d019bfa0 0x7f08d019c450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f08c8004af0 tx=0x7f08c8035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08c77fe640 1 -- 172.21.15.42:0/690480036 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80329e0 con 0x7f08d019bfa0 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08c77fe640 1 -- 172.21.15.42:0/690480036 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c8035e90 con 0x7f08d019bfa0 2024-03-12T04:18:20.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d01a4250 con 0x7f08d019bfa0 2024-03-12T04:18:20.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08c77fe640 1 -- 172.21.15.42:0/690480036 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8004720 con 0x7f08d019bfa0 2024-03-12T04:18:20.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.382+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01a4670 con 0x7f08d019bfa0 2024-03-12T04:18:20.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.384+0000 7f08c77fe640 1 -- 172.21.15.42:0/690480036 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f08c803b040 con 0x7f08d019bfa0 2024-03-12T04:18:20.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.384+0000 7f08db3a9740 1 Processor -- start 2024-03-12T04:18:20.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08c77fe640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08c77fe640 1 -- 172.21.15.42:0/690480036 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08c80f0170 con 0x7f08d019bfa0 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08db3a9740 1 -- start start 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d672b0 0x556b40d67690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d707b0 0x556b40d72ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08db3a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40cd6280 con 0x556b40d67bd0 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08db3a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40cd37b0 con 0x556b40d707b0 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08db3a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40ccef90 con 0x556b40d672b0 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d70270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41610/0 (socket says 172.21.15.42:41610) 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1878142661 learned_addr learned my addr 172.21.15.42:0/1878142661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d649b640 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d672b0 0x556b40d67690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005fd0 tx=0x7f08c0005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d707b0 0x556b40d72ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1878142661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d672b0 msgr2=0x556b40d67690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d672b0 0x556b40d67690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1878142661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d707b0 msgr2=0x556b40d72ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d707b0 0x556b40d72ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.385+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1878142661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b40c8d570 con 0x556b40d67bd0 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.386+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d70270 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f08c000bf90 tx=0x7f08c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767b8c387879339c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.386+0000 7f08c4ff9640 1 -- 172.21.15.42:0/1878142661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0004030 con 0x556b40d67bd0 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.386+0000 7f08c4ff9640 1 -- 172.21.15.42:0/1878142661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c00035f0 con 0x556b40d67bd0 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.386+0000 7f08c4ff9640 1 -- 172.21.15.42:0/1878142661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c001e6b0 con 0x556b40d67bd0 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.386+0000 7f08db3a9740 1 -- 172.21.15.42:0/1878142661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 msgr2=0x556b40d70270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.386+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d70270 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f08c000bf90 tx=0x7f08c0012000 comp rx=0 tx=0).stop 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- 172.21.15.42:0/1878142661 shutdown_connections 2024-03-12T04:18:20.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d707b0 0x556b40d72ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d70270 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1878142661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d672b0 0x556b40d67690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- 172.21.15.42:0/1878142661 >> 172.21.15.42:0/1878142661 conn(0x556b40d61990 msgr2=0x556b40d61d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- 172.21.15.42:0/1878142661 shutdown_connections 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- 172.21.15.42:0/1878142661 wait complete. 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- start start 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 Processor -- start 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d70250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d6dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b409a2430 con 0x556b40d67bd0 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4095ee00 con 0x556b40d672b0 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40c53fd0 con 0x556b40d707b0 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08db3a9740 10 client.0 _ll_register_callbacks cb 0x556b40ca9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41620/0 (socket says 172.21.15.42:41620) 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08d5c9a640 1 -- 172.21.15.42:0/2060532323 learned_addr learned my addr 172.21.15.42:0/2060532323 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.387+0000 7f08d649b640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d70250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d6dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d5c9a640 1 -- 172.21.15.42:0/2060532323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 msgr2=0x556b40d6dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d6dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d5c9a640 1 -- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 msgr2=0x556b40d70250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d70250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d5c9a640 1 -- 172.21.15.42:0/2060532323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c000bc40 con 0x556b40d67bd0 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d6dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d649b640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d70250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6d5e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003f30 tx=0x7f08c0011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b93a05783e61c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08b9ffb640 1 -- 172.21.15.42:0/2060532323 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c002e070 con 0x556b40d67bd0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08b9ffb640 1 -- 172.21.15.42:0/2060532323 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08c0004210 con 0x556b40d67bd0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b409ce7e0 con 0x556b40d67bd0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08b9ffb640 1 -- 172.21.15.42:0/2060532323 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c0003ac0 con 0x556b40d67bd0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08db3a9740 10 client.16467 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556b40ccd610 con 0x556b40d67bd0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.388+0000 7f08b9ffb640 1 -- 172.21.15.42:0/2060532323 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x556b409ce7e0 con 0x556b40d67bd0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08b9ffb640 1 -- 172.21.15.42:0/2060532323 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f08c0004af0 con 0x556b40d67bd0 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08db3a9740 10 client.16467 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08db3a9740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08db3a9740 20 client.16467 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08db3a9740 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40d80880 0x556b40d82cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08db3a9740 4 client.16467 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b40a0b720 con 0x556b40d80880 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08d649b640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40d80880 0x556b40d82cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08b9ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.389+0000 7f08d649b640 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40d80880 0x556b40d82cc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba3e08e594a3fe2 server_cookie=d9839485b50cf057 in_seq=0 out_seq=0 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08b9ffb640 1 -- 172.21.15.42:0/2060532323 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b40a0b720 con 0x556b40d80880 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08b9ffb640 10 client.16467 handle_command_reply: tid=0 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 client.16467 shutdown 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 msgr2=0x556b40d6d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6d5e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003f30 tx=0x7f08c0011f90 comp rx=0 tx=0).stop 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 shutdown_connections 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40d80880 0x556b40d82cc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d6dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6d5e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08b9ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 --2- 172.21.15.42:0/2060532323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d70250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 >> 172.21.15.42:0/2060532323 conn(0x556b40d61990 msgr2=0x556b40d63730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 shutdown_connections 2024-03-12T04:18:20.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 1 -- 172.21.15.42:0/2060532323 wait complete. 2024-03-12T04:18:20.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.390+0000 7f08db3a9740 20 client.16467 trim_cache size 0 max 16384 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 Processor -- start 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 -- start start 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d82ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d70250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40d7c570 con 0x556b40d67bd0 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40d80650 con 0x556b40d672b0 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08db3a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40d629e0 con 0x556b40d707b0 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41624/0 (socket says 172.21.15.42:41624) 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1174414374 learned_addr learned my addr 172.21.15.42:0/1174414374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08d649b640 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d82ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.405+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d70250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1174414374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 msgr2=0x556b40d70250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d70250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1174414374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 msgr2=0x556b40d82ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d82ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d5c9a640 1 -- 172.21.15.42:0/1174414374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b40c69ac0 con 0x556b40d67bd0 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d70250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d649b640 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d82ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6fd10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003d60 tx=0x7f08c0011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626631cabd06b136 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08bbfff640 1 -- 172.21.15.42:0/1174414374 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c002e070 con 0x556b40d67bd0 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08bbfff640 1 -- 172.21.15.42:0/1174414374 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c001eda0 con 0x556b40d67bd0 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08bbfff640 1 -- 172.21.15.42:0/1174414374 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0004210 con 0x556b40d67bd0 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08db3a9740 1 -- 172.21.15.42:0/1174414374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 msgr2=0x556b40d6fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6fd10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003d60 tx=0x7f08c0011e30 comp rx=0 tx=0).stop 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08db3a9740 1 -- 172.21.15.42:0/1174414374 shutdown_connections 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d70250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d6fd10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08db3a9740 1 --2- 172.21.15.42:0/1174414374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d82ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.406+0000 7f08db3a9740 1 -- 172.21.15.42:0/1174414374 >> 172.21.15.42:0/1174414374 conn(0x556b40d61990 msgr2=0x556b40d68910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 -- 172.21.15.42:0/1174414374 shutdown_connections 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 -- 172.21.15.42:0/1174414374 wait complete. 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 -- start start 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 Processor -- start 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d77330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d77870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d75820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40d75e80 con 0x556b40d67bd0 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40d75ff0 con 0x556b40d672b0 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b40d761b0 con 0x556b40d707b0 2024-03-12T04:18:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08db3a9740 10 client.0 _ll_register_callbacks cb 0x7f08c0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d77870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08d5c9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d77870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41630/0 (socket says 172.21.15.42:41630) 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08d5c9a640 1 -- 172.21.15.42:0/407876493 learned_addr learned my addr 172.21.15.42:0/407876493 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08d649b640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d77330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08d6c9c640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d75820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08d5c9a640 1 -- 172.21.15.42:0/407876493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 msgr2=0x556b40d75820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.407+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d75820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08d5c9a640 1 -- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 msgr2=0x556b40d77330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d77330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08d5c9a640 1 -- 172.21.15.42:0/407876493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b40ccd610 con 0x556b40d67bd0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08d649b640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d77330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08d5c9a640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d77870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f08c000b8a0 tx=0x7f08c00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26535a5ebb57256d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08b9ffb640 1 -- 172.21.15.42:0/407876493 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c002e070 con 0x556b40d67bd0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08b9ffb640 1 -- 172.21.15.42:0/407876493 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08c001ea90 con 0x556b40d67bd0 2024-03-12T04:18:20.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08c00035f0 con 0x556b40d67bd0 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08b9ffb640 1 -- 172.21.15.42:0/407876493 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c0004680 con 0x556b40d67bd0 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 10 client.16473 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556b40c8f6a0 con 0x556b40d67bd0 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08b9ffb640 1 -- 172.21.15.42:0/407876493 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c00035f0 con 0x556b40d67bd0 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08b9ffb640 1 -- 172.21.15.42:0/407876493 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f08c0050480 con 0x556b40d67bd0 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 10 client.16473 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 20 client.16473 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40dd7a90 0x556b40dd9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 4 client.16473 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.408+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556b40a5a890 con 0x556b40dd7a90 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08d649b640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40dd7a90 0x556b40dd9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08d649b640 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40dd7a90 0x556b40dd9eb0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3f4e492f5fc0a server_cookie=7c0e1d1094f5e169 in_seq=0 out_seq=0 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08b9ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08b9ffb640 1 -- 172.21.15.42:0/407876493 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556b40a5a890 con 0x556b40dd7a90 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08b9ffb640 10 client.16473 handle_command_reply: tid=0 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 client.16473 shutdown 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 msgr2=0x556b40d77870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d77870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f08c000b8a0 tx=0x7f08c00044c0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 shutdown_connections 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556b40dd7a90 0x556b40dd9eb0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b40d707b0 0x556b40d75820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b40d67bd0 0x556b40d77870 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08b9ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 --2- 172.21.15.42:0/407876493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b40d672b0 0x556b40d77330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 >> 172.21.15.42:0/407876493 conn(0x556b40d61990 msgr2=0x556b40d6be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 shutdown_connections 2024-03-12T04:18:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 1 -- 172.21.15.42:0/407876493 wait complete. 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.409+0000 7f08db3a9740 20 client.16473 trim_cache size 0 max 16384 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "132953ab-7816-43fd-90bc-8516fded045d": { 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "132953ab-7816-43fd-90bc-8516fded045d", 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:18:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:18:20.421 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:18:20.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f08bc0a1ba0 msgr2=0x7f08bc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005fd0 tx=0x7f08c0005850 comp rx=0 tx=0).stop 2024-03-12T04:18:20.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d019bfa0 msgr2=0x7f08d019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d019bfa0 0x7f08d019c450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f08c8004af0 tx=0x7f08c8035ca0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 shutdown_connections 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08d019bfa0 0x7f08d019c450 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d0106430 0x7f08d0197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 --2- 172.21.15.42:0/690480036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d0101360 0x7f08d0197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 >> 172.21.15.42:0/690480036 conn(0x7f08d00fcc20 msgr2=0x7f08d00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 shutdown_connections 2024-03-12T04:18:20.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:20.410+0000 7f08d86ff640 1 -- 172.21.15.42:0/690480036 wait complete. 2024-03-12T04:18:20.423 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:20 smithi042 ceph-mon[32260]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:18:20.423 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:20 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:18:20.423 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:20 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:18:20.423 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:20 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:18:20.423 INFO:tasks.cephfs.filesystem:scrub status for tag:132953ab-7816-43fd-90bc-8516fded045d - {'path': '/', 'tag': '132953ab-7816-43fd-90bc-8516fded045d', 'options': 'recursive,force'} 2024-03-12T04:18:20.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:20 smithi119 ceph-mon[36119]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:18:20.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:20 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:18:20.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:20 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:18:20.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:20 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:18:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:20 smithi136 ceph-mon[35571]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:18:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:20 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:18:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:20 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:18:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:20 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:18:21.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:21 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:18:21.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:21 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:21.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:21 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:18:21.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:21 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:21 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:18:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:21 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:21 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:18:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:21 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:21.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:21 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:18:21.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:21 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:21.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:21 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:18:21.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:21 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:22.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:22 smithi119 ceph-mon[36119]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:18:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:22 smithi136 ceph-mon[35571]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:18:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:22 smithi042 ceph-mon[32260]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:18:24.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:24 smithi119 ceph-mon[36119]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:18:24.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:24 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:18:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:24 smithi136 ceph-mon[35571]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:18:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:24 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:18:24.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:24 smithi042 ceph-mon[32260]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:18:24.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:24 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:18:26.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:26 smithi119 ceph-mon[36119]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:26 smithi136 ceph-mon[35571]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:26.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:26 smithi042 ceph-mon[32260]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:28.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:28 smithi119 ceph-mon[36119]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:28 smithi136 ceph-mon[35571]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:28.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:28 smithi042 ceph-mon[32260]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:30.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:30 smithi119 ceph-mon[36119]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:30.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:30 smithi136 ceph-mon[35571]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:30.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:30 smithi042 ceph-mon[32260]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:32.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:32 smithi119 ceph-mon[36119]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:32.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:32 smithi136 ceph-mon[35571]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:32.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:32 smithi042 ceph-mon[32260]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:18:34.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:34 smithi119 ceph-mon[36119]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:34 smithi136 ceph-mon[35571]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:34.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:34 smithi042 ceph-mon[32260]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:36 smithi119 ceph-mon[36119]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:36 smithi136 ceph-mon[35571]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:36.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:36 smithi042 ceph-mon[32260]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:38.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:38 smithi119 ceph-mon[36119]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:38.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:38 smithi136 ceph-mon[35571]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:38.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:38 smithi042 ceph-mon[32260]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:40.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:40 smithi119 ceph-mon[36119]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:40.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:40 smithi136 ceph-mon[35571]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:40.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:40 smithi042 ceph-mon[32260]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:42.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:42 smithi119 ceph-mon[36119]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:42 smithi136 ceph-mon[35571]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:42.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:42 smithi042 ceph-mon[32260]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:44.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:44 smithi119 ceph-mon[36119]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:44 smithi136 ceph-mon[35571]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:44 smithi042 ceph-mon[32260]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:46 smithi119 ceph-mon[36119]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:46 smithi136 ceph-mon[35571]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:46 smithi042 ceph-mon[32260]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:48 smithi119 ceph-mon[36119]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:48 smithi136 ceph-mon[35571]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:48.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:48 smithi042 ceph-mon[32260]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:50.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:18:50.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:18:50.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 -- 172.21.15.42:0/4260987978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210106430 msgr2=0x7f721010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 --2- 172.21.15.42:0/4260987978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210106430 0x7f721010e000 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7208009cd0 tx=0x7f720802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 -- 172.21.15.42:0/4260987978 shutdown_connections 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 --2- 172.21.15.42:0/4260987978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210106430 0x7f721010e000 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 --2- 172.21.15.42:0/4260987978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7210101d10 0x7f7210105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 --2- 172.21.15.42:0/4260987978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210101360 0x7f7210101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 -- 172.21.15.42:0/4260987978 >> 172.21.15.42:0/4260987978 conn(0x7f72100fcc20 msgr2=0x7f72100ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 -- 172.21.15.42:0/4260987978 shutdown_connections 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.541+0000 7f7216b56640 1 -- 172.21.15.42:0/4260987978 wait complete. 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 Processor -- start 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 -- start start 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210101360 0x7f7210073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7210106430 0x7f72100739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210073f00 0x7f72101a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101a5ef0 con 0x7f7210101360 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101a6060 con 0x7f7210073f00 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f7216b56640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101a6220 con 0x7f7210106430 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210101360 0x7f7210073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210101360 0x7f7210073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43344/0 (socket says 172.21.15.42:43344) 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 -- 172.21.15.42:0/3643454906 learned_addr learned my addr 172.21.15.42:0/3643454906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f720ffff640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7210106430 0x7f72100739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 -- 172.21.15.42:0/3643454906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7210106430 msgr2=0x7f72100739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72150f3640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210073f00 0x7f72101a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7210106430 0x7f72100739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 -- 172.21.15.42:0/3643454906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210073f00 msgr2=0x7f72101a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210073f00 0x7f72101a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72148f2640 1 -- 172.21.15.42:0/3643454906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7208009900 con 0x7f7210101360 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f720ffff640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7210106430 0x7f72100739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.542+0000 7f72150f3640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210073f00 0x7f72101a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.543+0000 7f72148f2640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210101360 0x7f7210073480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f720000d950 tx=0x7f720000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.543+0000 7f720dffb640 1 -- 172.21.15.42:0/3643454906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200009a30 con 0x7f7210101360 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.543+0000 7f720dffb640 1 -- 172.21.15.42:0/3643454906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f72000044d0 con 0x7f7210101360 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.543+0000 7f720dffb640 1 -- 172.21.15.42:0/3643454906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000047a0 con 0x7f7210101360 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.543+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72101a6500 con 0x7f7210101360 2024-03-12T04:18:50.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.543+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101a69c0 con 0x7f7210101360 2024-03-12T04:18:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f720dffb640 1 -- 172.21.15.42:0/3643454906 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7200009b90 con 0x7f7210101360 2024-03-12T04:18:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 Processor -- start 2024-03-12T04:18:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 -- start start 2024-03-12T04:18:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f720dffb640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f71f80a1c20 0x7f71f80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e16690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e16bd0 0x557d24e1f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e21ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24d85280 con 0x557d24e1f7b0 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24d827b0 con 0x557d24e162b0 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f7219808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24d7df90 con 0x557d24e16bd0 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f72150f3640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f71f80a1c20 0x7f71f80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f72148f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e16690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f720dffb640 1 -- 172.21.15.42:0/3643454906 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f72000cd220 con 0x7f7210101360 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f72148f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e16690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.42:55328/0 (socket says 172.21.15.42:55328) 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f72148f2640 1 -- 172.21.15.42:0/3239068969 learned_addr learned my addr 172.21.15.42:0/3239068969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.546+0000 7f72150f3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e21ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f720ffff640 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e16bd0 0x557d24e1f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f72148f2640 1 -- 172.21.15.42:0/3239068969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e16bd0 msgr2=0x557d24e1f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f72148f2640 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e16bd0 0x557d24e1f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f72148f2640 1 -- 172.21.15.42:0/3239068969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 msgr2=0x557d24e21ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f72148f2640 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e21ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f72148f2640 1 -- 172.21.15.42:0/3239068969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d24d3c570 con 0x557d24e162b0 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f72150f3640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f71f80a1c20 0x7f71f80a40e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7208004af0 tx=0x7f72080324b0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f72148f2640 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e16690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7200096300 tx=0x7f7200005130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b30aeb9a57dc870 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f71f6ffd640 1 -- 172.21.15.42:0/3239068969 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000b2bd0 con 0x557d24e162b0 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f71f6ffd640 1 -- 172.21.15.42:0/3239068969 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f720009a5a0 con 0x557d24e162b0 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.547+0000 7f71f6ffd640 1 -- 172.21.15.42:0/3239068969 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000b5520 con 0x557d24e162b0 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 -- 172.21.15.42:0/3239068969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 msgr2=0x557d24e16690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e16690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7200096300 tx=0x7f7200005130 comp rx=0 tx=0).stop 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 -- 172.21.15.42:0/3239068969 shutdown_connections 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e21ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e16bd0 0x557d24e1f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 --2- 172.21.15.42:0/3239068969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e16690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 -- 172.21.15.42:0/3239068969 >> 172.21.15.42:0/3239068969 conn(0x557d24e10990 msgr2=0x557d24e10d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 -- 172.21.15.42:0/3239068969 shutdown_connections 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 -- 172.21.15.42:0/3239068969 wait complete. 2024-03-12T04:18:50.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 -- start start 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.548+0000 7f7219808740 1 Processor -- start 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f7219808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e162b0 0x557d24e1f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f7219808740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f7219808740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e1ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f7219808740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24a51430 con 0x557d24e1f7b0 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f7219808740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24a0de00 con 0x557d24e16bd0 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f7219808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24d02fd0 con 0x557d24e162b0 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f7219808740 10 client.0 _ll_register_callbacks cb 0x557d24d58fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e1ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e1ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43366/0 (socket says 172.21.15.42:43366) 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 -- 172.21.15.42:0/1431003925 learned_addr learned my addr 172.21.15.42:0/1431003925 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72148f2640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e162b0 0x557d24e1f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f720ffff640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e1c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 -- 172.21.15.42:0/1431003925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e162b0 msgr2=0x557d24e1f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e162b0 0x557d24e1f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 -- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 msgr2=0x557d24e1c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e1c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f72150f3640 1 -- 172.21.15.42:0/1431003925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101a69c0 con 0x557d24e1f7b0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.549+0000 7f720ffff640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e1c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f72150f3640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e1ccd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7208032fe0 tx=0x7f7208045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33beb7a18d19718f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f71effff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f71effff640 1 -- 172.21.15.42:0/1431003925 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72080329e0 con 0x557d24e1f7b0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f71effff640 1 -- 172.21.15.42:0/1431003925 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7208003970 con 0x557d24e1f7b0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d24a7d7e0 con 0x557d24e1f7b0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f71effff640 1 -- 172.21.15.42:0/1431003925 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7208003b30 con 0x557d24e1f7b0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 10 client.16485 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557d24d7c610 con 0x557d24e1f7b0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f71effff640 1 -- 172.21.15.42:0/1431003925 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557d24a7d7e0 con 0x557d24e1f7b0 2024-03-12T04:18:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f71effff640 1 -- 172.21.15.42:0/1431003925 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7208005040 con 0x557d24e1f7b0 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 10 client.16485 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 20 client.16485 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e2fa50 0x557d24e31e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 4 client.16485 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.550+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d24aba720 con 0x557d24e2fa50 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.551+0000 7f72148f2640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e2fa50 0x557d24e31e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.551+0000 7f72148f2640 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e2fa50 0x557d24e31e70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5bd900b4984f18f server_cookie=5a2cd9070406b41e in_seq=0 out_seq=0 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.551+0000 7f71effff640 10 client.16485 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f71effff640 1 -- 172.21.15.42:0/1431003925 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d24aba720 con 0x557d24e2fa50 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f71effff640 10 client.16485 handle_command_reply: tid=0 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 client.16485 shutdown 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 msgr2=0x557d24e1ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e1ccd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7208032fe0 tx=0x7f7208045040 comp rx=0 tx=0).stop 2024-03-12T04:18:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 shutdown_connections 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e2fa50 0x557d24e31e70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e1f7b0 0x557d24e1ccd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e1c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 --2- 172.21.15.42:0/1431003925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e162b0 0x557d24e1f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f71effff640 0 client.16485 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 >> 172.21.15.42:0/1431003925 conn(0x557d24e10990 msgr2=0x557d24e12730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 shutdown_connections 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 1 -- 172.21.15.42:0/1431003925 wait complete. 2024-03-12T04:18:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.552+0000 7f7219808740 20 client.16485 trim_cache size 0 max 16384 2024-03-12T04:18:50.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 Processor -- start 2024-03-12T04:18:50.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 -- start start 2024-03-12T04:18:50.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e1ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e16bd0 0x557d24e1f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e1be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24aba750 con 0x557d24e16bd0 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24a7d810 con 0x557d24e162b0 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f7219808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24e0c0f0 con 0x557d24e1f7b0 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f720ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e16bd0 0x557d24e1f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f720ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e16bd0 0x557d24e1f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43370/0 (socket says 172.21.15.42:43370) 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.567+0000 7f720ffff640 1 -- 172.21.15.42:0/2697086878 learned_addr learned my addr 172.21.15.42:0/2697086878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f72150f3640 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e1be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f720ffff640 1 -- 172.21.15.42:0/2697086878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 msgr2=0x557d24e1be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f72148f2640 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e1ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f720ffff640 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e1be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f720ffff640 1 -- 172.21.15.42:0/2697086878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 msgr2=0x557d24e1ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f720ffff640 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e1ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f720ffff640 1 -- 172.21.15.42:0/2697086878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d24d18ac0 con 0x557d24e16bd0 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f72148f2640 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e1ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f720ffff640 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e16bd0 0x557d24e1f200 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7204009a00 tx=0x7f7204009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca4b48830008597 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f71f67fc640 1 -- 172.21.15.42:0/2697086878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204017070 con 0x557d24e16bd0 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f71f67fc640 1 -- 172.21.15.42:0/2697086878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7204005020 con 0x557d24e16bd0 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.568+0000 7f71f67fc640 1 -- 172.21.15.42:0/2697086878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204004ca0 con 0x557d24e16bd0 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- 172.21.15.42:0/2697086878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e16bd0 msgr2=0x557d24e1f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e16bd0 0x557d24e1f200 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7204009a00 tx=0x7f7204009ec0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- 172.21.15.42:0/2697086878 shutdown_connections 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e1be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e16bd0 0x557d24e1f200 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 --2- 172.21.15.42:0/2697086878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e162b0 0x557d24e1ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- 172.21.15.42:0/2697086878 >> 172.21.15.42:0/2697086878 conn(0x557d24e10990 msgr2=0x557d24e1ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- 172.21.15.42:0/2697086878 shutdown_connections 2024-03-12T04:18:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- 172.21.15.42:0/2697086878 wait complete. 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- start start 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 Processor -- start 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e162b0 0x557d24e26690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e23f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24e26f60 con 0x557d24e162b0 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24e244a0 con 0x557d24e16bd0 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.569+0000 7f7219808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d24e24660 con 0x557d24e1f7b0 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f7219808740 10 client.0 _ll_register_callbacks cb 0x7f72000046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e162b0 0x557d24e26690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e162b0 0x557d24e26690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43378/0 (socket says 172.21.15.42:43378) 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 -- 172.21.15.42:0/3995848090 learned_addr learned my addr 172.21.15.42:0/3995848090 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f720ffff640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72150f3640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e23f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 -- 172.21.15.42:0/3995848090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 msgr2=0x557d24e23f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e23f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 -- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 msgr2=0x557d24e23a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e23a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 -- 172.21.15.42:0/3995848090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72040096e0 con 0x557d24e162b0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72150f3640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e23f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f72148f2640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e162b0 0x557d24e26690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7200000c00 tx=0x7f720009a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1eb4f0ab7a70df server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f71effff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f71effff640 1 -- 172.21.15.42:0/3995848090 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720009dff0 con 0x557d24e162b0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f71effff640 1 -- 172.21.15.42:0/3995848090 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f72000b2bd0 con 0x557d24e162b0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f720009a5a0 con 0x557d24e162b0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f71effff640 1 -- 172.21.15.42:0/3995848090 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7200099ea0 con 0x557d24e162b0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f7219808740 10 client.16491 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.570+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557d24d3e6a0 con 0x557d24e162b0 2024-03-12T04:18:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f71effff640 1 -- 172.21.15.42:0/3995848090 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f720009a5a0 con 0x557d24e162b0 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f71effff640 1 -- 172.21.15.42:0/3995848090 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7200096510 con 0x557d24e162b0 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f7219808740 10 client.16491 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f7219808740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f7219808740 20 client.16491 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f7219808740 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e87e90 0x557d24e8a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f7219808740 4 client.16491 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557d24b09890 con 0x557d24e87e90 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.571+0000 7f720ffff640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e87e90 0x557d24e8a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f720ffff640 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e87e90 0x557d24e8a2b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13818bacedec3cab server_cookie=5b680d48aae19676 in_seq=0 out_seq=0 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f71effff640 10 client.16491 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f71effff640 1 -- 172.21.15.42:0/3995848090 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557d24b09890 con 0x557d24e87e90 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f71effff640 10 client.16491 handle_command_reply: tid=0 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 client.16491 shutdown 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e162b0 msgr2=0x557d24e26690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e162b0 0x557d24e26690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7200000c00 tx=0x7f720009a710 comp rx=0 tx=0).stop 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 shutdown_connections 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557d24e87e90 0x557d24e8a2b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d24e1f7b0 0x557d24e23f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f71effff640 0 client.16491 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d24e16bd0 0x557d24e23a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 --2- 172.21.15.42:0/3995848090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d24e162b0 0x557d24e26690 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 >> 172.21.15.42:0/3995848090 conn(0x557d24e10990 msgr2=0x557d24e1bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 shutdown_connections 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 1 -- 172.21.15.42:0/3995848090 wait complete. 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.572+0000 7f7219808740 20 client.16491 trim_cache size 0 max 16384 2024-03-12T04:18:50.581 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:18:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:18:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:18:50.582 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:18:50.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f71f80a1c20 msgr2=0x7f71f80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f71f80a1c20 0x7f71f80a40e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7208004af0 tx=0x7f72080324b0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210101360 msgr2=0x7f7210073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210101360 0x7f7210073480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f720000d950 tx=0x7f720000de10 comp rx=0 tx=0).stop 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 shutdown_connections 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f71f80a1c20 0x7f71f80a40e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210073f00 0x7f72101a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7210106430 0x7f72100739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 --2- 172.21.15.42:0/3643454906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7210101360 0x7f7210073480 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 >> 172.21.15.42:0/3643454906 conn(0x7f72100fcc20 msgr2=0x7f72100fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 shutdown_connections 2024-03-12T04:18:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.573+0000 7f7216b56640 1 -- 172.21.15.42:0/3643454906 wait complete. 2024-03-12T04:18:50.586 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:18:50.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:18:50.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.686+0000 7f29ed177640 1 -- 172.21.15.42:0/747080920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8102d10 msgr2=0x7f29e810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.686+0000 7f29ed177640 1 --2- 172.21.15.42:0/747080920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8102d10 0x7f29e810b1f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009900 tx=0x7f29dc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.686+0000 7f29ed177640 1 -- 172.21.15.42:0/747080920 shutdown_connections 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.686+0000 7f29ed177640 1 --2- 172.21.15.42:0/747080920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e810b7c0 0x7f29e810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.686+0000 7f29ed177640 1 --2- 172.21.15.42:0/747080920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8102d10 0x7f29e810b1f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.686+0000 7f29ed177640 1 --2- 172.21.15.42:0/747080920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e81023f0 0x7f29e81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.686+0000 7f29ed177640 1 -- 172.21.15.42:0/747080920 >> 172.21.15.42:0/747080920 conn(0x7f29e8077180 msgr2=0x7f29e8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.687+0000 7f29ed177640 1 -- 172.21.15.42:0/747080920 shutdown_connections 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.687+0000 7f29ed177640 1 -- 172.21.15.42:0/747080920 wait complete. 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.687+0000 7f29ed177640 1 Processor -- start 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.687+0000 7f29ed177640 1 -- start start 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29ed177640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e81023f0 0x7f29e8100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29ed177640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e8102d10 0x7f29e8100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29ed177640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e810b7c0 0x7f29e8100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29ed177640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81a5b00 con 0x7f29e810b7c0 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29ed177640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81a5c70 con 0x7f29e81023f0 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29ed177640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81a5e30 con 0x7f29e8102d10 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e810b7c0 0x7f29e8100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e810b7c0 0x7f29e8100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43396/0 (socket says 172.21.15.42:43396) 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 -- 172.21.15.42:0/3664269871 learned_addr learned my addr 172.21.15.42:0/3664269871 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e659c640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e8102d10 0x7f29e8100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e6d9d640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e81023f0 0x7f29e8100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 -- 172.21.15.42:0/3664269871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e8102d10 msgr2=0x7f29e8100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e8102d10 0x7f29e8100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 -- 172.21.15.42:0/3664269871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e81023f0 msgr2=0x7f29e8100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e81023f0 0x7f29e8100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 -- 172.21.15.42:0/3664269871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009560 con 0x7f29e810b7c0 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e659c640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e8102d10 0x7f29e8100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e6d9d640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e81023f0 0x7f29e8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29e759e640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e810b7c0 0x7f29e8100ac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f29e000cad0 tx=0x7f29e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29cffff640 1 -- 172.21.15.42:0/3664269871 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000ae20 con 0x7f29e810b7c0 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e81a6110 con 0x7f29e810b7c0 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.688+0000 7f29cffff640 1 -- 172.21.15.42:0/3664269871 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29e0005020 con 0x7f29e810b7c0 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.689+0000 7f29cffff640 1 -- 172.21.15.42:0/3664269871 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000f690 con 0x7f29e810b7c0 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.689+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e81a6670 con 0x7f29e810b7c0 2024-03-12T04:18:50.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.690+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e8067f70 con 0x7f29e810b7c0 2024-03-12T04:18:50.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:50 smithi119 ceph-mon[36119]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.691+0000 7f29cffff640 1 -- 172.21.15.42:0/3664269871 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f29e0005190 con 0x7f29e810b7c0 2024-03-12T04:18:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.691+0000 7f29cffff640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d00a1ba0 0x7f29d00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.691+0000 7f29e6d9d640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d00a1ba0 0x7f29d00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.691+0000 7f29cffff640 1 -- 172.21.15.42:0/3664269871 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f29e00cdc20 con 0x7f29e810b7c0 2024-03-12T04:18:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.692+0000 7f29e6d9d640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d00a1ba0 0x7f29d00a4060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009b40 tx=0x7f29d8005e50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.695+0000 7f29cffff640 1 -- 172.21.15.42:0/3664269871 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f29e0096b40 con 0x7f29e810b7c0 2024-03-12T04:18:50.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:50 smithi136 ceph-mon[35571]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:50.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:50 smithi042 ceph-mon[32260]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:50.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.914+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29e81068b0 con 0x7f29e810b7c0 2024-03-12T04:18:50.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.914+0000 7f29cffff640 1 -- 172.21.15.42:0/3664269871 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f29e001d020 con 0x7f29e810b7c0 2024-03-12T04:18:50.917 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:18:50.918 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:18:50.918 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:18:50.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d00a1ba0 msgr2=0x7f29d00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d00a1ba0 0x7f29d00a4060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009b40 tx=0x7f29d8005e50 comp rx=0 tx=0).stop 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e810b7c0 msgr2=0x7f29e8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e810b7c0 0x7f29e8100ac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f29e000cad0 tx=0x7f29e000cf90 comp rx=0 tx=0).stop 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 shutdown_connections 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d00a1ba0 0x7f29d00a4060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e810b7c0 0x7f29e8100ac0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e8102d10 0x7f29e8100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 --2- 172.21.15.42:0/3664269871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e81023f0 0x7f29e8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 >> 172.21.15.42:0/3664269871 conn(0x7f29e8077180 msgr2=0x7f29e80fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 shutdown_connections 2024-03-12T04:18:50.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:50.917+0000 7f29ed177640 1 -- 172.21.15.42:0/3664269871 wait complete. 2024-03-12T04:18:50.931 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:50.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:18:51.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 -- 172.21.15.42:0/3955387566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa320100c10 msgr2=0x7fa320104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 --2- 172.21.15.42:0/3955387566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa320100c10 0x7fa320104c50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa310009930 tx=0x7fa31002bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 -- 172.21.15.42:0/3955387566 shutdown_connections 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 --2- 172.21.15.42:0/3955387566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320105330 0x7fa32010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 --2- 172.21.15.42:0/3955387566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa320100c10 0x7fa320104c50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 --2- 172.21.15.42:0/3955387566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa320100260 0x7fa320100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 -- 172.21.15.42:0/3955387566 >> 172.21.15.42:0/3955387566 conn(0x7fa320077180 msgr2=0x7fa320077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 -- 172.21.15.42:0/3955387566 shutdown_connections 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 -- 172.21.15.42:0/3955387566 wait complete. 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 Processor -- start 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.033+0000 7fa327864640 1 -- start start 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320100260 0x7fa320197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa320105330 0x7fa320197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32019c010 0x7fa32019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32019ca00 con 0x7fa32019c010 2024-03-12T04:18:51.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32019cb70 con 0x7fa320105330 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32019cce0 con 0x7fa320100260 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32019c010 0x7fa32019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32019c010 0x7fa32019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43416/0 (socket says 172.21.15.42:43416) 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 -- 172.21.15.42:0/266230212 learned_addr learned my addr 172.21.15.42:0/266230212 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325600640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320100260 0x7fa320197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa324dff640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa320105330 0x7fa320197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 -- 172.21.15.42:0/266230212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320100260 msgr2=0x7fa320197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320100260 0x7fa320197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 -- 172.21.15.42:0/266230212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa320105330 msgr2=0x7fa320197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa320105330 0x7fa320197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 -- 172.21.15.42:0/266230212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa310009560 con 0x7fa32019c010 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa324dff640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa320105330 0x7fa320197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa325e01640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32019c010 0x7fa32019c4c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d750 tx=0x7fa31800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa3167fc640 1 -- 172.21.15.42:0/266230212 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318009bd0 con 0x7fa32019c010 2024-03-12T04:18:51.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3201a42c0 con 0x7fa32019c010 2024-03-12T04:18:51.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa3167fc640 1 -- 172.21.15.42:0/266230212 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa31800de60 con 0x7fa32019c010 2024-03-12T04:18:51.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa3167fc640 1 -- 172.21.15.42:0/266230212 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318004e00 con 0x7fa32019c010 2024-03-12T04:18:51.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.034+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3201a4770 con 0x7fa32019c010 2024-03-12T04:18:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.037+0000 7fa3167fc640 1 -- 172.21.15.42:0/266230212 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa318004f60 con 0x7fa32019c010 2024-03-12T04:18:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.037+0000 7fa32a436740 1 Processor -- start 2024-03-12T04:18:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.037+0000 7fa32a436740 1 -- start start 2024-03-12T04:18:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa32a436740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 0x559744571630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa32a436740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x55974457a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa32a436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x55974457cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa32a436740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974452c960 con 0x55974457a750 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa32a436740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597444e0310 con 0x559744571250 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa32a436740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597444dd840 con 0x559744571b70 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x55974457cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x55974457cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43432/0 (socket says 172.21.15.42:43432) 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 -- 172.21.15.42:0/3330437264 learned_addr learned my addr 172.21.15.42:0/3330437264 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 0x559744571630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa325600640 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x55974457a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 -- 172.21.15.42:0/3330437264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 msgr2=0x55974457a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x55974457a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 -- 172.21.15.42:0/3330437264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 msgr2=0x559744571630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 0x559744571630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa3167fc640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa30c0a1ba0 0x7fa30c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 -- 172.21.15.42:0/3330437264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55974449b060 con 0x55974457a750 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa325600640 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x55974457a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa325e01640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa30c0a1ba0 0x7fa30c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa324dff640 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x55974457cb40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa31002c710 tx=0x7fa310000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bb9a60083a7a2a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.038+0000 7fa3167fc640 1 -- 172.21.15.42:0/266230212 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa3180cd0b0 con 0x7fa32019c010 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa30b7fe640 1 -- 172.21.15.42:0/3330437264 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310004030 con 0x55974457a750 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa30b7fe640 1 -- 172.21.15.42:0/3330437264 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa31002de90 con 0x55974457a750 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa30b7fe640 1 -- 172.21.15.42:0/3330437264 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310004dc0 con 0x55974457a750 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 -- 172.21.15.42:0/3330437264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 msgr2=0x55974457cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa325e01640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa30c0a1ba0 0x7fa30c0a4060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3180ca900 tx=0x7fa3180c43f0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x55974457cb40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa31002c710 tx=0x7fa310000bc0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 -- 172.21.15.42:0/3330437264 shutdown_connections 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x55974457cb40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x55974457a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 --2- 172.21.15.42:0/3330437264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 0x559744571630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 -- 172.21.15.42:0/3330437264 >> 172.21.15.42:0/3330437264 conn(0x55974456b980 msgr2=0x55974456bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 -- 172.21.15.42:0/3330437264 shutdown_connections 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.039+0000 7fa32a436740 1 -- 172.21.15.42:0/3330437264 wait complete. 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 -- start start 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 Processor -- start 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744577810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571b70 0x559744577d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744578290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597444d9020 con 0x55974457a750 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597441ac430 con 0x559744571b70 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559744168e00 con 0x559744571250 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa32a436740 10 client.0 _ll_register_callbacks cb 0x559744480cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa324dff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744578290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa324dff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744578290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43448/0 (socket says 172.21.15.42:43448) 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.040+0000 7fa324dff640 1 -- 172.21.15.42:0/1343954716 learned_addr learned my addr 172.21.15.42:0/1343954716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa325600640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571b70 0x559744577d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa324dff640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744577810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa324dff640 1 -- 172.21.15.42:0/1343954716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 msgr2=0x559744577810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa324dff640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744577810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa324dff640 1 -- 172.21.15.42:0/1343954716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571b70 msgr2=0x559744577d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa324dff640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571b70 0x559744577d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa324dff640 1 -- 172.21.15.42:0/1343954716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32010d710 con 0x55974457a750 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa325600640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571b70 0x559744577d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa324dff640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744578290 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa31002c170 tx=0x7fa31002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcfc6c39cc99517 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa308ff9640 1 -- 172.21.15.42:0/1343954716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310005120 con 0x55974457a750 2024-03-12T04:18:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa308ff9640 1 -- 172.21.15.42:0/1343954716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa310000e40 con 0x55974457a750 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597441d87e0 con 0x55974457a750 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa308ff9640 1 -- 172.21.15.42:0/1343954716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa310003900 con 0x55974457a750 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa32a436740 10 client.16509 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5597444d76a0 con 0x55974457a750 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa308ff9640 1 -- 172.21.15.42:0/1343954716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5597441d87e0 con 0x55974457a750 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.041+0000 7fa308ff9640 1 -- 172.21.15.42:0/1343954716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa310043450 con 0x55974457a750 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa32a436740 10 client.16509 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa32a436740 10 client.16509 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa32a436740 20 client.16509 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa32a436740 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55974458a890 0x55974458ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa32a436740 4 client.16509 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559744215720 con 0x55974458a890 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa325600640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55974458a890 0x55974458ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa308ff9640 10 client.16509 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa325600640 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55974458a890 0x55974458ccb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee70f190b84792a server_cookie=1ed3844da5d889d2 in_seq=0 out_seq=0 2024-03-12T04:18:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa308ff9640 1 -- 172.21.15.42:0/1343954716 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559744215720 con 0x55974458a890 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.042+0000 7fa308ff9640 10 client.16509 handle_command_reply: tid=0 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 client.16509 shutdown 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 msgr2=0x559744578290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744578290 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa31002c170 tx=0x7fa31002bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 shutdown_connections 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55974458a890 0x55974458ccb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744578290 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571b70 0x559744577d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa308ff9640 0 client.16509 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 --2- 172.21.15.42:0/1343954716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744577810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 >> 172.21.15.42:0/1343954716 conn(0x55974456b980 msgr2=0x55974456d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 shutdown_connections 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 1 -- 172.21.15.42:0/1343954716 wait complete. 2024-03-12T04:18:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.043+0000 7fa32a436740 20 client.16509 trim_cache size 0 max 16384 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 Processor -- start 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 -- start start 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 0x559744579f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x559744572a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744572fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974445e130 con 0x55974457a750 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974458a690 con 0x559744571250 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa32a436740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974456c9b0 con 0x559744571b70 2024-03-12T04:18:51.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744572fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744572fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43462/0 (socket says 172.21.15.42:43462) 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 -- 172.21.15.42:0/1953087261 learned_addr learned my addr 172.21.15.42:0/1953087261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa325600640 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x559744572a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 -- 172.21.15.42:0/1953087261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 msgr2=0x559744572a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x559744572a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 -- 172.21.15.42:0/1953087261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 msgr2=0x559744579f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 0x559744579f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 -- 172.21.15.42:0/1953087261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559744473c20 con 0x55974457a750 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa324dff640 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744572fd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa310003b70 tx=0x7fa31002c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9456005ae30d0f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.055+0000 7fa30affd640 1 -- 172.21.15.42:0/1953087261 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310003e00 con 0x55974457a750 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa30affd640 1 -- 172.21.15.42:0/1953087261 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa31002c5d0 con 0x55974457a750 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa30affd640 1 -- 172.21.15.42:0/1953087261 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3100436b0 con 0x55974457a750 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 -- 172.21.15.42:0/1953087261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 msgr2=0x559744572fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744572fd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa310003b70 tx=0x7fa31002c800 comp rx=0 tx=0).stop 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 -- 172.21.15.42:0/1953087261 shutdown_connections 2024-03-12T04:18:51.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55974457a750 0x559744572fd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571b70 0x559744572a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 --2- 172.21.15.42:0/1953087261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559744571250 0x559744579f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 -- 172.21.15.42:0/1953087261 >> 172.21.15.42:0/1953087261 conn(0x55974456b980 msgr2=0x559744566a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 -- 172.21.15.42:0/1953087261 shutdown_connections 2024-03-12T04:18:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 -- 172.21.15.42:0/1953087261 wait complete. 2024-03-12T04:18:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.056+0000 7fa32a436740 1 -- start start 2024-03-12T04:18:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 1 Processor -- start 2024-03-12T04:18:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744581af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559744571b70 0x55974457fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974457a750 0x5597445805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559744580ca0 con 0x559744571b70 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597445e0d60 con 0x55974457a750 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597445e0ea0 con 0x559744571250 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa32a436740 10 client.0 _ll_register_callbacks cb 0x7fa310009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559744571b70 0x55974457fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559744571b70 0x55974457fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43466/0 (socket says 172.21.15.42:43466) 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 -- 172.21.15.42:0/4227369390 learned_addr learned my addr 172.21.15.42:0/4227369390 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa324dff640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744581af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa324dff640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974457a750 0x5597445805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 -- 172.21.15.42:0/4227369390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 msgr2=0x559744581af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744581af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 -- 172.21.15.42:0/4227369390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974457a750 msgr2=0x5597445805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974457a750 0x5597445805d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa325600640 1 -- 172.21.15.42:0/4227369390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597444d76a0 con 0x559744571b70 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.057+0000 7fa324dff640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744581af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa325600640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559744571b70 0x55974457fe90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55974458d580 tx=0x7fa31c011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3bda8088204fb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa308ff9640 1 -- 172.21.15.42:0/4227369390 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c016040 con 0x559744571b70 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa31002de90 con 0x559744571b70 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa308ff9640 1 -- 172.21.15.42:0/4227369390 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa31c01d890 con 0x559744571b70 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa308ff9640 1 -- 172.21.15.42:0/4227369390 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa31c004b20 con 0x559744571b70 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 10 client.16515 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5597444998c0 con 0x559744571b70 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa308ff9640 1 -- 172.21.15.42:0/4227369390 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa31002de90 con 0x559744571b70 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa308ff9640 1 -- 172.21.15.42:0/4227369390 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa31c0052e0 con 0x559744571b70 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 10 client.16515 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 10 client.16515 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 20 client.16515 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5597445e2730 0x5597445e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 4 client.16515 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559744264890 con 0x5597445e2730 2024-03-12T04:18:51.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.058+0000 7fa324dff640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5597445e2730 0x5597445e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa308ff9640 10 client.16515 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa324dff640 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5597445e2730 0x5597445e4b50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8c8a99e78d0e43b server_cookie=987024cc7341b085 in_seq=0 out_seq=0 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa308ff9640 1 -- 172.21.15.42:0/4227369390 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559744264890 con 0x5597445e2730 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa308ff9640 10 client.16515 handle_command_reply: tid=0 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 client.16515 shutdown 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559744571b70 msgr2=0x55974457fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559744571b70 0x55974457fe90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55974458d580 tx=0x7fa31c011d20 comp rx=0 tx=0).stop 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 shutdown_connections 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974457a750 0x5597445805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5597445e2730 0x5597445e4b50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559744571b70 0x55974457fe90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa308ff9640 0 client.16515 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:18:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 --2- 172.21.15.42:0/4227369390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559744571250 0x559744581af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 >> 172.21.15.42:0/4227369390 conn(0x55974456b980 msgr2=0x559744572a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 shutdown_connections 2024-03-12T04:18:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 1 -- 172.21.15.42:0/4227369390 wait complete. 2024-03-12T04:18:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.059+0000 7fa32a436740 20 client.16515 trim_cache size 0 max 16384 2024-03-12T04:18:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:51.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa30c0a1ba0 msgr2=0x7fa30c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa30c0a1ba0 0x7fa30c0a4060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3180ca900 tx=0x7fa3180c43f0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32019c010 msgr2=0x7fa32019c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32019c010 0x7fa32019c4c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d750 tx=0x7fa31800dc10 comp rx=0 tx=0).stop 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 shutdown_connections 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa30c0a1ba0 0x7fa30c0a4060 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32019c010 0x7fa32019c4c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa320105330 0x7fa320197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 --2- 172.21.15.42:0/266230212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320100260 0x7fa320197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 >> 172.21.15.42:0/266230212 conn(0x7fa320077180 msgr2=0x7fa320104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 shutdown_connections 2024-03-12T04:18:51.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.060+0000 7fa327864640 1 -- 172.21.15.42:0/266230212 wait complete. 2024-03-12T04:18:51.073 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:51.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:18:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 -- 172.21.15.42:0/127049112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c102d10 msgr2=0x7f397c10b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 --2- 172.21.15.42:0/127049112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c102d10 0x7f397c10b1f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f396c009900 tx=0x7f396c02bcd0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 -- 172.21.15.42:0/127049112 shutdown_connections 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 --2- 172.21.15.42:0/127049112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f397c10b7c0 0x7f397c10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 --2- 172.21.15.42:0/127049112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c102d10 0x7f397c10b1f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 --2- 172.21.15.42:0/127049112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f397c1023f0 0x7f397c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 -- 172.21.15.42:0/127049112 >> 172.21.15.42:0/127049112 conn(0x7f397c0771e0 msgr2=0x7f397c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 -- 172.21.15.42:0/127049112 shutdown_connections 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.175+0000 7f398289d640 1 -- 172.21.15.42:0/127049112 wait complete. 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 Processor -- start 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 -- start start 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f397c1023f0 0x7f397c100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f397c102d10 0x7f397c100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c10b7c0 0x7f397c100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1a5af0 con 0x7f397c10b7c0 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1a5c60 con 0x7f397c102d10 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f398289d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1a5e20 con 0x7f397c1023f0 2024-03-12T04:18:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f3980e3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c10b7c0 0x7f397c100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f3980e3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c10b7c0 0x7f397c100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43492/0 (socket says 172.21.15.42:43492) 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f3980e3a640 1 -- 172.21.15.42:0/1446804693 learned_addr learned my addr 172.21.15.42:0/1446804693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f397bfff640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f397c1023f0 0x7f397c100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.176+0000 7f397b7fe640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f397c102d10 0x7f397c100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f3980e3a640 1 -- 172.21.15.42:0/1446804693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f397c1023f0 msgr2=0x7f397c100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f3980e3a640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f397c1023f0 0x7f397c100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f3980e3a640 1 -- 172.21.15.42:0/1446804693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f397c102d10 msgr2=0x7f397c100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f3980e3a640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f397c102d10 0x7f397c100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f3980e3a640 1 -- 172.21.15.42:0/1446804693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c009560 con 0x7f397c10b7c0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f3980e3a640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c10b7c0 0x7f397c100ac0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f397400cad0 tx=0x7f397400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f39797fa640 1 -- 172.21.15.42:0/1446804693 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400ae20 con 0x7f397c10b7c0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f39797fa640 1 -- 172.21.15.42:0/1446804693 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3974005020 con 0x7f397c10b7c0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c1a6100 con 0x7f397c10b7c0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f39797fa640 1 -- 172.21.15.42:0/1446804693 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400f690 con 0x7f397c10b7c0 2024-03-12T04:18:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.177+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c1a6610 con 0x7f397c10b7c0 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.179+0000 7f39797fa640 1 -- 172.21.15.42:0/1446804693 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3974005190 con 0x7f397c10b7c0 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.179+0000 7f398547b740 1 Processor -- start 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.179+0000 7f398547b740 1 -- start start 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f398547b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb3036630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f398547b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f398547b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb3041b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f398547b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb2ff1960 con 0x560bb303f750 2024-03-12T04:18:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f398547b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb2fa5310 con 0x560bb3036b70 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f398547b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb2fa2840 con 0x560bb3036250 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f39797fa640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39680a1ba0 0x7f39680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb3041b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb3041b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43508/0 (socket says 172.21.15.42:43508) 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 -- 172.21.15.42:0/781376856 learned_addr learned my addr 172.21.15.42:0/781376856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f3980e3a640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39680a1ba0 0x7f39680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb3036630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397bfff640 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f39797fa640 1 -- 172.21.15.42:0/1446804693 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f39740cd770 con 0x7f397c10b7c0 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 -- 172.21.15.42:0/781376856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 msgr2=0x560bb3036630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb3036630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 -- 172.21.15.42:0/781376856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 msgr2=0x560bb303f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 -- 172.21.15.42:0/781376856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bb2f60060 con 0x560bb303f750 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f3980e3a640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39680a1ba0 0x7f39680a4060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3974096850 tx=0x7f3974085040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.180+0000 7f397b7fe640 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb3041b40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f396c02c210 tx=0x7f396c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b838bd6249c698 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f39727fc640 1 -- 172.21.15.42:0/781376856 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c02cc80 con 0x560bb303f750 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f39727fc640 1 -- 172.21.15.42:0/781376856 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f396c0044f0 con 0x560bb303f750 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f39727fc640 1 -- 172.21.15.42:0/781376856 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c004dc0 con 0x560bb303f750 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 -- 172.21.15.42:0/781376856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 msgr2=0x560bb3041b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb3041b40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f396c02c210 tx=0x7f396c0058b0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 -- 172.21.15.42:0/781376856 shutdown_connections 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb3041b40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 --2- 172.21.15.42:0/781376856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb3036630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 -- 172.21.15.42:0/781376856 >> 172.21.15.42:0/781376856 conn(0x560bb3030980 msgr2=0x560bb3030d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 -- 172.21.15.42:0/781376856 shutdown_connections 2024-03-12T04:18:51.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 -- 172.21.15.42:0/781376856 wait complete. 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.181+0000 7f398547b740 1 -- start start 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 1 Processor -- start 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb303c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb303d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb2f9e020 con 0x560bb303f750 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb2c71430 con 0x560bb3036b70 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb2c2de00 con 0x560bb3036250 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f398547b740 10 client.0 _ll_register_callbacks cb 0x560bb2f45cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb303d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb303d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43518/0 (socket says 172.21.15.42:43518) 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 -- 172.21.15.42:0/3269353676 learned_addr learned my addr 172.21.15.42:0/3269353676 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb303c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397bfff640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 -- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 msgr2=0x560bb303c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb303c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 -- 172.21.15.42:0/3269353676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 msgr2=0x560bb303cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397b7fe640 1 -- 172.21.15.42:0/3269353676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c10e8b0 con 0x560bb303f750 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.182+0000 7f397bfff640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f397b7fe640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb303d250 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f396c003c20 tx=0x7f396c046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c045b0673d753d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f395b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f395b7fe640 1 -- 172.21.15.42:0/3269353676 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c004200 con 0x560bb303f750 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f395b7fe640 1 -- 172.21.15.42:0/3269353676 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f396c004dc0 con 0x560bb303f750 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bb2c9d7e0 con 0x560bb303f750 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f395b7fe640 1 -- 172.21.15.42:0/3269353676 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f396c004360 con 0x560bb303f750 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 10 client.16527 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560bb2f9c6a0 con 0x560bb303f750 2024-03-12T04:18:51.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f395b7fe640 1 -- 172.21.15.42:0/3269353676 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x560bb2c9d7e0 con 0x560bb303f750 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f395b7fe640 1 -- 172.21.15.42:0/3269353676 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f396c0405a0 con 0x560bb303f750 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 10 client.16527 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 10 client.16527 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 20 client.16527 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb304f850 0x560bb3051c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 4 client.16527 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.183+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bb2cda720 con 0x560bb304f850 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.184+0000 7f397bfff640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb304f850 0x560bb3051c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.184+0000 7f395b7fe640 10 client.16527 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.184+0000 7f397bfff640 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb304f850 0x560bb3051c70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf3ec971693e20ab server_cookie=b5e82018dd88da02 in_seq=0 out_seq=0 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f395b7fe640 1 -- 172.21.15.42:0/3269353676 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560bb2cda720 con 0x560bb304f850 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f395b7fe640 10 client.16527 handle_command_reply: tid=0 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 client.16527 shutdown 2024-03-12T04:18:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 msgr2=0x560bb303d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb303d250 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f396c003c20 tx=0x7f396c046040 comp rx=0 tx=0).stop 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 shutdown_connections 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb304f850 0x560bb3051c70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb303f750 0x560bb303d250 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb303cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 --2- 172.21.15.42:0/3269353676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036250 0x560bb303c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f395b7fe640 0 client.16527 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 >> 172.21.15.42:0/3269353676 conn(0x560bb3030980 msgr2=0x560bb30326d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 shutdown_connections 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 1 -- 172.21.15.42:0/3269353676 wait complete. 2024-03-12T04:18:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.185+0000 7f398547b740 20 client.16527 trim_cache size 0 max 16384 2024-03-12T04:18:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 Processor -- start 2024-03-12T04:18:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 -- start start 2024-03-12T04:18:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb303ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb302ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb303f750 0x560bb3037a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb2f23130 con 0x560bb3036250 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb3031980 con 0x560bb3036b70 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f398547b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb3056e40 con 0x560bb303f750 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb303ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb303ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43524/0 (socket says 172.21.15.42:43524) 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 -- 172.21.15.42:0/2507064861 learned_addr learned my addr 172.21.15.42:0/2507064861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb303f750 0x560bb3037a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397bfff640 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb302ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 -- 172.21.15.42:0/2507064861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb303f750 msgr2=0x560bb3037a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb303f750 0x560bb3037a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 -- 172.21.15.42:0/2507064861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 msgr2=0x560bb302ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb302ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.197+0000 7f397b7fe640 1 -- 172.21.15.42:0/2507064861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bb2f38c20 con 0x560bb3036250 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f397b7fe640 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb303ef00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f396c003c90 tx=0x7f396c02dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241cc553a94b6cfa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f3971ffb640 1 -- 172.21.15.42:0/2507064861 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c040d70 con 0x560bb3036250 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f3971ffb640 1 -- 172.21.15.42:0/2507064861 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f396c059030 con 0x560bb3036250 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f3971ffb640 1 -- 172.21.15.42:0/2507064861 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0406e0 con 0x560bb3036250 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 -- 172.21.15.42:0/2507064861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 msgr2=0x560bb303ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb303ef00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f396c003c90 tx=0x7f396c02dde0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 -- 172.21.15.42:0/2507064861 shutdown_connections 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb303f750 0x560bb3037a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb3036b70 0x560bb302ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 --2- 172.21.15.42:0/2507064861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb303ef00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 -- 172.21.15.42:0/2507064861 >> 172.21.15.42:0/2507064861 conn(0x560bb3030980 msgr2=0x560bb3036f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 -- 172.21.15.42:0/2507064861 shutdown_connections 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.198+0000 7f398547b740 1 -- 172.21.15.42:0/2507064861 wait complete. 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 -- start start 2024-03-12T04:18:51.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 Processor -- start 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb3046710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036b70 0x560bb3044ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb303f750 0x560bb3044ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb3045530 con 0x560bb3036250 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb30456a0 con 0x560bb303f750 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bb3045810 con 0x560bb3036b70 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f398547b740 10 client.0 _ll_register_callbacks cb 0x7f396c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb3046710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb3046710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43532/0 (socket says 172.21.15.42:43532) 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f397b7fe640 1 -- 172.21.15.42:0/1742662813 learned_addr learned my addr 172.21.15.42:0/1742662813 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f397bfff640 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036b70 0x560bb3044ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f397b7fe640 1 -- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036b70 msgr2=0x560bb3044ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f397b7fe640 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036b70 0x560bb3044ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.199+0000 7f397b7fe640 1 -- 172.21.15.42:0/1742662813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb303f750 msgr2=0x560bb3044ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f397b7fe640 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb303f750 0x560bb3044ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f397b7fe640 1 -- 172.21.15.42:0/1742662813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bb2f9c6a0 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f397b7fe640 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb3046710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f396c000c00 tx=0x7f396c02ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc67593a188c2cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f395b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f395b7fe640 1 -- 172.21.15.42:0/1742662813 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c003a50 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f395b7fe640 1 -- 172.21.15.42:0/1742662813 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f396c000e40 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bb303c600 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f395b7fe640 1 -- 172.21.15.42:0/1742662813 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f396c0046c0 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 10 client.16533 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560bb2f5e8c0 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f395b7fe640 1 -- 172.21.15.42:0/1742662813 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x560bb303c600 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f395b7fe640 1 -- 172.21.15.42:0/1742662813 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f396c066040 con 0x560bb3036250 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 10 client.16533 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 10 client.16533 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 20 client.16533 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb30a6da0 0x560bb30a9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 4 client.16533 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.200+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f396c0044f0 con 0x560bb30a6da0 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f397bfff640 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb30a6da0 0x560bb30a9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f397bfff640 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb30a6da0 0x560bb30a9190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af13bbb234c3183a server_cookie=4584d42b90f8e4fb in_seq=0 out_seq=0 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f395b7fe640 10 client.16533 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f395b7fe640 1 -- 172.21.15.42:0/1742662813 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f396c0044f0 con 0x560bb30a6da0 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f395b7fe640 10 client.16533 handle_command_reply: tid=0 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 client.16533 shutdown 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 msgr2=0x560bb3046710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb3046710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f396c000c00 tx=0x7f396c02ca40 comp rx=0 tx=0).stop 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 shutdown_connections 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560bb30a6da0 0x560bb30a9190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bb303f750 0x560bb3044ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f395b7fe640 0 client.16533 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bb3036b70 0x560bb3044ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 --2- 172.21.15.42:0/1742662813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bb3036250 0x560bb3046710 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 >> 172.21.15.42:0/1742662813 conn(0x560bb3030980 msgr2=0x560bb302ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 shutdown_connections 2024-03-12T04:18:51.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.201+0000 7f398547b740 1 -- 172.21.15.42:0/1742662813 wait complete. 2024-03-12T04:18:51.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398547b740 20 client.16533 trim_cache size 0 max 16384 2024-03-12T04:18:51.214 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:51.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39680a1ba0 msgr2=0x7f39680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39680a1ba0 0x7f39680a4060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3974096850 tx=0x7f3974085040 comp rx=0 tx=0).stop 2024-03-12T04:18:51.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c10b7c0 msgr2=0x7f397c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c10b7c0 0x7f397c100ac0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f397400cad0 tx=0x7f397400cf90 comp rx=0 tx=0).stop 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 shutdown_connections 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f39680a1ba0 0x7f39680a4060 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f397c10b7c0 0x7f397c100ac0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f397c102d10 0x7f397c100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 --2- 172.21.15.42:0/1446804693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f397c1023f0 0x7f397c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 >> 172.21.15.42:0/1446804693 conn(0x7f397c0771e0 msgr2=0x7f397c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 shutdown_connections 2024-03-12T04:18:51.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.202+0000 7f398289d640 1 -- 172.21.15.42:0/1446804693 wait complete. 2024-03-12T04:18:51.217 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:51.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:18:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 -- 172.21.15.42:0/165551433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 msgr2=0x7fd620105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 --2- 172.21.15.42:0/165551433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd620105400 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd610009930 tx=0x7fd61002bc30 comp rx=0 tx=0).stop 2024-03-12T04:18:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 -- 172.21.15.42:0/165551433 shutdown_connections 2024-03-12T04:18:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 --2- 172.21.15.42:0/165551433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd620101550 0x7fd62010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 --2- 172.21.15.42:0/165551433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd620100b90 0x7fd620101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 --2- 172.21.15.42:0/165551433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd620105400 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 -- 172.21.15.42:0/165551433 >> 172.21.15.42:0/165551433 conn(0x7fd6200752e0 msgr2=0x7fd6200756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 -- 172.21.15.42:0/165551433 shutdown_connections 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.319+0000 7fd6255ac640 1 -- 172.21.15.42:0/165551433 wait complete. 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 Processor -- start 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 -- start start 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd620100b90 0x7fd6201965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd620101550 0x7fd620196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd62019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019b460 con 0x7fd620105020 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019b5d0 con 0x7fd620101550 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd6255ac640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019b790 con 0x7fd620100b90 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd62019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd62019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43548/0 (socket says 172.21.15.42:43548) 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 -- 172.21.15.42:0/201407866 learned_addr learned my addr 172.21.15.42:0/201407866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61effd640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd620100b90 0x7fd6201965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd617fff640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd620101550 0x7fd620196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 -- 172.21.15.42:0/201407866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd620100b90 msgr2=0x7fd6201965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd620100b90 0x7fd6201965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 -- 172.21.15.42:0/201407866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd620101550 msgr2=0x7fd620196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd620101550 0x7fd620196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.320+0000 7fd61f7fe640 1 -- 172.21.15.42:0/201407866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610009560 con 0x7fd620105020 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.321+0000 7fd61f7fe640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd62019ae60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd61800eaa0 tx=0x7fd61800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.321+0000 7fd61cff9640 1 -- 172.21.15.42:0/201407866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61800cdb0 con 0x7fd620105020 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.321+0000 7fd61cff9640 1 -- 172.21.15.42:0/201407866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd618005020 con 0x7fd620105020 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.321+0000 7fd61cff9640 1 -- 172.21.15.42:0/201407866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180116b0 con 0x7fd620105020 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.321+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62019ba70 con 0x7fd620105020 2024-03-12T04:18:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.321+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6200ffc60 con 0x7fd620105020 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd61cff9640 1 -- 172.21.15.42:0/201407866 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd618005190 con 0x7fd620105020 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 Processor -- start 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 -- start start 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81c1750 0x564cb81c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8173960 con 0x564cb81b8250 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8127310 con 0x564cb81b8b70 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd62817f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8124840 con 0x564cb81c1750 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43558/0 (socket says 172.21.15.42:43558) 2024-03-12T04:18:51.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd617fff640 1 -- 172.21.15.42:0/1927822338 learned_addr learned my addr 172.21.15.42:0/1927822338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd61cff9640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd60c0a1ba0 0x7fd60c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd61effd640 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.324+0000 7fd61f7fe640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd60c0a1ba0 0x7fd60c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd617fff640 1 -- 172.21.15.42:0/1927822338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81c1750 msgr2=0x564cb81c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd617fff640 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81c1750 0x564cb81c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd61cff9640 1 -- 172.21.15.42:0/201407866 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd6180cd860 con 0x7fd620105020 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd617fff640 1 -- 172.21.15.42:0/1927822338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 msgr2=0x564cb81c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd617fff640 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd617fff640 1 -- 172.21.15.42:0/1927822338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb80e2060 con 0x564cb81b8250 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd617fff640 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81b8630 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd60800bdc0 tx=0x7fd6080127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0a0f61b6cb6bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd61f7fe640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd60c0a1ba0 0x7fd60c0a4060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cb060 tx=0x7fd618085040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd6157fa640 1 -- 172.21.15.42:0/1927822338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800beb0 con 0x564cb81b8250 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd6157fa640 1 -- 172.21.15.42:0/1927822338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6080054d0 con 0x564cb81b8250 2024-03-12T04:18:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.325+0000 7fd6157fa640 1 -- 172.21.15.42:0/1927822338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608010430 con 0x564cb81b8250 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- 172.21.15.42:0/1927822338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 msgr2=0x564cb81b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81b8630 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd60800bdc0 tx=0x7fd6080127c0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- 172.21.15.42:0/1927822338 shutdown_connections 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81c1750 0x564cb81c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 --2- 172.21.15.42:0/1927822338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81b8630 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- 172.21.15.42:0/1927822338 >> 172.21.15.42:0/1927822338 conn(0x564cb81b2980 msgr2=0x564cb81b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- 172.21.15.42:0/1927822338 shutdown_connections 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- 172.21.15.42:0/1927822338 wait complete. 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- start start 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 Processor -- start 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 0x564cb81be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81becf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8120020 con 0x564cb81c1750 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb7df3430 con 0x564cb81b8b70 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb7dafe00 con 0x564cb81b8250 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.326+0000 7fd62817f740 10 client.0 _ll_register_callbacks cb 0x564cb80c7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43570/0 (socket says 172.21.15.42:43570) 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 -- 172.21.15.42:0/3239175169 learned_addr learned my addr 172.21.15.42:0/3239175169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 0x564cb81be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd61effd640 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 -- 172.21.15.42:0/3239175169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 msgr2=0x564cb81be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 0x564cb81be7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 -- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 msgr2=0x564cb81becf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81becf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 -- 172.21.15.42:0/3239175169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60800ba70 con 0x564cb81c1750 2024-03-12T04:18:51.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd617fff640 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bf230 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd608005fd0 tx=0x7fd608014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965ed0c47c3f4c16 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd6067fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd6067fc640 1 -- 172.21.15.42:0/3239175169 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080121d0 con 0x564cb81c1750 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd6067fc640 1 -- 172.21.15.42:0/3239175169 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd6080048d0 con 0x564cb81c1750 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cb7e1f7e0 con 0x564cb81c1750 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd6067fc640 1 -- 172.21.15.42:0/3239175169 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd608005000 con 0x564cb81c1750 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd62817f740 10 client.16545 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564cb811e6a0 con 0x564cb81c1750 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.327+0000 7fd6067fc640 1 -- 172.21.15.42:0/3239175169 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564cb7e1f7e0 con 0x564cb81c1750 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd6067fc640 1 -- 172.21.15.42:0/3239175169 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd608010430 con 0x564cb81c1750 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd62817f740 10 client.16545 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd62817f740 10 client.16545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd62817f740 20 client.16545 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd62817f740 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb81d1a90 0x564cb81d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd62817f740 4 client.16545 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cb7e5c720 con 0x564cb81d1a90 2024-03-12T04:18:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd61effd640 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb81d1a90 0x564cb81d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd61effd640 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb81d1a90 0x564cb81d3e80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52950438a03d7d23 server_cookie=e99bac22b0192b3b in_seq=0 out_seq=0 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.328+0000 7fd6067fc640 10 client.16545 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd6067fc640 1 -- 172.21.15.42:0/3239175169 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cb7e5c720 con 0x564cb81d1a90 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd6067fc640 10 client.16545 handle_command_reply: tid=0 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 client.16545 shutdown 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 msgr2=0x564cb81bf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bf230 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd608005fd0 tx=0x7fd608014040 comp rx=0 tx=0).stop 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 shutdown_connections 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb81d1a90 0x564cb81d3e80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bf230 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81becf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd6067fc640 0 client.16545 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 --2- 172.21.15.42:0/3239175169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 0x564cb81be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 >> 172.21.15.42:0/3239175169 conn(0x564cb81b2980 msgr2=0x564cb81b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 shutdown_connections 2024-03-12T04:18:51.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 1 -- 172.21.15.42:0/3239175169 wait complete. 2024-03-12T04:18:51.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.329+0000 7fd62817f740 20 client.16545 trim_cache size 0 max 16384 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 Processor -- start 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 -- start start 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 0x564cb81c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb80a5130 con 0x564cb81c1750 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb81ba3e0 con 0x564cb81b8b70 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd62817f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb81ce690 con 0x564cb81b8250 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43574/0 (socket says 172.21.15.42:43574) 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd61effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.341+0000 7fd617fff640 1 -- 172.21.15.42:0/342754409 learned_addr learned my addr 172.21.15.42:0/342754409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd617fff640 1 -- 172.21.15.42:0/342754409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 msgr2=0x564cb81c0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd617fff640 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 0x564cb81c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd617fff640 1 -- 172.21.15.42:0/342754409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 msgr2=0x564cb81bba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd617fff640 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81bba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd617fff640 1 -- 172.21.15.42:0/342754409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb80bac20 con 0x564cb81c1750 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd61effd640 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81bba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd617fff640 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bbf50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c010 tx=0x7fd608012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7a37610fab4693 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd614ff9640 1 -- 172.21.15.42:0/342754409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800b8c0 con 0x564cb81c1750 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd614ff9640 1 -- 172.21.15.42:0/342754409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd608012cc0 con 0x564cb81c1750 2024-03-12T04:18:51.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.342+0000 7fd614ff9640 1 -- 172.21.15.42:0/342754409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608012160 con 0x564cb81c1750 2024-03-12T04:18:51.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- 172.21.15.42:0/342754409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 msgr2=0x564cb81bbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bbf50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c010 tx=0x7fd608012f90 comp rx=0 tx=0).stop 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- 172.21.15.42:0/342754409 shutdown_connections 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81c1750 0x564cb81bbf50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81b8b70 0x564cb81bba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 --2- 172.21.15.42:0/342754409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8250 0x564cb81c0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- 172.21.15.42:0/342754409 >> 172.21.15.42:0/342754409 conn(0x564cb81b2980 msgr2=0x564cb81d87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- 172.21.15.42:0/342754409 shutdown_connections 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- 172.21.15.42:0/342754409 wait complete. 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- start start 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 Processor -- start 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8b70 0x564cb81c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81c1750 0x564cb81c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8227fa0 con 0x564cb81b8250 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8228110 con 0x564cb81c1750 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb82282d0 con 0x564cb81b8b70 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd62817f740 10 client.0 _ll_register_callbacks cb 0x7fd60800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd617fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43576/0 (socket says 172.21.15.42:43576) 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.343+0000 7fd617fff640 1 -- 172.21.15.42:0/4060049497 learned_addr learned my addr 172.21.15.42:0/4060049497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd61effd640 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8b70 0x564cb81c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd617fff640 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81c1750 0x564cb81c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd617fff640 1 -- 172.21.15.42:0/4060049497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8b70 msgr2=0x564cb81c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd617fff640 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8b70 0x564cb81c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd617fff640 1 -- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81c1750 msgr2=0x564cb81c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd617fff640 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81c1750 0x564cb81c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd617fff640 1 -- 172.21.15.42:0/4060049497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62010d800 con 0x564cb81b8250 2024-03-12T04:18:51.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd617fff640 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81c7110 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd608004d10 tx=0x7fd60801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256508c902b40ec2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd6067fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd6067fc640 1 -- 172.21.15.42:0/4060049497 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608012070 con 0x564cb81b8250 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd6067fc640 1 -- 172.21.15.42:0/4060049497 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd608036020 con 0x564cb81b8250 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6080054d0 con 0x564cb81b8250 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd6067fc640 1 -- 172.21.15.42:0/4060049497 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6080121d0 con 0x564cb81b8250 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd62817f740 10 client.16551 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564cb811e6a0 con 0x564cb81b8250 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd6067fc640 1 -- 172.21.15.42:0/4060049497 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6080054d0 con 0x564cb81b8250 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.344+0000 7fd6067fc640 1 -- 172.21.15.42:0/4060049497 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd60803b600 con 0x564cb81b8250 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 10 client.16551 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 10 client.16551 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 20 client.16551 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb82295c0 0x564cb822b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 4 client.16551 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564cb7eab890 con 0x564cb82295c0 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd61effd640 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb82295c0 0x564cb822b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd61effd640 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb82295c0 0x564cb822b9b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b22fde6500a6bc29 server_cookie=793b744477b06de6 in_seq=0 out_seq=0 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd6067fc640 10 client.16551 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd6067fc640 1 -- 172.21.15.42:0/4060049497 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cb7eab890 con 0x564cb82295c0 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd6067fc640 10 client.16551 handle_command_reply: tid=0 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 1 client.16551 shutdown 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 msgr2=0x564cb81c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81c7110 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd608004d10 tx=0x7fd60801c040 comp rx=0 tx=0).stop 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 shutdown_connections 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.345+0000 7fd62817f740 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x564cb82295c0 0x564cb822b9b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd62817f740 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cb81c1750 0x564cb81c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd62817f740 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cb81b8b70 0x564cb81c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd62817f740 1 --2- 172.21.15.42:0/4060049497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cb81b8250 0x564cb81c7110 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6067fc640 0 client.16551 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:18:51.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 >> 172.21.15.42:0/4060049497 conn(0x564cb81b2980 msgr2=0x564cb81be750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 shutdown_connections 2024-03-12T04:18:51.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd62817f740 1 -- 172.21.15.42:0/4060049497 wait complete. 2024-03-12T04:18:51.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd62817f740 20 client.16551 trim_cache size 0 max 16384 2024-03-12T04:18:51.355 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd60c0a1ba0 msgr2=0x7fd60c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd60c0a1ba0 0x7fd60c0a4060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cb060 tx=0x7fd618085040 comp rx=0 tx=0).stop 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 msgr2=0x7fd62019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd62019ae60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd61800eaa0 tx=0x7fd61800ef60 comp rx=0 tx=0).stop 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 shutdown_connections 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd60c0a1ba0 0x7fd60c0a4060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd620105020 0x7fd62019ae60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd620101550 0x7fd620196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 --2- 172.21.15.42:0/201407866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd620100b90 0x7fd6201965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 >> 172.21.15.42:0/201407866 conn(0x7fd6200752e0 msgr2=0x7fd6200fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 shutdown_connections 2024-03-12T04:18:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.346+0000 7fd6255ac640 1 -- 172.21.15.42:0/201407866 wait complete. 2024-03-12T04:18:51.359 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:51.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:18:51.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- 172.21.15.42:0/1622985906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105330 msgr2=0x7f597810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 --2- 172.21.15.42:0/1622985906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105330 0x7f597810cf00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5970009ca0 tx=0x7f597002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- 172.21.15.42:0/1622985906 shutdown_connections 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 --2- 172.21.15.42:0/1622985906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105330 0x7f597810cf00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 --2- 172.21.15.42:0/1622985906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5978100c10 0x7f5978104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 --2- 172.21.15.42:0/1622985906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978100260 0x7f5978100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- 172.21.15.42:0/1622985906 >> 172.21.15.42:0/1622985906 conn(0x7f5978077180 msgr2=0x7f5978077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- 172.21.15.42:0/1622985906 shutdown_connections 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- 172.21.15.42:0/1622985906 wait complete. 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 Processor -- start 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- start start 2024-03-12T04:18:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100260 0x7f5978197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978105330 0x7f5978197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f597819bf30 0x7f597819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819c920 con 0x7f5978100260 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819ca90 con 0x7f5978105330 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.462+0000 7f597f46f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819cc00 con 0x7f597819bf30 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100260 0x7f5978197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100260 0x7f5978197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45928/0 (socket says 172.21.15.42:45928) 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 -- 172.21.15.42:0/2382567328 learned_addr learned my addr 172.21.15.42:0/2382567328 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597da0c640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f597819bf30 0x7f597819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978105330 0x7f5978197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 -- 172.21.15.42:0/2382567328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f597819bf30 msgr2=0x7f597819c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f597819bf30 0x7f597819c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 -- 172.21.15.42:0/2382567328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978105330 msgr2=0x7f5978197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978105330 0x7f5978197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 -- 172.21.15.42:0/2382567328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009900 con 0x7f5978100260 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978105330 0x7f5978197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597d20b640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100260 0x7f5978197670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f597400e9f0 tx=0x7f597400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f596e7fc640 1 -- 172.21.15.42:0/2382567328 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974009700 con 0x7f5978100260 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f596e7fc640 1 -- 172.21.15.42:0/2382567328 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f597400cd80 con 0x7f5978100260 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59781a41e0 con 0x7f5978100260 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f596e7fc640 1 -- 172.21.15.42:0/2382567328 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974018840 con 0x7f5978100260 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.463+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59781a46e0 con 0x7f5978100260 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.465+0000 7f596e7fc640 1 -- 172.21.15.42:0/2382567328 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f59740189a0 con 0x7f5978100260 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 Processor -- start 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 -- start start 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 0x558642e96360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f596e7fc640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1c20 0x7f59640a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642ea3b60 0x558642ea5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642e51960 con 0x558642ea3b60 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642e05310 con 0x558642e968a0 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f5982043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642e02840 con 0x558642e95f80 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1c20 0x7f59640a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f597da0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642ea3b60 0x558642ea5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f597da0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642ea3b60 0x558642ea5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45938/0 (socket says 172.21.15.42:45938) 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f597da0c640 1 -- 172.21.15.42:0/1953235731 learned_addr learned my addr 172.21.15.42:0/1953235731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f596e7fc640 1 -- 172.21.15.42:0/2382567328 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5974014070 con 0x7f5978100260 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f597d20b640 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.466+0000 7f597da0c640 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 0x558642e96360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597da0c640 1 -- 172.21.15.42:0/1953235731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 msgr2=0x558642e96360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597da0c640 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 0x558642e96360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597da0c640 1 -- 172.21.15.42:0/1953235731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 msgr2=0x558642ea3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597da0c640 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597da0c640 1 -- 172.21.15.42:0/1953235731 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558642dc0060 con 0x558642ea3b60 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597d20b640 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1c20 0x7f59640a40e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5968005fd0 tx=0x7f59680094d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f597da0c640 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642ea3b60 0x558642ea5f50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5970037040 tx=0x7f597002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807624a316e84f6e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f59637fe640 1 -- 172.21.15.42:0/1953235731 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970034780 con 0x558642ea3b60 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f59637fe640 1 -- 172.21.15.42:0/1953235731 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59700325d0 con 0x558642ea3b60 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.467+0000 7f59637fe640 1 -- 172.21.15.42:0/1953235731 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970003c90 con 0x558642ea3b60 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 -- 172.21.15.42:0/1953235731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642ea3b60 msgr2=0x558642ea5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642ea3b60 0x558642ea5f50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5970037040 tx=0x7f597002e040 comp rx=0 tx=0).stop 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 -- 172.21.15.42:0/1953235731 shutdown_connections 2024-03-12T04:18:51.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642ea3b60 0x558642ea5f50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 --2- 172.21.15.42:0/1953235731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 0x558642e96360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 -- 172.21.15.42:0/1953235731 >> 172.21.15.42:0/1953235731 conn(0x558642e90980 msgr2=0x558642e90d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 -- 172.21.15.42:0/1953235731 shutdown_connections 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 -- 172.21.15.42:0/1953235731 wait complete. 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 -- start start 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.468+0000 7f5982043740 1 Processor -- start 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e95f80 0x558642ea0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 0x558642ea1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642dfe020 con 0x558642e95f80 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642ad1430 con 0x558642e968a0 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642a8de00 con 0x558642ea3b60 2024-03-12T04:18:51.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 10 client.0 _ll_register_callbacks cb 0x558642da5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e95f80 0x558642ea0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e95f80 0x558642ea0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45950/0 (socket says 172.21.15.42:45950) 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 -- 172.21.15.42:0/2720070954 learned_addr learned my addr 172.21.15.42:0/2720070954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 -- 172.21.15.42:0/2720070954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 msgr2=0x558642ea1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 0x558642ea1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 -- 172.21.15.42:0/2720070954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 msgr2=0x558642ea1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 -- 172.21.15.42:0/2720070954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978068660 con 0x558642e95f80 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f597d20b640 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e95f80 0x558642ea0bd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f597400e9c0 tx=0x7f597409ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521c52783a64db28 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5960ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5960ff9640 1 -- 172.21.15.42:0/2720070954 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740a2f10 con 0x558642e95f80 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5960ff9640 1 -- 172.21.15.42:0/2720070954 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5974096de0 con 0x558642e95f80 2024-03-12T04:18:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.469+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558642afd7e0 con 0x558642e95f80 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5960ff9640 1 -- 172.21.15.42:0/2720070954 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5974096fa0 con 0x558642e95f80 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 10 client.16563 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558642dfc6a0 con 0x558642e95f80 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5960ff9640 1 -- 172.21.15.42:0/2720070954 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558642afd7e0 con 0x558642e95f80 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5960ff9640 1 -- 172.21.15.42:0/2720070954 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59740ba220 con 0x558642e95f80 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 10 client.16563 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 10 client.16563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 20 client.16563 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642eb3fb0 0x558642eb63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 4 client.16563 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558642b3a720 con 0x558642eb3fb0 2024-03-12T04:18:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.470+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642eb3fb0 0x558642eb63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f597ca0a640 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642eb3fb0 0x558642eb63d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6352a19dfec419b server_cookie=6a817aae0b3635ca in_seq=0 out_seq=0 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5960ff9640 10 client.16563 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5960ff9640 1 -- 172.21.15.42:0/2720070954 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558642b3a720 con 0x558642eb3fb0 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5960ff9640 10 client.16563 handle_command_reply: tid=0 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 client.16563 shutdown 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e95f80 msgr2=0x558642ea0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e95f80 0x558642ea0bd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f597400e9c0 tx=0x7f597409ae70 comp rx=0 tx=0).stop 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 shutdown_connections 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642eb3fb0 0x558642eb63d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 0x558642ea1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5960ff9640 0 client.16563 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e968a0 0x558642ea1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.471+0000 7f5982043740 1 --2- 172.21.15.42:0/2720070954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e95f80 0x558642ea0bd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.472+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 >> 172.21.15.42:0/2720070954 conn(0x558642e90980 msgr2=0x558642e926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.472+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 shutdown_connections 2024-03-12T04:18:51.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.472+0000 7f5982043740 1 -- 172.21.15.42:0/2720070954 wait complete. 2024-03-12T04:18:51.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.472+0000 7f5982043740 20 client.16563 trim_cache size 0 max 16384 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.483+0000 7f5982043740 1 Processor -- start 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.483+0000 7f5982043740 1 -- start start 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5982043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 0x558642ea3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5982043740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5982043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642ea3b60 0x558642ebad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5982043740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642d83130 con 0x558642e968a0 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5982043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642e8c320 con 0x558642ea3b60 2024-03-12T04:18:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5982043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642e9c7f0 con 0x558642e95f80 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45960/0 (socket says 172.21.15.42:45960) 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 -- 172.21.15.42:0/647002797 learned_addr learned my addr 172.21.15.42:0/647002797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 -- 172.21.15.42:0/647002797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 msgr2=0x558642ea3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 --2- 172.21.15.42:0/647002797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 0x558642ea3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 -- 172.21.15.42:0/647002797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642ea3b60 msgr2=0x558642ebad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 --2- 172.21.15.42:0/647002797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642ea3b60 0x558642ebad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 -- 172.21.15.42:0/647002797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558642d98c20 con 0x558642e968a0 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f597ca0a640 1 --2- 172.21.15.42:0/647002797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea1470 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f596800ecd0 tx=0x7f5968014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbffd68d62c1cf25 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5962ffd640 1 -- 172.21.15.42:0/647002797 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968024070 con 0x558642e968a0 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5962ffd640 1 -- 172.21.15.42:0/647002797 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5968004020 con 0x558642e968a0 2024-03-12T04:18:51.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.484+0000 7f5962ffd640 1 -- 172.21.15.42:0/647002797 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680054b0 con 0x558642e968a0 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 -- 172.21.15.42:0/647002797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 msgr2=0x558642ea1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 --2- 172.21.15.42:0/647002797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea1470 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f596800ecd0 tx=0x7f5968014000 comp rx=0 tx=0).stop 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 -- 172.21.15.42:0/647002797 shutdown_connections 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 --2- 172.21.15.42:0/647002797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642ea3b60 0x558642ebad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 --2- 172.21.15.42:0/647002797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea1470 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 --2- 172.21.15.42:0/647002797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642e95f80 0x558642ea3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 -- 172.21.15.42:0/647002797 >> 172.21.15.42:0/647002797 conn(0x558642e90980 msgr2=0x558642e90d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 -- 172.21.15.42:0/647002797 shutdown_connections 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 -- 172.21.15.42:0/647002797 wait complete. 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 -- start start 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.485+0000 7f5982043740 1 Processor -- start 2024-03-12T04:18:51.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e95f80 0x558642ea9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 0x558642eaa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642f0a620 con 0x558642e968a0 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642f0a790 con 0x558642e95f80 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558642f0a950 con 0x558642ea3b60 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 10 client.0 _ll_register_callbacks cb 0x7f5974005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45974/0 (socket says 172.21.15.42:45974) 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 -- 172.21.15.42:0/637577608 learned_addr learned my addr 172.21.15.42:0/637577608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597d20b640 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e95f80 0x558642ea9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 -- 172.21.15.42:0/637577608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 msgr2=0x558642eaa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 0x558642eaa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 -- 172.21.15.42:0/637577608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e95f80 msgr2=0x558642ea9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e95f80 0x558642ea9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 -- 172.21.15.42:0/637577608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596800ee00 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597d20b640 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e95f80 0x558642ea9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f597ca0a640 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea9b60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f59680097b0 tx=0x7f59680044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b2470b89e6346a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5960ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5960ff9640 1 -- 172.21.15.42:0/637577608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968024050 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5960ff9640 1 -- 172.21.15.42:0/637577608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5968005130 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.486+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59700325d0 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5960ff9640 1 -- 172.21.15.42:0/637577608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59680052f0 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 10 client.16569 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558642dbe8c0 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5960ff9640 1 -- 172.21.15.42:0/637577608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59700325d0 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5960ff9640 1 -- 172.21.15.42:0/637577608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f596802c070 con 0x558642e968a0 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 10 client.16569 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 10 client.16569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 20 client.16569 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642f0c0b0 0x558642f0e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 4 client.16569 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558642b89890 con 0x558642f0c0b0 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.487+0000 7f597d20b640 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642f0c0b0 0x558642f0e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5960ff9640 10 client.16569 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f597d20b640 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642f0c0b0 0x558642f0e4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8508c90d736e5a68 server_cookie=6e4c5b3f74c5983d in_seq=0 out_seq=0 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5960ff9640 1 -- 172.21.15.42:0/637577608 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558642b89890 con 0x558642f0c0b0 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5960ff9640 10 client.16569 handle_command_reply: tid=0 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 client.16569 shutdown 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 msgr2=0x558642ea9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea9b60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f59680097b0 tx=0x7f59680044f0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 shutdown_connections 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558642f0c0b0 0x558642f0e4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558642ea3b60 0x558642eaa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5960ff9640 0 client.16569 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558642e968a0 0x558642ea9b60 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 --2- 172.21.15.42:0/637577608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558642e95f80 0x558642ea9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 >> 172.21.15.42:0/637577608 conn(0x558642e90980 msgr2=0x558642ebad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 shutdown_connections 2024-03-12T04:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 1 -- 172.21.15.42:0/637577608 wait complete. 2024-03-12T04:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f5982043740 20 client.16569 trim_cache size 0 max 16384 2024-03-12T04:18:51.497 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.488+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1c20 msgr2=0x7f59640a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1c20 0x7f59640a40e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5968005fd0 tx=0x7f59680094d0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100260 msgr2=0x7f5978197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100260 0x7f5978197670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f597400e9f0 tx=0x7f597400eeb0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 shutdown_connections 2024-03-12T04:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1c20 0x7f59640a40e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f597819bf30 0x7f597819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978105330 0x7f5978197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 --2- 172.21.15.42:0/2382567328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100260 0x7f5978197670 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 >> 172.21.15.42:0/2382567328 conn(0x7f5978077180 msgr2=0x7f59780fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 shutdown_connections 2024-03-12T04:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.489+0000 7f597f46f640 1 -- 172.21.15.42:0/2382567328 wait complete. 2024-03-12T04:18:51.501 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:51 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3664269871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:18:51.502 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:18:51.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:18:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/2642656839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38106450 msgr2=0x7f6e3810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/2642656839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38106450 0x7f6e3810e020 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009cd0 tx=0x7f6e3002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/2642656839 shutdown_connections 2024-03-12T04:18:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/2642656839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38106450 0x7f6e3810e020 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/2642656839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e3806a430 0x7f6e38105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/2642656839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e38069a80 0x7f6e38069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/2642656839 >> 172.21.15.42:0/2642656839 conn(0x7f6e38077150 msgr2=0x7f6e38077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/2642656839 shutdown_connections 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.604+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/2642656839 wait complete. 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 Processor -- start 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 -- start start 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38069a80 0x7f6e381976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e38106450 0x7f6e38197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e3819bf70 0x7f6e3819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3819c960 con 0x7f6e38069a80 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3819cad0 con 0x7f6e38106450 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3f0f2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3819cc40 con 0x7f6e3819bf70 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38069a80 0x7f6e381976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38069a80 0x7f6e381976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45992/0 (socket says 172.21.15.42:45992) 2024-03-12T04:18:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 -- 172.21.15.42:0/416220768 learned_addr learned my addr 172.21.15.42:0/416220768 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e3819bf70 0x7f6e3819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e37fff640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e38106450 0x7f6e38197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 -- 172.21.15.42:0/416220768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e3819bf70 msgr2=0x7f6e3819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e3819bf70 0x7f6e3819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 -- 172.21.15.42:0/416220768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e38106450 msgr2=0x7f6e38197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e38106450 0x7f6e38197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e3ce8e640 1 -- 172.21.15.42:0/416220768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30009900 con 0x7f6e38069a80 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.605+0000 7f6e37fff640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e38106450 0x7f6e38197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.606+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38069a80 0x7f6e381976b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00c890 tx=0x7f6e2c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.606+0000 7f6e35ffb640 1 -- 172.21.15.42:0/416220768 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c004220 con 0x7f6e38069a80 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.606+0000 7f6e35ffb640 1 -- 172.21.15.42:0/416220768 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e2c007e60 con 0x7f6e38069a80 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.606+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e381a4220 con 0x7f6e38069a80 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.606+0000 7f6e35ffb640 1 -- 172.21.15.42:0/416220768 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c004a00 con 0x7f6e38069a80 2024-03-12T04:18:51.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.606+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e381a46d0 con 0x7f6e38069a80 2024-03-12T04:18:51.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.608+0000 7f6e35ffb640 1 -- 172.21.15.42:0/416220768 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f6e2c0079b0 con 0x7f6e38069a80 2024-03-12T04:18:51.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.608+0000 7f6e41d9d740 1 Processor -- start 2024-03-12T04:18:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.608+0000 7f6e41d9d740 1 -- start start 2024-03-12T04:18:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393634e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936357210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x563936359b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563936309960 con 0x563936357750 2024-03-12T04:18:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e35ffb640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6e280a1ba0 0x7f6e280a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639362bd310 con 0x56393634eb70 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639362ba840 con 0x56393634e250 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x563936359b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6e280a1ba0 0x7f6e280a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x563936359b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46004/0 (socket says 172.21.15.42:46004) 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 -- 172.21.15.42:0/757141247 learned_addr learned my addr 172.21.15.42:0/757141247 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e35ffb640 1 -- 172.21.15.42:0/416220768 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6e2c0cd520 con 0x7f6e38069a80 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393634e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936357210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 -- 172.21.15.42:0/757141247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 msgr2=0x56393634e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393634e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 -- 172.21.15.42:0/757141247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 msgr2=0x563936357210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936357210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 -- 172.21.15.42:0/757141247 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563936278060 con 0x563936357750 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6e280a1ba0 0x7f6e280a4060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e30005cc0 tx=0x7f6e30032c90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.609+0000 7f6e37fff640 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x563936359b40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009c30 tx=0x7f6e240117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f5ed877fa3b530 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e22ffd640 1 -- 172.21.15.42:0/757141247 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004020 con 0x563936357750 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e22ffd640 1 -- 172.21.15.42:0/757141247 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e240035f0 con 0x563936357750 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e22ffd640 1 -- 172.21.15.42:0/757141247 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24012670 con 0x563936357750 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 -- 172.21.15.42:0/757141247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 msgr2=0x563936359b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x563936359b40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009c30 tx=0x7f6e240117b0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 -- 172.21.15.42:0/757141247 shutdown_connections 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x563936359b40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936357210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/757141247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393634e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 -- 172.21.15.42:0/757141247 >> 172.21.15.42:0/757141247 conn(0x563936348980 msgr2=0x563936348d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 -- 172.21.15.42:0/757141247 shutdown_connections 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.610+0000 7f6e41d9d740 1 -- 172.21.15.42:0/757141247 wait complete. 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 -- start start 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 Processor -- start 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x563936354730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936354c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639362b6020 con 0x563936357750 2024-03-12T04:18:51.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563935f89430 con 0x56393634eb70 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x563935f45e00 con 0x56393634e250 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e41d9d740 10 client.0 _ll_register_callbacks cb 0x56393625dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e3d68f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e3d68f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46018/0 (socket says 172.21.15.42:46018) 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2428758731 learned_addr learned my addr 172.21.15.42:0/2428758731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x563936354730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.611+0000 7f6e37fff640 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936354c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2428758731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 msgr2=0x563936354730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x563936354730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 msgr2=0x563936354c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936354c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2428758731 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e24009860 con 0x563936357750 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363551b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6e30036000 tx=0x7f6e30005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e289da6806c3367 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e17fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e17fff640 1 -- 172.21.15.42:0/2428758731 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30048070 con 0x563936357750 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e17fff640 1 -- 172.21.15.42:0/2428758731 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6e30004d50 con 0x563936357750 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563935fb57e0 con 0x563936357750 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e17fff640 1 -- 172.21.15.42:0/2428758731 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e30003970 con 0x563936357750 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 10 client.16581 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5639362b46a0 con 0x563936357750 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e17fff640 1 -- 172.21.15.42:0/2428758731 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e3005c020 con 0x563936357750 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e17fff640 1 -- 172.21.15.42:0/2428758731 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e300045b0 con 0x563936357750 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 10 client.16581 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 20 client.16581 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x563936367b50 0x563936369f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 4 client.16581 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.612+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563935ff2720 con 0x563936367b50 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.613+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x563936367b50 0x563936369f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.613+0000 7f6e17fff640 10 client.16581 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:51.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.613+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x563936367b50 0x563936369f40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40c94c83bc90575 server_cookie=f46524ff672f3d2e in_seq=0 out_seq=0 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e17fff640 1 -- 172.21.15.42:0/2428758731 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563935ff2720 con 0x563936367b50 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e17fff640 10 client.16581 handle_command_reply: tid=0 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 client.16581 shutdown 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 msgr2=0x5639363551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363551b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6e30036000 tx=0x7f6e30005c50 comp rx=0 tx=0).stop 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 shutdown_connections 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x563936367b50 0x563936369f40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363551b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x563936354c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e17fff640 0 client.16581 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2428758731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x563936354730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 >> 172.21.15.42:0/2428758731 conn(0x563936348980 msgr2=0x56393634a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 shutdown_connections 2024-03-12T04:18:51.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2428758731 wait complete. 2024-03-12T04:18:51.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.614+0000 7f6e41d9d740 20 client.16581 trim_cache size 0 max 16384 2024-03-12T04:18:51.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 Processor -- start 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 -- start start 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634e250 0x563936347a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56393634eb70 0x56393635e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563936357750 0x56393635eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393623b130 con 0x56393634eb70 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393634dc90 con 0x56393634e250 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393634deb0 con 0x563936357750 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56393634eb70 0x56393635e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56393634eb70 0x56393635e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46024/0 (socket says 172.21.15.42:46024) 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 -- 172.21.15.42:0/2490214514 learned_addr learned my addr 172.21.15.42:0/2490214514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563936357750 0x56393635eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634e250 0x563936347a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 -- 172.21.15.42:0/2490214514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563936357750 msgr2=0x56393635eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563936357750 0x56393635eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 -- 172.21.15.42:0/2490214514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634e250 msgr2=0x563936347a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634e250 0x563936347a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e37fff640 1 -- 172.21.15.42:0/2490214514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563936250c20 con 0x56393634eb70 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.626+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634e250 0x563936347a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e37fff640 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56393634eb70 0x56393635e600 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x563936349b10 tx=0x7f6e24009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b2d15114ceb042 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e227fc640 1 -- 172.21.15.42:0/2490214514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24009da0 con 0x56393634eb70 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e227fc640 1 -- 172.21.15.42:0/2490214514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e240044b0 con 0x56393634eb70 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e227fc640 1 -- 172.21.15.42:0/2490214514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240039c0 con 0x56393634eb70 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2490214514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56393634eb70 msgr2=0x56393635e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56393634eb70 0x56393635e600 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x563936349b10 tx=0x7f6e24009680 comp rx=0 tx=0).stop 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2490214514 shutdown_connections 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563936357750 0x56393635eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56393634eb70 0x56393635e600 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2490214514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634e250 0x563936347a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2490214514 >> 172.21.15.42:0/2490214514 conn(0x563936348980 msgr2=0x5639363472f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2490214514 shutdown_connections 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.627+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2490214514 wait complete. 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 -- start start 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 Processor -- start 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393635d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x56393635dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639363b86b0 con 0x563936357750 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639363b8820 con 0x56393634eb70 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639363b89e0 con 0x56393634e250 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e41d9d740 10 client.0 _ll_register_callbacks cb 0x7f6e240095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363b8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46042/0 (socket says 172.21.15.42:46042) 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2401862146 learned_addr learned my addr 172.21.15.42:0/2401862146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393635d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e37fff640 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x56393635dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2401862146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 msgr2=0x56393635d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393635d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 msgr2=0x56393635dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x56393635dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.628+0000 7f6e3d68f640 1 -- 172.21.15.42:0/2401862146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2401e040 con 0x563936357750 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e3d68f640 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363b8170 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6e300049d0 tx=0x7f6e30009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129b4b88588b03b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e17fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e17fff640 1 -- 172.21.15.42:0/2401862146 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30048050 con 0x563936357750 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e17fff640 1 -- 172.21.15.42:0/2401862146 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6e3005d080 con 0x563936357750 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563936041890 con 0x563936357750 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e17fff640 1 -- 172.21.15.42:0/2401862146 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e300586d0 con 0x563936357750 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 10 client.16587 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5639362768c0 con 0x563936357750 2024-03-12T04:18:51.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e17fff640 1 -- 172.21.15.42:0/2401862146 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x563936041890 con 0x563936357750 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e17fff640 1 -- 172.21.15.42:0/2401862146 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e30035040 con 0x563936357750 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 10 client.16587 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 20 client.16587 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5639363bff30 0x5639363c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 4 client.16587 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.629+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563935f61700 con 0x5639363bff30 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5639363bff30 0x5639363c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e17fff640 10 client.16587 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e3ce8e640 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5639363bff30 0x5639363c2350 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ca5b1ab9204088 server_cookie=add94b9df2847530 in_seq=0 out_seq=0 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e17fff640 1 -- 172.21.15.42:0/2401862146 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563935f61700 con 0x5639363bff30 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e17fff640 10 client.16587 handle_command_reply: tid=0 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 client.16587 shutdown 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 msgr2=0x5639363b8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363b8170 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6e300049d0 tx=0x7f6e30009c90 comp rx=0 tx=0).stop 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 shutdown_connections 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5639363bff30 0x5639363c2350 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563936357750 0x5639363b8170 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56393634eb70 0x56393635dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e17fff640 0 client.16587 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 --2- 172.21.15.42:0/2401862146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56393634e250 0x56393635d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 >> 172.21.15.42:0/2401862146 conn(0x563936348980 msgr2=0x563936347b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 shutdown_connections 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.630+0000 7f6e41d9d740 1 -- 172.21.15.42:0/2401862146 wait complete. 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e41d9d740 20 client.16587 trim_cache size 0 max 16384 2024-03-12T04:18:51.640 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:18:51.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6e280a1ba0 msgr2=0x7f6e280a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6e280a1ba0 0x7f6e280a4060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e30005cc0 tx=0x7f6e30032c90 comp rx=0 tx=0).stop 2024-03-12T04:18:51.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38069a80 msgr2=0x7f6e381976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:51.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38069a80 0x7f6e381976b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00c890 tx=0x7f6e2c00cd50 comp rx=0 tx=0).stop 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 shutdown_connections 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6e280a1ba0 0x7f6e280a4060 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e3819bf70 0x7f6e3819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e38106450 0x7f6e38197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 --2- 172.21.15.42:0/416220768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e38069a80 0x7f6e381976b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 >> 172.21.15.42:0/416220768 conn(0x7f6e38077150 msgr2=0x7f6e380fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 shutdown_connections 2024-03-12T04:18:51.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:51.631+0000 7f6e3f0f2640 1 -- 172.21.15.42:0/416220768 wait complete. 2024-03-12T04:18:51.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:51 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3664269871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:18:51.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:51 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3664269871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:18:52.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:18:52.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:18:52.646 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:18:52.674 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:52 smithi042 ceph-mon[32260]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:52.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:52 smithi119 ceph-mon[36119]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:52 smithi136 ceph-mon[35571]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 -- 172.21.15.42:0/3503868116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 msgr2=0x7face810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 --2- 172.21.15.42:0/3503868116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face810e000 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7face0009cd0 tx=0x7face002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 -- 172.21.15.42:0/3503868116 shutdown_connections 2024-03-12T04:18:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 --2- 172.21.15.42:0/3503868116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face810e000 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 --2- 172.21.15.42:0/3503868116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80696f0 0x7face8069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 --2- 172.21.15.42:0/3503868116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face81071d0 0x7face81075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 -- 172.21.15.42:0/3503868116 >> 172.21.15.42:0/3503868116 conn(0x7face8077060 msgr2=0x7face8077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 -- 172.21.15.42:0/3503868116 shutdown_connections 2024-03-12T04:18:52.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 -- 172.21.15.42:0/3503868116 wait complete. 2024-03-12T04:18:52.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.763+0000 7facedd3e640 1 Processor -- start 2024-03-12T04:18:52.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7facedd3e640 1 -- start start 2024-03-12T04:18:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7facedd3e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80696f0 0x7face81976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7facedd3e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7facedd3e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face81071d0 0x7face819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7facedd3e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face819c570 con 0x7face806a070 2024-03-12T04:18:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7facedd3e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face819c6e0 con 0x7face81071d0 2024-03-12T04:18:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7facedd3e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face819c8a0 con 0x7face80696f0 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face8197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46052/0 (socket says 172.21.15.42:46052) 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 -- 172.21.15.42:0/350106057 learned_addr learned my addr 172.21.15.42:0/350106057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face77fe640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80696f0 0x7face81976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face7fff640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face81071d0 0x7face819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 -- 172.21.15.42:0/350106057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80696f0 msgr2=0x7face81976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80696f0 0x7face81976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 -- 172.21.15.42:0/350106057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face81071d0 msgr2=0x7face819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face81071d0 0x7face819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.764+0000 7face6ffd640 1 -- 172.21.15.42:0/350106057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0009900 con 0x7face806a070 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.765+0000 7face6ffd640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face8197bf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7facdc00e9f0 tx=0x7facdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.765+0000 7face4ff9640 1 -- 172.21.15.42:0/350106057 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc009720 con 0x7face806a070 2024-03-12T04:18:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.765+0000 7face4ff9640 1 -- 172.21.15.42:0/350106057 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7facdc00cd80 con 0x7face806a070 2024-03-12T04:18:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.765+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face819cb80 con 0x7face806a070 2024-03-12T04:18:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.765+0000 7face4ff9640 1 -- 172.21.15.42:0/350106057 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc010430 con 0x7face806a070 2024-03-12T04:18:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.765+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face81a4400 con 0x7face806a070 2024-03-12T04:18:52.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face4ff9640 1 -- 172.21.15.42:0/350106057 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7facdc005020 con 0x7face806a070 2024-03-12T04:18:52.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 Processor -- start 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 -- start start 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd04b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd054740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd055170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bcfb9f90 con 0x55d8bd04b260 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bcfb74c0 con 0x55d8bd054d10 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7facf09ee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bcfb2ca0 con 0x55d8bd04bc10 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd04b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd04b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46060/0 (socket says 172.21.15.42:46060) 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 -- 172.21.15.42:0/654493330 learned_addr learned my addr 172.21.15.42:0/654493330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face4ff9640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7facd00a1ba0 0x7facd00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face77fe640 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd054740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face6ffd640 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd055170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 -- 172.21.15.42:0/654493330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 msgr2=0x55d8bd054740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd054740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 -- 172.21.15.42:0/654493330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 msgr2=0x55d8bd055170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd055170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face7fff640 1 -- 172.21.15.42:0/654493330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8bcf750b0 con 0x55d8bd04b260 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.768+0000 7face6ffd640 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd055170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.769+0000 7face4ff9640 1 -- 172.21.15.42:0/350106057 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7facdc014070 con 0x7face806a070 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.769+0000 7face7fff640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7facd00a1ba0 0x7facd00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.769+0000 7face7fff640 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd04b640 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7face002c700 tx=0x7face0032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edee77a3ce2b479a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.769+0000 7faccdffb640 1 -- 172.21.15.42:0/654493330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0034e20 con 0x55d8bd04b260 2024-03-12T04:18:52.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.769+0000 7faccdffb640 1 -- 172.21.15.42:0/654493330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7face0032a80 con 0x55d8bd04b260 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.769+0000 7face7fff640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7facd00a1ba0 0x7facd00a4060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7face0005e30 tx=0x7face0045000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.769+0000 7faccdffb640 1 -- 172.21.15.42:0/654493330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0004240 con 0x55d8bd04b260 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 -- 172.21.15.42:0/654493330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 msgr2=0x55d8bd04b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd04b640 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7face002c700 tx=0x7face0032340 comp rx=0 tx=0).stop 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 -- 172.21.15.42:0/654493330 shutdown_connections 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd055170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd054740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 --2- 172.21.15.42:0/654493330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd04b640 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 -- 172.21.15.42:0/654493330 >> 172.21.15.42:0/654493330 conn(0x55d8bd045840 msgr2=0x55d8bd045c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 -- 172.21.15.42:0/654493330 shutdown_connections 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 -- 172.21.15.42:0/654493330 wait complete. 2024-03-12T04:18:52.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 -- start start 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.770+0000 7facf09ee740 1 Processor -- start 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04b260 0x55d8bd058750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04bc10 0x55d8bd058c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd054d10 0x55d8bd0591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bcf57f40 con 0x55d8bd054d10 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bcc864b0 con 0x55d8bd04bc10 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bcf38670 con 0x55d8bd04b260 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 10 client.0 _ll_register_callbacks cb 0x55d8bcfa5f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd054d10 0x55d8bd0591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd054d10 0x55d8bd0591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46072/0 (socket says 172.21.15.42:46072) 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 -- 172.21.15.42:0/477459154 learned_addr learned my addr 172.21.15.42:0/477459154 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face6ffd640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04bc10 0x55d8bd058c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face77fe640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04b260 0x55d8bd058750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 -- 172.21.15.42:0/477459154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04b260 msgr2=0x55d8bd058750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04b260 0x55d8bd058750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 -- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04bc10 msgr2=0x55d8bd058c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04bc10 0x55d8bd058c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 -- 172.21.15.42:0/477459154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face810e900 con 0x55d8bd054d10 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face6ffd640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04bc10 0x55d8bd058c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face77fe640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04b260 0x55d8bd058750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7face7fff640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd054d10 0x55d8bd0591d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7face0005dc0 tx=0x7face0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f52744a3f7c88f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facc6ffd640 1 -- 172.21.15.42:0/477459154 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0004740 con 0x55d8bd054d10 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facc6ffd640 1 -- 172.21.15.42:0/477459154 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7face0003970 con 0x55d8bd054d10 2024-03-12T04:18:52.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.771+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8bcf8cf40 con 0x55d8bd054d10 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facc6ffd640 1 -- 172.21.15.42:0/477459154 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7face00048a0 con 0x55d8bd054d10 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 10 client.16599 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d8bcfb1320 con 0x55d8bd054d10 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facc6ffd640 1 -- 172.21.15.42:0/477459154 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d8bcf8cf40 con 0x55d8bd054d10 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facc6ffd640 1 -- 172.21.15.42:0/477459154 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7face0055790 con 0x55d8bd054d10 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 10 client.16599 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 20 client.16599 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd068db0 0x55d8bd06b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 4 client.16599 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8bcfa9310 con 0x55d8bd068db0 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.772+0000 7face77fe640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd068db0 0x55d8bd06b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.773+0000 7face77fe640 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd068db0 0x55d8bd06b1a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d97cd153bd4af51 server_cookie=702c6476243f276a in_seq=0 out_seq=0 2024-03-12T04:18:52.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.773+0000 7facc6ffd640 10 client.16599 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facc6ffd640 1 -- 172.21.15.42:0/477459154 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8bcfa9310 con 0x55d8bd068db0 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facc6ffd640 10 client.16599 handle_command_reply: tid=0 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 client.16599 shutdown 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd054d10 msgr2=0x55d8bd0591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd054d10 0x55d8bd0591d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7face0005dc0 tx=0x7face0004270 comp rx=0 tx=0).stop 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 shutdown_connections 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd068db0 0x55d8bd06b1a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd054d10 0x55d8bd0591d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facc6ffd640 0 client.16599 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04bc10 0x55d8bd058c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 --2- 172.21.15.42:0/477459154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04b260 0x55d8bd058750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 >> 172.21.15.42:0/477459154 conn(0x55d8bd045840 msgr2=0x55d8bd0477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 shutdown_connections 2024-03-12T04:18:52.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 1 -- 172.21.15.42:0/477459154 wait complete. 2024-03-12T04:18:52.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.774+0000 7facf09ee740 20 client.16599 trim_cache size 0 max 16384 2024-03-12T04:18:52.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 Processor -- start 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 -- start start 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04b260 0x55d8bd054600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04bc10 0x55d8bd058ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd054d10 0x55d8bd044050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bcf8cf70 con 0x55d8bd04bc10 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bd03fdd0 con 0x55d8bd04b260 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.789+0000 7facf09ee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bd05b270 con 0x55d8bd054d10 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04bc10 0x55d8bd058ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04bc10 0x55d8bd058ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46080/0 (socket says 172.21.15.42:46080) 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 -- 172.21.15.42:0/270709851 learned_addr learned my addr 172.21.15.42:0/270709851 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face7fff640 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd054d10 0x55d8bd044050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face77fe640 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04b260 0x55d8bd054600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 -- 172.21.15.42:0/270709851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd054d10 msgr2=0x55d8bd044050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd054d10 0x55d8bd044050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 -- 172.21.15.42:0/270709851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04b260 msgr2=0x55d8bd054600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04b260 0x55d8bd054600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 -- 172.21.15.42:0/270709851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8bcf4dc00 con 0x55d8bd04bc10 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face7fff640 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd054d10 0x55d8bd044050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7face6ffd640 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04bc10 0x55d8bd058ff0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7facdc0971d0 tx=0x7facdc06db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3e14fb24af20cc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7faccd7fa640 1 -- 172.21.15.42:0/270709851 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0046c0 con 0x55d8bd04bc10 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7faccd7fa640 1 -- 172.21.15.42:0/270709851 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7facdc066060 con 0x55d8bd04bc10 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.790+0000 7faccd7fa640 1 -- 172.21.15.42:0/270709851 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc096e70 con 0x55d8bd04bc10 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 -- 172.21.15.42:0/270709851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04bc10 msgr2=0x55d8bd058ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04bc10 0x55d8bd058ff0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7facdc0971d0 tx=0x7facdc06db70 comp rx=0 tx=0).stop 2024-03-12T04:18:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 -- 172.21.15.42:0/270709851 shutdown_connections 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd054d10 0x55d8bd044050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04bc10 0x55d8bd058ff0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 --2- 172.21.15.42:0/270709851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd04b260 0x55d8bd054600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 -- 172.21.15.42:0/270709851 >> 172.21.15.42:0/270709851 conn(0x55d8bd045840 msgr2=0x55d8bd053ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 -- 172.21.15.42:0/270709851 shutdown_connections 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 -- 172.21.15.42:0/270709851 wait complete. 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 -- start start 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.791+0000 7facf09ee740 1 Processor -- start 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd0b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd0ba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd05f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bd05f9f0 con 0x55d8bd04b260 2024-03-12T04:18:52.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bd05fb60 con 0x55d8bd054d10 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8bd05fd20 con 0x55d8bd04bc10 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 10 client.0 _ll_register_callbacks cb 0x7face0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd0b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd0b9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46094/0 (socket says 172.21.15.42:46094) 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 -- 172.21.15.42:0/3452616692 learned_addr learned my addr 172.21.15.42:0/3452616692 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face7fff640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd05f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face6ffd640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd0ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 -- 172.21.15.42:0/3452616692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 msgr2=0x55d8bd0ba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd0ba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 -- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 msgr2=0x55d8bd05f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd05f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 -- 172.21.15.42:0/3452616692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face81a4400 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face6ffd640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd0ba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face7fff640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd05f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7face77fe640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd0b9b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55d8bd06ba50 tx=0x7facd800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece61c5abb0a46cb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facc6ffd640 1 -- 172.21.15.42:0/3452616692 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd801ad70 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facc6ffd640 1 -- 172.21.15.42:0/3452616692 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7facd80035f0 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.792+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8bcfd8900 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facc6ffd640 1 -- 172.21.15.42:0/3452616692 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7facd801a900 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 10 client.16605 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d8bcf73910 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facc6ffd640 1 -- 172.21.15.42:0/3452616692 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d8bcfd8900 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facc6ffd640 1 -- 172.21.15.42:0/3452616692 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7facd802d710 con 0x55d8bd04b260 2024-03-12T04:18:52.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 10 client.16605 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 20 client.16605 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd0c18e0 0x55d8bd0c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 4 client.16605 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d8bccb2860 con 0x55d8bd0c18e0 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.793+0000 7face6ffd640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd0c18e0 0x55d8bd0c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.794+0000 7face6ffd640 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd0c18e0 0x55d8bd0c3cd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2272c0e3ade0d2e5 server_cookie=2b30cfdff67c665a in_seq=0 out_seq=0 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.794+0000 7facc6ffd640 10 client.16605 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facc6ffd640 1 -- 172.21.15.42:0/3452616692 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d8bccb2860 con 0x55d8bd0c18e0 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facc6ffd640 10 client.16605 handle_command_reply: tid=0 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 client.16605 shutdown 2024-03-12T04:18:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 msgr2=0x55d8bd0b9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd0b9b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55d8bd06ba50 tx=0x7facd800bc80 comp rx=0 tx=0).stop 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 shutdown_connections 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d8bd0c18e0 0x55d8bd0c3cd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8bd054d10 0x55d8bd05f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d8bd04bc10 0x55d8bd0ba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facc6ffd640 0 client.16605 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 --2- 172.21.15.42:0/3452616692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8bd04b260 0x55d8bd0b9b10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 >> 172.21.15.42:0/3452616692 conn(0x55d8bd045840 msgr2=0x55d8bd059e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 shutdown_connections 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 1 -- 172.21.15.42:0/3452616692 wait complete. 2024-03-12T04:18:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.795+0000 7facf09ee740 20 client.16605 trim_cache size 0 max 16384 2024-03-12T04:18:52.808 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:18:52.808 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:18:52.808 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "5205c112-ed9c-4afa-8a94-aaea9aa1bc4b", 2024-03-12T04:18:52.808 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:18:52.808 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:18:52.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7facd00a1ba0 msgr2=0x7facd00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7facd00a1ba0 0x7facd00a4060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7face0005e30 tx=0x7face0045000 comp rx=0 tx=0).stop 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 msgr2=0x7face8197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face8197bf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7facdc00e9f0 tx=0x7facdc00edf0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 shutdown_connections 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7facd00a1ba0 0x7facd00a4060 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face81071d0 0x7face819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face806a070 0x7face8197bf0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 --2- 172.21.15.42:0/350106057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80696f0 0x7face81976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 >> 172.21.15.42:0/350106057 conn(0x7face8077060 msgr2=0x7face80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 shutdown_connections 2024-03-12T04:18:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.796+0000 7facedd3e640 1 -- 172.21.15.42:0/350106057 wait complete. 2024-03-12T04:18:52.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:18:52.811 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:18:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 -- 172.21.15.42:0/97355008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738105340 msgr2=0x7f373810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 --2- 172.21.15.42:0/97355008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738105340 0x7f373810cf10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3730009cd0 tx=0x7f373002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 -- 172.21.15.42:0/97355008 shutdown_connections 2024-03-12T04:18:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 --2- 172.21.15.42:0/97355008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738105340 0x7f373810cf10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 --2- 172.21.15.42:0/97355008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3738100c20 0x7f3738104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 --2- 172.21.15.42:0/97355008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3738100270 0x7f3738100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 -- 172.21.15.42:0/97355008 >> 172.21.15.42:0/97355008 conn(0x7f3738077230 msgr2=0x7f3738077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 -- 172.21.15.42:0/97355008 shutdown_connections 2024-03-12T04:18:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.952+0000 7f373ccd4640 1 -- 172.21.15.42:0/97355008 wait complete. 2024-03-12T04:18:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 Processor -- start 2024-03-12T04:18:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 -- start start 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738100270 0x7f3738197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3738105340 0x7f3738197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373819bed0 0x7f373819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373819c8c0 con 0x7f3738100270 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373819ca30 con 0x7f373819bed0 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373ccd4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373819cba0 con 0x7f3738105340 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738100270 0x7f3738197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738100270 0x7f3738197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46126/0 (socket says 172.21.15.42:46126) 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 -- 172.21.15.42:0/2580541584 learned_addr learned my addr 172.21.15.42:0/2580541584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f3735d9b640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3738105340 0x7f3738197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f3736d9d640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373819bed0 0x7f373819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 -- 172.21.15.42:0/2580541584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3738105340 msgr2=0x7f3738197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3738105340 0x7f3738197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 -- 172.21.15.42:0/2580541584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373819bed0 msgr2=0x7f373819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373819bed0 0x7f373819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f373659c640 1 -- 172.21.15.42:0/2580541584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3730009900 con 0x7f3738100270 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.953+0000 7f3736d9d640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373819bed0 0x7f373819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.954+0000 7f373659c640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738100270 0x7f3738197610 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f372800c890 tx=0x7f372800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.954+0000 7f37277fe640 1 -- 172.21.15.42:0/2580541584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728004220 con 0x7f3738100270 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.954+0000 7f37277fe640 1 -- 172.21.15.42:0/2580541584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3728007e60 con 0x7f3738100270 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.954+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37381a4180 con 0x7f3738100270 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.954+0000 7f37277fe640 1 -- 172.21.15.42:0/2580541584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728004a00 con 0x7f3738100270 2024-03-12T04:18:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.954+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37381a4630 con 0x7f3738100270 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.956+0000 7f37277fe640 1 -- 172.21.15.42:0/2580541584 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f37280079b0 con 0x7f3738100270 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.956+0000 7f373f97d740 1 Processor -- start 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.956+0000 7f373f97d740 1 -- start start 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f37277fe640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f37200a1ba0 0x7f37200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373f97d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb24690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373f97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373f97d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373f97d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffca93280 con 0x55dffcb242b0 2024-03-12T04:18:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373f97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffca907b0 con 0x55dffcb24bd0 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373f97d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffca8bf90 con 0x55dffcb2d7b0 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb24690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f3736d9d640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f37200a1ba0 0x7f37200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb24690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46134/0 (socket says 172.21.15.42:46134) 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 -- 172.21.15.42:0/1604998581 learned_addr learned my addr 172.21.15.42:0/1604998581 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f37277fe640 1 -- 172.21.15.42:0/2580541584 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f37280cd4c0 con 0x7f3738100270 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f3735d9b640 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 -- 172.21.15.42:0/1604998581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 msgr2=0x55dffcb2fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 -- 172.21.15.42:0/1604998581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 msgr2=0x55dffcb2d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 -- 172.21.15.42:0/1604998581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dffca4a570 con 0x55dffcb242b0 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f3735d9b640 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f3736d9d640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f37200a1ba0 0x7f37200a4060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3730006010 tx=0x7f37300323d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f373659c640 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb24690 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3728005fd0 tx=0x7f3728008a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159b30c528d13fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f3724ff9640 1 -- 172.21.15.42:0/1604998581 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728099aa0 con 0x55dffcb242b0 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f3724ff9640 1 -- 172.21.15.42:0/1604998581 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f37280ac990 con 0x55dffcb242b0 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.957+0000 7f3724ff9640 1 -- 172.21.15.42:0/1604998581 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37280ac580 con 0x55dffcb242b0 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 -- 172.21.15.42:0/1604998581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 msgr2=0x55dffcb24690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb24690 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3728005fd0 tx=0x7f3728008a40 comp rx=0 tx=0).stop 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 -- 172.21.15.42:0/1604998581 shutdown_connections 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 --2- 172.21.15.42:0/1604998581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb24690 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 -- 172.21.15.42:0/1604998581 >> 172.21.15.42:0/1604998581 conn(0x55dffcb1e990 msgr2=0x55dffcb1ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 -- 172.21.15.42:0/1604998581 shutdown_connections 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.958+0000 7f373f97d740 1 -- 172.21.15.42:0/1604998581 wait complete. 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 -- start start 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 Processor -- start 2024-03-12T04:18:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb2d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb24bd0 0x55dffcb2a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffc75f430 con 0x55dffcb242b0 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffc71be00 con 0x55dffcb2d7b0 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffca10fd0 con 0x55dffcb24bd0 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373f97d740 10 client.0 _ll_register_callbacks cb 0x55dffca66fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb2d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb2d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46148/0 (socket says 172.21.15.42:46148) 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 -- 172.21.15.42:0/3451903002 learned_addr learned my addr 172.21.15.42:0/3451903002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f3736d9d640 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 -- 172.21.15.42:0/3451903002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb24bd0 msgr2=0x55dffcb2a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb24bd0 0x55dffcb2a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 -- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb2d7b0 msgr2=0x55dffcb2ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 -- 172.21.15.42:0/3451903002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37381a4630 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f3736d9d640 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f373659c640 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb2d260 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f372809a0d0 tx=0x7f37280ae540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad07c3897d50a0ec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.959+0000 7f371dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f371dffb640 1 -- 172.21.15.42:0/3451903002 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37280adf30 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f371dffb640 1 -- 172.21.15.42:0/3451903002 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f372809a140 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dffc78b7e0 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f371dffb640 1 -- 172.21.15.42:0/3451903002 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f372809e290 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 10 client.16617 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dffca8a610 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f371dffb640 1 -- 172.21.15.42:0/3451903002 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55dffc78b7e0 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f371dffb640 1 -- 172.21.15.42:0/3451903002 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37280ac020 con 0x55dffcb242b0 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 10 client.16617 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:52.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 20 client.16617 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb3d500 0x55dffcb3f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 4 client.16617 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dffc7c8720 con 0x55dffcb3d500 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.960+0000 7f3735d9b640 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb3d500 0x55dffcb3f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.961+0000 7f371dffb640 10 client.16617 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.961+0000 7f3735d9b640 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb3d500 0x55dffcb3f920 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e15995e5fd137 server_cookie=4284f7bbc535d987 in_seq=0 out_seq=0 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.961+0000 7f371dffb640 1 -- 172.21.15.42:0/3451903002 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dffc7c8720 con 0x55dffcb3d500 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.961+0000 7f371dffb640 10 client.16617 handle_command_reply: tid=0 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.961+0000 7f373f97d740 1 client.16617 shutdown 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 msgr2=0x55dffcb2d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb2d260 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f372809a0d0 tx=0x7f37280ae540 comp rx=0 tx=0).stop 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 shutdown_connections 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb3d500 0x55dffcb3f920 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb24bd0 0x55dffcb2a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f371dffb640 0 client.16617 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 --2- 172.21.15.42:0/3451903002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb2d260 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 >> 172.21.15.42:0/3451903002 conn(0x55dffcb1e990 msgr2=0x55dffcb20730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 shutdown_connections 2024-03-12T04:18:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 1 -- 172.21.15.42:0/3451903002 wait complete. 2024-03-12T04:18:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.962+0000 7f373f97d740 20 client.16617 trim_cache size 0 max 16384 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 Processor -- start 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 -- start start 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb242b0 0x55dffcb2a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffcb1f9e0 con 0x55dffcb2d7b0 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffcb2ff20 con 0x55dffcb24bd0 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373f97d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffcb23db0 con 0x55dffcb242b0 2024-03-12T04:18:52.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46160/0 (socket says 172.21.15.42:46160) 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 -- 172.21.15.42:0/4176745231 learned_addr learned my addr 172.21.15.42:0/4176745231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f373659c640 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb242b0 0x55dffcb2a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3735d9b640 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 -- 172.21.15.42:0/4176745231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb242b0 msgr2=0x55dffcb2a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb242b0 0x55dffcb2a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 -- 172.21.15.42:0/4176745231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 msgr2=0x55dffcb2cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3736d9d640 1 -- 172.21.15.42:0/4176745231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dffca26ac0 con 0x55dffcb2d7b0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.977+0000 7f3735d9b640 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f3736d9d640 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2d260 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3730009cd0 tx=0x7f3730040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d07e7ccd025049 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f371ffff640 1 -- 172.21.15.42:0/4176745231 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730034740 con 0x55dffcb2d7b0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f371ffff640 1 -- 172.21.15.42:0/4176745231 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3730032a80 con 0x55dffcb2d7b0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f371ffff640 1 -- 172.21.15.42:0/4176745231 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37300044f0 con 0x55dffcb2d7b0 2024-03-12T04:18:52.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 -- 172.21.15.42:0/4176745231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb2d7b0 msgr2=0x55dffcb2d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2d260 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3730009cd0 tx=0x7f3730040000 comp rx=0 tx=0).stop 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 -- 172.21.15.42:0/4176745231 shutdown_connections 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb2d7b0 0x55dffcb2d260 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb2cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 --2- 172.21.15.42:0/4176745231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb242b0 0x55dffcb2a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 -- 172.21.15.42:0/4176745231 >> 172.21.15.42:0/4176745231 conn(0x55dffcb1e990 msgr2=0x55dffcb258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 -- 172.21.15.42:0/4176745231 shutdown_connections 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.978+0000 7f373f97d740 1 -- 172.21.15.42:0/4176745231 wait complete. 2024-03-12T04:18:52.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 -- start start 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 Processor -- start 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 0x55dffcb93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffcb322b0 con 0x55dffcb242b0 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffcb93f40 con 0x55dffcb24bd0 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffcb94080 con 0x55dffcb2d7b0 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373f97d740 10 client.0 _ll_register_callbacks cb 0x7f372800ef80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb316a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46172/0 (socket says 172.21.15.42:46172) 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 -- 172.21.15.42:0/493710221 learned_addr learned my addr 172.21.15.42:0/493710221 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f3735d9b640 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 -- 172.21.15.42:0/493710221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 msgr2=0x55dffcb93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 0x55dffcb93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 -- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 msgr2=0x55dffcb31be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb31be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 -- 172.21.15.42:0/493710221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37380682e0 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f3735d9b640 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb31be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f373659c640 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb316a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f37280cb260 tx=0x7f372809a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcb3c6e70acc82d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f371dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.979+0000 7f371dffb640 1 -- 172.21.15.42:0/493710221 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372809e290 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f371dffb640 1 -- 172.21.15.42:0/493710221 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f37280a28e0 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37280ac990 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f371dffb640 1 -- 172.21.15.42:0/493710221 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3728005670 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 10 client.16623 fetch_fsmap learned FSMap version 27 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dffca4c6a0 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f371dffb640 1 -- 172.21.15.42:0/493710221 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f37280ac990 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f371dffb640 1 -- 172.21.15.42:0/493710221 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37280ac040 con 0x55dffcb242b0 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 10 client.16623 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:18:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 20 client.16623 populate_metadata read hostname 'smithi042' 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb952d0 0x55dffcb976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 4 client.16623 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dffc817890 con 0x55dffcb952d0 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.980+0000 7f3735d9b640 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb952d0 0x55dffcb976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f371dffb640 10 client.16623 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f3735d9b640 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb952d0 0x55dffcb976c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f95f8493fa3b52 server_cookie=b9543e96ae6427f1 in_seq=0 out_seq=0 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f371dffb640 1 -- 172.21.15.42:0/493710221 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dffc817890 con 0x55dffcb952d0 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f371dffb640 10 client.16623 handle_command_reply: tid=0 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 client.16623 shutdown 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 msgr2=0x55dffcb316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb316a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f37280cb260 tx=0x7f372809a140 comp rx=0 tx=0).stop 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 shutdown_connections 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55dffcb952d0 0x55dffcb976c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dffcb2d7b0 0x55dffcb93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dffcb24bd0 0x55dffcb31be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f371dffb640 0 client.16623 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 --2- 172.21.15.42:0/493710221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dffcb242b0 0x55dffcb316a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 >> 172.21.15.42:0/493710221 conn(0x55dffcb1e990 msgr2=0x55dffcb28e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 shutdown_connections 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 1 -- 172.21.15.42:0/493710221 wait complete. 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.981+0000 7f373f97d740 20 client.16623 trim_cache size 0 max 16384 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stdout: "5205c112-ed9c-4afa-8a94-aaea9aa1bc4b": { 2024-03-12T04:18:52.992 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:18:52.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "5205c112-ed9c-4afa-8a94-aaea9aa1bc4b", 2024-03-12T04:18:52.993 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:18:52.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:18:52.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:18:52.993 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f37200a1ba0 msgr2=0x7f37200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f37200a1ba0 0x7f37200a4060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3730006010 tx=0x7f37300323d0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738100270 msgr2=0x7f3738197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738100270 0x7f3738197610 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f372800c890 tx=0x7f372800cd50 comp rx=0 tx=0).stop 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 shutdown_connections 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f37200a1ba0 0x7f37200a4060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373819bed0 0x7f373819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3738105340 0x7f3738197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 --2- 172.21.15.42:0/2580541584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3738100270 0x7f3738197610 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 >> 172.21.15.42:0/2580541584 conn(0x7f3738077230 msgr2=0x7f37380fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 shutdown_connections 2024-03-12T04:18:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:18:52.982+0000 7f373ccd4640 1 -- 172.21.15.42:0/2580541584 wait complete. 2024-03-12T04:18:52.995 INFO:tasks.cephfs.filesystem:scrub status for tag:5205c112-ed9c-4afa-8a94-aaea9aa1bc4b - {'path': '/', 'tag': '5205c112-ed9c-4afa-8a94-aaea9aa1bc4b', 'options': 'recursive,force'} 2024-03-12T04:18:53.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:53 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:18:53.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:53 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:53.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:53 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:18:53.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:53 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:53 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:18:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:53 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:53 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:18:53.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:53 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:53.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:53 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:18:53.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:53 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:53.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:53 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:18:53.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:53 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:18:54.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:54 smithi119 ceph-mon[36119]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:54.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:54 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:18:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:54 smithi136 ceph-mon[35571]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:54 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:18:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:54 smithi042 ceph-mon[32260]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:18:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:54 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:18:56.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:56 smithi119 ceph-mon[36119]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:56.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:56 smithi136 ceph-mon[35571]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:56.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:56 smithi042 ceph-mon[32260]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:18:58.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:18:58 smithi119 ceph-mon[36119]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:58.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:18:58 smithi136 ceph-mon[35571]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:18:58.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:18:58 smithi042 ceph-mon[32260]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:00.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:00 smithi119 ceph-mon[36119]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:00.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:00 smithi136 ceph-mon[35571]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:00.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:00 smithi042 ceph-mon[32260]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:02 smithi119 ceph-mon[36119]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:02.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:02 smithi136 ceph-mon[35571]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:02.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:02 smithi042 ceph-mon[32260]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:04.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:04 smithi119 ceph-mon[36119]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:04.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:04 smithi136 ceph-mon[35571]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:04 smithi042 ceph-mon[32260]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:06.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:06 smithi119 ceph-mon[36119]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:06.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:06 smithi136 ceph-mon[35571]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:06.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:06 smithi042 ceph-mon[32260]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:08.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:08 smithi119 ceph-mon[36119]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:08 smithi136 ceph-mon[35571]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:08 smithi042 ceph-mon[32260]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:10.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:10 smithi119 ceph-mon[36119]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:10.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:10 smithi136 ceph-mon[35571]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:10.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:10 smithi042 ceph-mon[32260]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:12.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:12 smithi119 ceph-mon[36119]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:12 smithi136 ceph-mon[35571]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:12.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:12 smithi042 ceph-mon[32260]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:19:14.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:14 smithi119 ceph-mon[36119]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:14 smithi136 ceph-mon[35571]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:14 smithi042 ceph-mon[32260]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:19:16.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:16 smithi119 ceph-mon[36119]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:19:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:16 smithi136 ceph-mon[35571]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:19:16.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:16 smithi042 ceph-mon[32260]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:19:18.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:18 smithi119 ceph-mon[36119]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:19:18.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:18 smithi136 ceph-mon[35571]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:19:18.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:18 smithi042 ceph-mon[32260]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:19:20.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:20 smithi119 ceph-mon[36119]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:19:20.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:20 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:19:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:20 smithi136 ceph-mon[35571]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:19:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:20 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:19:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:20 smithi042 ceph-mon[32260]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:19:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:20 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:19:21.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:21 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:19:21.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:21 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:19:21.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:21 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:19:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:21 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:19:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:21 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:19:21.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:21 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:19:21.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:21 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:19:21.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:21 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:19:21.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:21 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:19:22.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:22 smithi119 ceph-mon[36119]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:19:22.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:22 smithi136 ceph-mon[35571]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:19:22.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:19:22.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:19:23.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:22 smithi042 ceph-mon[32260]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:19:23.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 -- 172.21.15.42:0/4055074136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa4106450 msgr2=0x7f9aa410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/4055074136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa4106450 0x7f9aa410e020 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009930 tx=0x7f9a9c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:23.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 -- 172.21.15.42:0/4055074136 shutdown_connections 2024-03-12T04:19:23.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/4055074136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa4106450 0x7f9aa410e020 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/4055074136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9aa4101d30 0x7f9aa4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/4055074136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa4101380 0x7f9aa4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 -- 172.21.15.42:0/4055074136 >> 172.21.15.42:0/4055074136 conn(0x7f9aa40fcc20 msgr2=0x7f9aa40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 -- 172.21.15.42:0/4055074136 shutdown_connections 2024-03-12T04:19:23.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.113+0000 7f9aa94a7640 1 -- 172.21.15.42:0/4055074136 wait complete. 2024-03-12T04:19:23.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 Processor -- start 2024-03-12T04:19:23.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 -- start start 2024-03-12T04:19:23.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa4101380 0x7f9aa41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9aa4106450 0x7f9aa4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa419bf20 0x7f9aa419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa419c910 con 0x7f9aa419bf20 2024-03-12T04:19:23.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa419ca80 con 0x7f9aa4101380 2024-03-12T04:19:23.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa94a7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa419cbf0 con 0x7f9aa4106450 2024-03-12T04:19:23.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa419bf20 0x7f9aa419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa419bf20 0x7f9aa419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56926/0 (socket says 172.21.15.42:56926) 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa37fe640 1 -- 172.21.15.42:0/74313033 learned_addr learned my addr 172.21.15.42:0/74313033 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa4101380 0x7f9aa41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.114+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9aa4106450 0x7f9aa4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa37fe640 1 -- 172.21.15.42:0/74313033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9aa4106450 msgr2=0x7f9aa4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9aa4106450 0x7f9aa4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa37fe640 1 -- 172.21.15.42:0/74313033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa4101380 msgr2=0x7f9aa41976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa4101380 0x7f9aa41976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa37fe640 1 -- 172.21.15.42:0/74313033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c009560 con 0x7f9aa419bf20 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa419bf20 0x7f9aa419c3d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c005940 tx=0x7f9a9c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9a9bfff640 1 -- 172.21.15.42:0/74313033 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0044c0 con 0x7f9aa419bf20 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa41a41d0 con 0x7f9aa419bf20 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9a9bfff640 1 -- 172.21.15.42:0/74313033 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a9c02de90 con 0x7f9aa419bf20 2024-03-12T04:19:23.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9a9bfff640 1 -- 172.21.15.42:0/74313033 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0053c0 con 0x7f9aa419bf20 2024-03-12T04:19:23.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.115+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa41a4650 con 0x7f9aa419bf20 2024-03-12T04:19:23.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.117+0000 7f9a9bfff640 1 -- 172.21.15.42:0/74313033 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f9a9c005520 con 0x7f9aa419bf20 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 Processor -- start 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 -- start start 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9a9bfff640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a900a1ba0 0x7f9a900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8d4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a834280 con 0x55d53a8c4c90 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a8317b0 con 0x55d53a8d2bc0 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aac07e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a82cf90 con 0x55d53a8c5640 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9a9bfff640 1 -- 172.21.15.42:0/74313033 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9a9c0ef7e0 con 0x7f9aa419bf20 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56942/0 (socket says 172.21.15.42:56942) 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/1118676831 learned_addr learned my addr 172.21.15.42:0/1118676831 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a900a1ba0 0x7f9a900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.118+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8d4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/1118676831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 msgr2=0x55d53a8d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/1118676831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 msgr2=0x55d53a8d4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8d4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/1118676831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53a7eb570 con 0x55d53a8c4c90 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8c5070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9a94007f90 tx=0x7f9a9400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad12321657001ff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a900a1ba0 0x7f9a900a4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4100360 tx=0x7f9a8c004160 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9a997fa640 1 -- 172.21.15.42:0/1118676831 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94004d20 con 0x55d53a8c4c90 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9a997fa640 1 -- 172.21.15.42:0/1118676831 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a94004e80 con 0x55d53a8c4c90 2024-03-12T04:19:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.119+0000 7f9a997fa640 1 -- 172.21.15.42:0/1118676831 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940055e0 con 0x55d53a8c4c90 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 -- 172.21.15.42:0/1118676831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 msgr2=0x55d53a8c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8c5070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9a94007f90 tx=0x7f9a9400f850 comp rx=0 tx=0).stop 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 -- 172.21.15.42:0/1118676831 shutdown_connections 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1118676831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8c5070 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 -- 172.21.15.42:0/1118676831 >> 172.21.15.42:0/1118676831 conn(0x55d53a8bf990 msgr2=0x55d53a8bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 -- 172.21.15.42:0/1118676831 shutdown_connections 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 -- 172.21.15.42:0/1118676831 wait complete. 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 -- start start 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.120+0000 7f9aac07e740 1 Processor -- start 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a500430 con 0x55d53a8d2bc0 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a4bce00 con 0x55d53a8c4c90 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a7b1fd0 con 0x55d53a8c5640 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 10 client.0 _ll_register_callbacks cb 0x55d53a807fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56956/0 (socket says 172.21.15.42:56956) 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 -- 172.21.15.42:0/1067309137 learned_addr learned my addr 172.21.15.42:0/1067309137 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 -- 172.21.15.42:0/1067309137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 msgr2=0x55d53a8cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 -- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8c4c90 msgr2=0x55d53a8d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 -- 172.21.15.42:0/1067309137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94007bc0 con 0x55d53a8d2bc0 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:19:23.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cff30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0bc500 tx=0x7f9a9c0bc470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100ae09534117c2d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9a8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/1067309137 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c042170 con 0x55d53a8d2bc0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/1067309137 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9a9c003dc0 con 0x55d53a8d2bc0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.121+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d53a569720 con 0x55d53a8d2bc0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/1067309137 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a9c0bcc70 con 0x55d53a8d2bc0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 10 client.16635 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d53a82b610 con 0x55d53a8d2bc0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/1067309137 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d53a569720 con 0x55d53a8d2bc0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/1067309137 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a9c03c070 con 0x55d53a8d2bc0 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 10 client.16635 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 20 client.16635 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a8e2a10 0x55d53a8e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 4 client.16635 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d53a5b8890 con 0x55d53a8e2a10 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.122+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a8e2a10 0x55d53a8e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.123+0000 7f9a8a7fc640 10 client.16635 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.123+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a8e2a10 0x55d53a8e4e00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258b942fc72dce0a server_cookie=df4f550625745cee in_seq=0 out_seq=0 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.123+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/1067309137 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d53a5b8890 con 0x55d53a8e2a10 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.123+0000 7f9a8a7fc640 10 client.16635 handle_command_reply: tid=0 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.123+0000 7f9aac07e740 1 client.16635 shutdown 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8d2bc0 msgr2=0x55d53a8cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cff30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0bc500 tx=0x7f9a9c0bc470 comp rx=0 tx=0).stop 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 shutdown_connections 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a8e2a10 0x55d53a8e4e00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cff30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 --2- 172.21.15.42:0/1067309137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9a8a7fc640 0 client.16635 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 >> 172.21.15.42:0/1067309137 conn(0x55d53a8bf990 msgr2=0x55d53a8c1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 shutdown_connections 2024-03-12T04:19:23.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 1 -- 172.21.15.42:0/1067309137 wait complete. 2024-03-12T04:19:23.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.124+0000 7f9aac07e740 20 client.16635 trim_cache size 0 max 16384 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 Processor -- start 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 -- start start 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a5b8890 con 0x55d53a8c4c90 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a569720 con 0x55d53a8d2bc0 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.139+0000 7f9aac07e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a8ca850 con 0x55d53a8c5640 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56968/0 (socket says 172.21.15.42:56968) 2024-03-12T04:19:23.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3937147600 learned_addr learned my addr 172.21.15.42:0/3937147600 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3937147600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 msgr2=0x55d53a8d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3937147600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 msgr2=0x55d53a8cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3937147600 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53a7c7ac0 con 0x55d53a8c4c90 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2050 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9a94007980 tx=0x7f9a9400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0989dffc4fac066 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9a98ff9640 1 -- 172.21.15.42:0/3937147600 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94007a60 con 0x55d53a8c4c90 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9a98ff9640 1 -- 172.21.15.42:0/3937147600 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a94005830 con 0x55d53a8c4c90 2024-03-12T04:19:23.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.140+0000 7f9a98ff9640 1 -- 172.21.15.42:0/3937147600 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940284e0 con 0x55d53a8c4c90 2024-03-12T04:19:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 -- 172.21.15.42:0/3937147600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 msgr2=0x55d53a8d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2050 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9a94007980 tx=0x7f9a9400f850 comp rx=0 tx=0).stop 2024-03-12T04:19:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 -- 172.21.15.42:0/3937147600 shutdown_connections 2024-03-12T04:19:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a8cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3937147600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d2050 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 -- 172.21.15.42:0/3937147600 >> 172.21.15.42:0/3937147600 conn(0x55d53a8bf990 msgr2=0x55d53a8ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 -- 172.21.15.42:0/3937147600 shutdown_connections 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 -- 172.21.15.42:0/3937147600 wait complete. 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 -- start start 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.141+0000 7f9aac07e740 1 Processor -- start 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a938ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a8d77f0 con 0x55d53a8c4c90 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a939430 con 0x55d53a8d2bc0 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53a9395f0 con 0x55d53a8c5640 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 10 client.0 _ll_register_callbacks cb 0x7f9a940079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56982/0 (socket says 172.21.15.42:56982) 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3779867450 learned_addr learned my addr 172.21.15.42:0/3779867450 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3779867450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 msgr2=0x55d53a8d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3779867450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 msgr2=0x55d53a938ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a938ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 -- 172.21.15.42:0/3779867450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d53a7eb570 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aa2ffd640 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d6b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9a94000f80 tx=0x7f9a94005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c09b320f19d694 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9a8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/3779867450 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94010700 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/3779867450 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9a94010860 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d53a52c7e0 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.142+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/3779867450 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a94010a20 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 10 client.16641 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d53a7ed6a0 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/3779867450 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d53a52c7e0 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/3779867450 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a94025440 con 0x55d53a8c4c90 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 10 client.16641 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 20 client.16641 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a93aaf0 0x55d53a93cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 4 client.16641 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d53a51b4e0 con 0x55d53a93aaf0 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.143+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a93aaf0 0x55d53a93cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aa27fc640 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a93aaf0 0x55d53a93cee0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271bc435036e81bc server_cookie=9b57417d8dbe91a9 in_seq=0 out_seq=0 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9a8a7fc640 10 client.16641 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9a8a7fc640 1 -- 172.21.15.42:0/3779867450 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d53a51b4e0 con 0x55d53a93aaf0 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9a8a7fc640 10 client.16641 handle_command_reply: tid=0 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 client.16641 shutdown 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 msgr2=0x55d53a8d6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d6b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9a94000f80 tx=0x7f9a94005770 comp rx=0 tx=0).stop 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 shutdown_connections 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d53a93aaf0 0x55d53a93cee0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d53a8d2bc0 0x55d53a938ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d53a8c5640 0x55d53a8d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 --2- 172.21.15.42:0/3779867450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d53a8c4c90 0x55d53a8d6b50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9a8a7fc640 0 client.16641 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 >> 172.21.15.42:0/3779867450 conn(0x55d53a8bf990 msgr2=0x55d53a8cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 shutdown_connections 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 1 -- 172.21.15.42:0/3779867450 wait complete. 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.144+0000 7f9aac07e740 20 client.16641 trim_cache size 0 max 16384 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:19:23.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:19:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a900a1ba0 msgr2=0x7f9a900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a900a1ba0 0x7f9a900a4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4100360 tx=0x7f9a8c004160 comp rx=0 tx=0).stop 2024-03-12T04:19:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa419bf20 msgr2=0x7f9aa419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa419bf20 0x7f9aa419c3d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c005940 tx=0x7f9a9c0040c0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 shutdown_connections 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a900a1ba0 0x7f9a900a4060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aa419bf20 0x7f9aa419c3d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9aa4106450 0x7f9aa4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 --2- 172.21.15.42:0/74313033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa4101380 0x7f9aa41976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 >> 172.21.15.42:0/74313033 conn(0x7f9aa40fcc20 msgr2=0x7f9aa40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 shutdown_connections 2024-03-12T04:19:23.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.145+0000 7f9aa94a7640 1 -- 172.21.15.42:0/74313033 wait complete. 2024-03-12T04:19:23.157 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:19:23.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 -- 172.21.15.42:0/3008223620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 msgr2=0x7f1a101027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 --2- 172.21.15.42:0/3008223620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a101027d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009900 tx=0x7f1a0002bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 -- 172.21.15.42:0/3008223620 shutdown_connections 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 --2- 172.21.15.42:0/3008223620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1010b7c0 0x7f1a1010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 --2- 172.21.15.42:0/3008223620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a10102d10 0x7f1a1010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 --2- 172.21.15.42:0/3008223620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a101027d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 -- 172.21.15.42:0/3008223620 >> 172.21.15.42:0/3008223620 conn(0x7f1a10077180 msgr2=0x7f1a10077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 -- 172.21.15.42:0/3008223620 shutdown_connections 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 -- 172.21.15.42:0/3008223620 wait complete. 2024-03-12T04:19:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 Processor -- start 2024-03-12T04:19:23.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.301+0000 7f1a16257640 1 -- start start 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a16257640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a10197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a16257640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a10102d10 0x7f1a10197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a16257640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1010b7c0 0x7f1a1019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a16257640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1019c5d0 con 0x7f1a101023f0 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a16257640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1019c740 con 0x7f1a1010b7c0 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a16257640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1019c900 con 0x7f1a10102d10 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a10197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a10197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57000/0 (socket says 172.21.15.42:57000) 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 -- 172.21.15.42:0/3441101841 learned_addr learned my addr 172.21.15.42:0/3441101841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0ffff640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1010b7c0 0x7f1a1019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 -- 172.21.15.42:0/3441101841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a10102d10 msgr2=0x7f1a10197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0effd640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a10102d10 0x7f1a10197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a10102d10 0x7f1a10197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 -- 172.21.15.42:0/3441101841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1010b7c0 msgr2=0x7f1a1019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1010b7c0 0x7f1a1019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 -- 172.21.15.42:0/3441101841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a00009560 con 0x7f1a101023f0 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0effd640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a10102d10 0x7f1a10197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0f7fe640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a10197710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1a00006010 tx=0x7f1a000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0cff9640 1 -- 172.21.15.42:0/3441101841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a000044c0 con 0x7f1a101023f0 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0cff9640 1 -- 172.21.15.42:0/3441101841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a0002de90 con 0x7f1a101023f0 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1019cb80 con 0x7f1a101023f0 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.302+0000 7f1a0cff9640 1 -- 172.21.15.42:0/3441101841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a000053c0 con 0x7f1a101023f0 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.303+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a101a4460 con 0x7f1a101023f0 2024-03-12T04:19:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.304+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a10067f70 con 0x7f1a101023f0 2024-03-12T04:19:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.305+0000 7f1a0cff9640 1 -- 172.21.15.42:0/3441101841 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f1a00005520 con 0x7f1a101023f0 2024-03-12T04:19:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.305+0000 7f1a0cff9640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f19f80a1ba0 0x7f19f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.305+0000 7f1a0effd640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f19f80a1ba0 0x7f19f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.305+0000 7f1a0cff9640 1 -- 172.21.15.42:0/3441101841 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1a000ef830 con 0x7f1a101023f0 2024-03-12T04:19:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.306+0000 7f1a0effd640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f19f80a1ba0 0x7f19f80a4060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1a100ffaa0 tx=0x7f1a04005d30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.310+0000 7f1a0cff9640 1 -- 172.21.15.42:0/3441101841 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a000b8860 con 0x7f1a101023f0 2024-03-12T04:19:23.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.528+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a101068b0 con 0x7f1a101023f0 2024-03-12T04:19:23.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.529+0000 7f1a0cff9640 1 -- 172.21.15.42:0/3441101841 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f1a000bbf20 con 0x7f1a101023f0 2024-03-12T04:19:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:19:23.531 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:19:23.531 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f19f80a1ba0 msgr2=0x7f19f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f19f80a1ba0 0x7f19f80a4060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1a100ffaa0 tx=0x7f1a04005d30 comp rx=0 tx=0).stop 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 msgr2=0x7f1a10197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a10197710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1a00006010 tx=0x7f1a000040c0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 shutdown_connections 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f19f80a1ba0 0x7f19f80a4060 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1010b7c0 0x7f1a1019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a10102d10 0x7f1a10197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 --2- 172.21.15.42:0/3441101841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a101023f0 0x7f1a10197710 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 >> 172.21.15.42:0/3441101841 conn(0x7f1a10077180 msgr2=0x7f1a100fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 shutdown_connections 2024-03-12T04:19:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.531+0000 7f1a16257640 1 -- 172.21.15.42:0/3441101841 wait complete. 2024-03-12T04:19:23.544 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:23.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 -- 172.21.15.42:0/3551301024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880106450 msgr2=0x7f588010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 --2- 172.21.15.42:0/3551301024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880106450 0x7f588010e020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5878009cd0 tx=0x7f587802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 -- 172.21.15.42:0/3551301024 shutdown_connections 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 --2- 172.21.15.42:0/3551301024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880106450 0x7f588010e020 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 --2- 172.21.15.42:0/3551301024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5880101d30 0x7f5880105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 --2- 172.21.15.42:0/3551301024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880101380 0x7f5880101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 -- 172.21.15.42:0/3551301024 >> 172.21.15.42:0/3551301024 conn(0x7f5880078580 msgr2=0x7f5880078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 -- 172.21.15.42:0/3551301024 shutdown_connections 2024-03-12T04:19:23.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.648+0000 7f5886cc9640 1 -- 172.21.15.42:0/3551301024 wait complete. 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 Processor -- start 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 -- start start 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5880101380 0x7f5880197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880106450 0x7f5880197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f588019bfd0 0x7f588019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019c9c0 con 0x7f588019bfd0 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019cb30 con 0x7f5880106450 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5886cc9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019cca0 con 0x7f5880101380 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f588019bfd0 0x7f588019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f588019bfd0 0x7f588019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57014/0 (socket says 172.21.15.42:57014) 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 -- 172.21.15.42:0/4117669965 learned_addr learned my addr 172.21.15.42:0/4117669965 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f587ffff640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880106450 0x7f5880197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5884a65640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5880101380 0x7f5880197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 -- 172.21.15.42:0/4117669965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5880101380 msgr2=0x7f5880197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5880101380 0x7f5880197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 -- 172.21.15.42:0/4117669965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880106450 msgr2=0x7f5880197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880106450 0x7f5880197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5885266640 1 -- 172.21.15.42:0/4117669965 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878009900 con 0x7f588019bfd0 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.649+0000 7f5884a65640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5880101380 0x7f5880197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.650+0000 7f5885266640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f588019bfd0 0x7f588019c480 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5878004af0 tx=0x7f5878035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.650+0000 7f587dffb640 1 -- 172.21.15.42:0/4117669965 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780329e0 con 0x7f588019bfd0 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.650+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58801a4280 con 0x7f588019bfd0 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.650+0000 7f587dffb640 1 -- 172.21.15.42:0/4117669965 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5878035e90 con 0x7f588019bfd0 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.650+0000 7f587dffb640 1 -- 172.21.15.42:0/4117669965 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878004720 con 0x7f588019bfd0 2024-03-12T04:19:23.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.650+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58801a4700 con 0x7f588019bfd0 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.652+0000 7f587dffb640 1 -- 172.21.15.42:0/4117669965 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f587803b040 con 0x7f588019bfd0 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.652+0000 7f588997b740 1 Processor -- start 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.652+0000 7f588997b740 1 -- start start 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.652+0000 7f587dffb640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58700a1ba0 0x7f58700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f588997b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b15630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f588997b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b1e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f588997b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f588997b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391ad0960 con 0x55b391b15250 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f588997b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391a84310 con 0x55b391b1e750 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f588997b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391a81840 con 0x55b391b15b70 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f587dffb640 1 -- 172.21.15.42:0/4117669965 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f58780f00b0 con 0x7f588019bfd0 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f587ffff640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58700a1ba0 0x7f58700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b15630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b15630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57016/0 (socket says 172.21.15.42:57016) 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 -- 172.21.15.42:0/3444207089 learned_addr learned my addr 172.21.15.42:0/3444207089 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f587ffff640 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b1e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5885266640 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 -- 172.21.15.42:0/3444207089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 msgr2=0x55b391b1e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b1e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 -- 172.21.15.42:0/3444207089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 msgr2=0x55b391b20b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b20b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 -- 172.21.15.42:0/3444207089 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b391a3f060 con 0x55b391b15250 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f587ffff640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58700a1ba0 0x7f58700a4060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5880100d10 tx=0x7f586c00a670 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f5884a65640 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b15630 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f587400bd40 tx=0x7f587400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9152f02e72e3998d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.653+0000 7f586affd640 1 -- 172.21.15.42:0/3444207089 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587401a660 con 0x55b391b15250 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f586affd640 1 -- 172.21.15.42:0/3444207089 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f587400be30 con 0x55b391b15250 2024-03-12T04:19:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f586affd640 1 -- 172.21.15.42:0/3444207089 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874011a70 con 0x55b391b15250 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 -- 172.21.15.42:0/3444207089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 msgr2=0x55b391b15630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b15630 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f587400bd40 tx=0x7f587400f730 comp rx=0 tx=0).stop 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 -- 172.21.15.42:0/3444207089 shutdown_connections 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b20b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b1e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 --2- 172.21.15.42:0/3444207089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b15630 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 -- 172.21.15.42:0/3444207089 >> 172.21.15.42:0/3444207089 conn(0x55b391b0f980 msgr2=0x55b391b0fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 -- 172.21.15.42:0/3444207089 shutdown_connections 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.654+0000 7f588997b740 1 -- 172.21.15.42:0/3444207089 wait complete. 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 -- start start 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 Processor -- start 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15b70 0x55b391b1bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b1e750 0x55b391b1c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391a7d020 con 0x55b391b15250 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391750430 con 0x55b391b15b70 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39170ce00 con 0x55b391b1e750 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f588997b740 10 client.0 _ll_register_callbacks cb 0x55b391a24cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57018/0 (socket says 172.21.15.42:57018) 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 -- 172.21.15.42:0/4082420602 learned_addr learned my addr 172.21.15.42:0/4082420602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5885266640 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b1e750 0x55b391b1c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f587ffff640 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15b70 0x55b391b1bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 -- 172.21.15.42:0/4082420602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b1e750 msgr2=0x55b391b1c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b1e750 0x55b391b1c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 -- 172.21.15.42:0/4082420602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15b70 msgr2=0x55b391b1bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15b70 0x55b391b1bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.655+0000 7f5884a65640 1 -- 172.21.15.42:0/4082420602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587400b9a0 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f5884a65640 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1b780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f587401afa0 tx=0x7f5874005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25427dd7cbc923a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f585ffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f585ffff640 1 -- 172.21.15.42:0/4082420602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874005670 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f585ffff640 1 -- 172.21.15.42:0/4082420602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f58740119f0 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b39177c7e0 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f585ffff640 1 -- 172.21.15.42:0/4082420602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5874011bb0 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 10 client.16659 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b391a7b6a0 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f585ffff640 1 -- 172.21.15.42:0/4082420602 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b39177c7e0 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f585ffff640 1 -- 172.21.15.42:0/4082420602 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5874023030 con 0x55b391b15250 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 10 client.16659 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 10 client.16659 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:19:23.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 20 client.16659 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b2eb60 0x55b391b30f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 4 client.16659 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.656+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3917b9720 con 0x55b391b2eb60 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.657+0000 7f587ffff640 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b2eb60 0x55b391b30f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.657+0000 7f585ffff640 10 client.16659 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.657+0000 7f587ffff640 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b2eb60 0x55b391b30f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97400928fd35078a server_cookie=13a9ce1a138d3756 in_seq=0 out_seq=0 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.657+0000 7f585ffff640 1 -- 172.21.15.42:0/4082420602 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3917b9720 con 0x55b391b2eb60 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.657+0000 7f585ffff640 10 client.16659 handle_command_reply: tid=0 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.657+0000 7f588997b740 1 client.16659 shutdown 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 msgr2=0x55b391b1b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1b780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f587401afa0 tx=0x7f5874005270 comp rx=0 tx=0).stop 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 shutdown_connections 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b1e750 0x55b391b1c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15b70 0x55b391b1bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b2eb60 0x55b391b30f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 --2- 172.21.15.42:0/4082420602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1b780 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 >> 172.21.15.42:0/4082420602 conn(0x55b391b0f980 msgr2=0x55b391b116d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f585ffff640 0 client.16659 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 shutdown_connections 2024-03-12T04:19:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 1 -- 172.21.15.42:0/4082420602 wait complete. 2024-03-12T04:19:23.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.658+0000 7f588997b740 20 client.16659 trim_cache size 0 max 16384 2024-03-12T04:19:23.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.669+0000 7f588997b740 1 Processor -- start 2024-03-12T04:19:23.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.669+0000 7f588997b740 1 -- start start 2024-03-12T04:19:23.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f588997b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f588997b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b0ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f588997b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f588997b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391a02130 con 0x55b391b15250 2024-03-12T04:19:23.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f588997b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391b0b5f0 con 0x55b391b1e750 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f588997b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391b10a00 con 0x55b391b15b70 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57030/0 (socket says 172.21.15.42:57030) 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 -- 172.21.15.42:0/472803650 learned_addr learned my addr 172.21.15.42:0/472803650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5885266640 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 -- 172.21.15.42:0/472803650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 msgr2=0x55b391b0ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b0ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 -- 172.21.15.42:0/472803650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 msgr2=0x55b391b25600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b25600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 -- 172.21.15.42:0/472803650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b391a17c20 con 0x55b391b15250 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5885266640 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b25600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f5884a65640 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1c020 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f587401afd0 tx=0x7f587400ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d50a7307f19e2e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f586a7fc640 1 -- 172.21.15.42:0/472803650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874005670 con 0x55b391b15250 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f586a7fc640 1 -- 172.21.15.42:0/472803650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58740052e0 con 0x55b391b15250 2024-03-12T04:19:23.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.670+0000 7f586a7fc640 1 -- 172.21.15.42:0/472803650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400b3a0 con 0x55b391b15250 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 -- 172.21.15.42:0/472803650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 msgr2=0x55b391b1c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1c020 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f587401afd0 tx=0x7f587400ff80 comp rx=0 tx=0).stop 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 -- 172.21.15.42:0/472803650 shutdown_connections 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b1e750 0x55b391b25600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b0ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 --2- 172.21.15.42:0/472803650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b15250 0x55b391b1c020 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 -- 172.21.15.42:0/472803650 >> 172.21.15.42:0/472803650 conn(0x55b391b0f980 msgr2=0x55b391b0e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 -- 172.21.15.42:0/472803650 shutdown_connections 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 -- 172.21.15.42:0/472803650 wait complete. 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 -- start start 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.671+0000 7f588997b740 1 Processor -- start 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15250 0x55b391b248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b24e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b1e750 0x55b391b7f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391b7f630 con 0x55b391b1e750 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391b7f7a0 con 0x55b391b15250 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b391b7f960 con 0x55b391b15b70 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 10 client.0 _ll_register_callbacks cb 0x7f587400b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b1e750 0x55b391b7f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b1e750 0x55b391b7f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57032/0 (socket says 172.21.15.42:57032) 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 -- 172.21.15.42:0/2689265306 learned_addr learned my addr 172.21.15.42:0/2689265306 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f587ffff640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b24e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5884a65640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15250 0x55b391b248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 -- 172.21.15.42:0/2689265306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 msgr2=0x55b391b24e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b24e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 -- 172.21.15.42:0/2689265306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15250 msgr2=0x55b391b248f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15250 0x55b391b248f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 -- 172.21.15.42:0/2689265306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b391a7b6a0 con 0x55b391b1e750 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5884a65640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15250 0x55b391b248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f5885266640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b1e750 0x55b391b7f0f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5878009430 tx=0x7f5878032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc21217155d087c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f585ffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f585ffff640 1 -- 172.21.15.42:0/2689265306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780c1030 con 0x55b391b1e750 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f585ffff640 1 -- 172.21.15.42:0/2689265306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f58780c5680 con 0x55b391b1e750 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b391808890 con 0x55b391b1e750 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f585ffff640 1 -- 172.21.15.42:0/2689265306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58780bd140 con 0x55b391b1e750 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 10 client.16665 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.672+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b391a3d8c0 con 0x55b391b1e750 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f585ffff640 1 -- 172.21.15.42:0/2689265306 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b391808890 con 0x55b391b1e750 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f585ffff640 1 -- 172.21.15.42:0/2689265306 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f58780d7d20 con 0x55b391b1e750 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 10 client.16665 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 10 client.16665 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 20 client.16665 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b86f10 0x55b391b89330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 4 client.16665 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b391728700 con 0x55b391b86f10 2024-03-12T04:19:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f5884a65640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b86f10 0x55b391b89330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f5884a65640 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b86f10 0x55b391b89330 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1eec99e40357916 server_cookie=d53694560fb24e0d in_seq=0 out_seq=0 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f585ffff640 10 client.16665 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f585ffff640 1 -- 172.21.15.42:0/2689265306 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b391728700 con 0x55b391b86f10 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f585ffff640 10 client.16665 handle_command_reply: tid=0 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 1 client.16665 shutdown 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.673+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b1e750 msgr2=0x55b391b7f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b1e750 0x55b391b7f0f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5878009430 tx=0x7f5878032470 comp rx=0 tx=0).stop 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 shutdown_connections 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55b391b86f10 0x55b391b89330 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b391b1e750 0x55b391b7f0f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f585ffff640 0 client.16665 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b391b15b70 0x55b391b24e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 --2- 172.21.15.42:0/2689265306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b391b15250 0x55b391b248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 >> 172.21.15.42:0/2689265306 conn(0x55b391b0f980 msgr2=0x55b391b0eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 shutdown_connections 2024-03-12T04:19:23.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 1 -- 172.21.15.42:0/2689265306 wait complete. 2024-03-12T04:19:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f588997b740 20 client.16665 trim_cache size 0 max 16384 2024-03-12T04:19:23.682 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58700a1ba0 msgr2=0x7f58700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58700a1ba0 0x7f58700a4060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5880100d10 tx=0x7f586c00a670 comp rx=0 tx=0).stop 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f588019bfd0 msgr2=0x7f588019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f588019bfd0 0x7f588019c480 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5878004af0 tx=0x7f5878035ca0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 shutdown_connections 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58700a1ba0 0x7f58700a4060 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f588019bfd0 0x7f588019c480 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880106450 0x7f5880197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 --2- 172.21.15.42:0/4117669965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5880101380 0x7f5880197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 >> 172.21.15.42:0/4117669965 conn(0x7f5880078580 msgr2=0x7f58800ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 shutdown_connections 2024-03-12T04:19:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.674+0000 7f5886cc9640 1 -- 172.21.15.42:0/4117669965 wait complete. 2024-03-12T04:19:23.687 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:23.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:19:23.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 -- 172.21.15.42:0/273384514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c101380 msgr2=0x7f6a7c101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/273384514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c101380 0x7f6a7c101760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009930 tx=0x7f6a6c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:23.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 -- 172.21.15.42:0/273384514 shutdown_connections 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/273384514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a7c106450 0x7f6a7c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/273384514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a7c101d30 0x7f6a7c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/273384514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c101380 0x7f6a7c101760 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 -- 172.21.15.42:0/273384514 >> 172.21.15.42:0/273384514 conn(0x7f6a7c078580 msgr2=0x7f6a7c078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 -- 172.21.15.42:0/273384514 shutdown_connections 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 -- 172.21.15.42:0/273384514 wait complete. 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.790+0000 7f6a81d8a640 1 Processor -- start 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a81d8a640 1 -- start start 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a81d8a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a7c101380 0x7f6a7c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a81d8a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a7c106450 0x7f6a7c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a81d8a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c073f00 0x7f6a7c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a81d8a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a5ee0 con 0x7f6a7c073f00 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a81d8a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a6050 con 0x7f6a7c106450 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a81d8a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a6210 con 0x7f6a7c101380 2024-03-12T04:19:23.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c073f00 0x7f6a7c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c073f00 0x7f6a7c1a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57046/0 (socket says 172.21.15.42:57046) 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 -- 172.21.15.42:0/4259865982 learned_addr learned my addr 172.21.15.42:0/4259865982 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 -- 172.21.15.42:0/4259865982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a7c101380 msgr2=0x7f6a7c073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a7c101380 0x7f6a7c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 -- 172.21.15.42:0/4259865982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a7c106450 msgr2=0x7f6a7c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a7c106450 0x7f6a7c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 -- 172.21.15.42:0/4259865982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c009560 con 0x7f6a7c073f00 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.791+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c073f00 0x7f6a7c1a5970 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400eaa0 tx=0x7f6a7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.792+0000 7f6a78ff9640 1 -- 172.21.15.42:0/4259865982 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400cdb0 con 0x7f6a7c073f00 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.792+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c1a64f0 con 0x7f6a7c073f00 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.792+0000 7f6a78ff9640 1 -- 172.21.15.42:0/4259865982 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a74005020 con 0x7f6a7c073f00 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.792+0000 7f6a78ff9640 1 -- 172.21.15.42:0/4259865982 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a740116b0 con 0x7f6a7c073f00 2024-03-12T04:19:23.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.792+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1a6a00 con 0x7f6a7c073f00 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.794+0000 7f6a78ff9640 1 -- 172.21.15.42:0/4259865982 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f6a74005190 con 0x7f6a7c073f00 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.794+0000 7f6a84a35740 1 Processor -- start 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.794+0000 7f6a84a35740 1 -- start start 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.794+0000 7f6a78ff9640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6a640a1ba0 0x7f6a640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.794+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154e5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.794+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e68210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.794+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e68750 0x55a154e6ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a84a35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154e1a960 con 0x55a154e5f250 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a84a35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154dce310 con 0x55a154e5fb70 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a84a35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154dcb840 con 0x55a154e68750 2024-03-12T04:19:23.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154e5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7affd640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6a640a1ba0 0x7f6a640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154e5f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57052/0 (socket says 172.21.15.42:57052) 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/4070033114 learned_addr learned my addr 172.21.15.42:0/4070033114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a78ff9640 1 -- 172.21.15.42:0/4259865982 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a740cda60 con 0x7f6a7c073f00 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e68750 0x55a154e6ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7affd640 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e68210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/4070033114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e68750 msgr2=0x55a154e6ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e68750 0x55a154e6ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/4070033114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 msgr2=0x55a154e68210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e68210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/4070033114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a154d89060 con 0x55a154e5f250 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7affd640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6a640a1ba0 0x7f6a640a4060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a70005d00 tx=0x7f6a70009f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154e5f630 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009430 tx=0x7f6a6c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb54cf172c8002b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a61ffb640 1 -- 172.21.15.42:0/4070033114 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c004840 con 0x55a154e5f250 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a61ffb640 1 -- 172.21.15.42:0/4070033114 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a6c02de90 con 0x55a154e5f250 2024-03-12T04:19:23.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.795+0000 7f6a61ffb640 1 -- 172.21.15.42:0/4070033114 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0035f0 con 0x55a154e5f250 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 -- 172.21.15.42:0/4070033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 msgr2=0x55a154e5f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154e5f630 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009430 tx=0x7f6a6c005e80 comp rx=0 tx=0).stop 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 -- 172.21.15.42:0/4070033114 shutdown_connections 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e68750 0x55a154e6ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e68210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 --2- 172.21.15.42:0/4070033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154e5f630 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 -- 172.21.15.42:0/4070033114 >> 172.21.15.42:0/4070033114 conn(0x55a154e59980 msgr2=0x55a154e59d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 -- 172.21.15.42:0/4070033114 shutdown_connections 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 -- 172.21.15.42:0/4070033114 wait complete. 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.796+0000 7f6a84a35740 1 -- start start 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 1 Processor -- start 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5fb70 0x55a154e65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154dc7020 con 0x55a154e5fb70 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154a9a430 con 0x55a154e68750 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154a56e00 con 0x55a154e5f250 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a84a35740 10 client.0 _ll_register_callbacks cb 0x55a154d6ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5fb70 0x55a154e65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5fb70 0x55a154e65cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57056/0 (socket says 172.21.15.42:57056) 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 -- 172.21.15.42:0/914834718 learned_addr learned my addr 172.21.15.42:0/914834718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 -- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 msgr2=0x55a154e65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 -- 172.21.15.42:0/914834718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 msgr2=0x55a154e661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 -- 172.21.15.42:0/914834718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c10e830 con 0x55a154e5fb70 2024-03-12T04:19:23.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a7affd640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5fb70 0x55a154e65cb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6a70004430 tx=0x7f6a70005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36969b32d92e225 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.797+0000 7f6a5affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a5affd640 1 -- 172.21.15.42:0/914834718 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7001d810 con 0x55a154e5fb70 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a5affd640 1 -- 172.21.15.42:0/914834718 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a70018040 con 0x55a154e5fb70 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a154ac67e0 con 0x55a154e5fb70 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a5affd640 1 -- 172.21.15.42:0/914834718 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a7001d970 con 0x55a154e5fb70 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 10 client.16677 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a154dc56a0 con 0x55a154e5fb70 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a5affd640 1 -- 172.21.15.42:0/914834718 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a154ac67e0 con 0x55a154e5fb70 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a5affd640 1 -- 172.21.15.42:0/914834718 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a70012b20 con 0x55a154e5fb70 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 10 client.16677 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 10 client.16677 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 20 client.16677 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154e78a90 0x55a154e7aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 4 client.16677 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a154b03720 con 0x55a154e78a90 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.798+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154e78a90 0x55a154e7aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.799+0000 7f6a5affd640 10 client.16677 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.799+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154e78a90 0x55a154e7aeb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bcb1bfdf8093a3d9 server_cookie=1d344d7ab4c4b62a in_seq=0 out_seq=0 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a5affd640 1 -- 172.21.15.42:0/914834718 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a154b03720 con 0x55a154e78a90 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a5affd640 10 client.16677 handle_command_reply: tid=0 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 client.16677 shutdown 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5fb70 msgr2=0x55a154e65cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5fb70 0x55a154e65cb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6a70004430 tx=0x7f6a70005850 comp rx=0 tx=0).stop 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 shutdown_connections 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154e78a90 0x55a154e7aeb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a5affd640 0 client.16677 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:23.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5fb70 0x55a154e65cb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 --2- 172.21.15.42:0/914834718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 >> 172.21.15.42:0/914834718 conn(0x55a154e59980 msgr2=0x55a154e5b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 shutdown_connections 2024-03-12T04:19:23.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 1 -- 172.21.15.42:0/914834718 wait complete. 2024-03-12T04:19:23.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.800+0000 7f6a84a35740 20 client.16677 trim_cache size 0 max 16384 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.811+0000 7f6a84a35740 1 Processor -- start 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a84a35740 1 -- start start 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e66010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e68750 0x55a154e6f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a84a35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154d4c130 con 0x55a154e68750 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a84a35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154e555c0 con 0x55a154e5fb70 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a84a35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154e80130 con 0x55a154e5f250 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e68750 0x55a154e6f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e68750 0x55a154e6f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57072/0 (socket says 172.21.15.42:57072) 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 -- 172.21.15.42:0/2349319283 learned_addr learned my addr 172.21.15.42:0/2349319283 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 -- 172.21.15.42:0/2349319283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 msgr2=0x55a154e66010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e66010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 -- 172.21.15.42:0/2349319283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 msgr2=0x55a154e58a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7affd640 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e58a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 -- 172.21.15.42:0/2349319283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a154d61c20 con 0x55a154e68750 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e68750 0x55a154e6f5a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6a7409a610 tx=0x7f6a7409a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ca6b0b494b98ff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a617fa640 1 -- 172.21.15.42:0/2349319283 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7406c420 con 0x55a154e68750 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a617fa640 1 -- 172.21.15.42:0/2349319283 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a7409a680 con 0x55a154e68750 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.812+0000 7f6a617fa640 1 -- 172.21.15.42:0/2349319283 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74005520 con 0x55a154e68750 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 -- 172.21.15.42:0/2349319283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e68750 msgr2=0x55a154e6f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e68750 0x55a154e6f5a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6a7409a610 tx=0x7f6a7409a540 comp rx=0 tx=0).stop 2024-03-12T04:19:23.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 -- 172.21.15.42:0/2349319283 shutdown_connections 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e68750 0x55a154e6f5a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e5fb70 0x55a154e58a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 --2- 172.21.15.42:0/2349319283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5f250 0x55a154e66010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 -- 172.21.15.42:0/2349319283 >> 172.21.15.42:0/2349319283 conn(0x55a154e59980 msgr2=0x55a154e582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 -- 172.21.15.42:0/2349319283 shutdown_connections 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 -- 172.21.15.42:0/2349319283 wait complete. 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.813+0000 7f6a84a35740 1 -- start start 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 1 Processor -- start 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154ec9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5fb70 0x55a154ec9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e6eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154e6f1c0 con 0x55a154e5f250 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154e6f330 con 0x55a154e68750 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a154e6f4f0 con 0x55a154e5fb70 2024-03-12T04:19:23.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a84a35740 10 client.0 _ll_register_callbacks cb 0x7f6a6c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154ec9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154ec9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57082/0 (socket says 172.21.15.42:57082) 2024-03-12T04:19:23.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/556217174 learned_addr learned my addr 172.21.15.42:0/556217174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e6eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.819 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:23 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3441101841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7affd640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5fb70 0x55a154ec9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5fb70 msgr2=0x55a154ec9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5fb70 0x55a154ec9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/556217174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 msgr2=0x55a154e6eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e6eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7b7fe640 1 -- 172.21.15.42:0/556217174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1a6a00 con 0x55a154e5f250 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.814+0000 7f6a7bfff640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e6eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a7b7fe640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154ec9730 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009900 tx=0x7f6a6c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eba18a0bc9b8bb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a5affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a5affd640 1 -- 172.21.15.42:0/556217174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c053070 con 0x55a154e5f250 2024-03-12T04:19:23.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a5affd640 1 -- 172.21.15.42:0/556217174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a6c02c980 con 0x55a154e5f250 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a6c02de90 con 0x55a154e5f250 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a5affd640 1 -- 172.21.15.42:0/556217174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a6c02cb60 con 0x55a154e5f250 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 10 client.16683 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a154d878c0 con 0x55a154e5f250 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a5affd640 1 -- 172.21.15.42:0/556217174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a6c02de90 con 0x55a154e5f250 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a5affd640 1 -- 172.21.15.42:0/556217174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a6c056030 con 0x55a154e5f250 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 10 client.16683 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 10 client.16683 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 20 client.16683 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154ed1290 0x55a154ed36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 4 client.16683 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.815+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a154b52890 con 0x55a154ed1290 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a7affd640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154ed1290 0x55a154ed36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a5affd640 10 client.16683 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a7affd640 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154ed1290 0x55a154ed36b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e9d157a3b674c8f server_cookie=6ebe49ff62e0dc32 in_seq=0 out_seq=0 2024-03-12T04:19:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a5affd640 1 -- 172.21.15.42:0/556217174 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a154b52890 con 0x55a154ed1290 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a5affd640 10 client.16683 handle_command_reply: tid=0 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 client.16683 shutdown 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 msgr2=0x55a154ec9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154ec9730 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009900 tx=0x7f6a6c005850 comp rx=0 tx=0).stop 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 shutdown_connections 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55a154ed1290 0x55a154ed36b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a154e68750 0x55a154e6eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a5affd640 0 client.16683 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a154e5fb70 0x55a154ec9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 --2- 172.21.15.42:0/556217174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a154e5f250 0x55a154ec9730 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 >> 172.21.15.42:0/556217174 conn(0x55a154e59980 msgr2=0x55a154e58b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 shutdown_connections 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 1 -- 172.21.15.42:0/556217174 wait complete. 2024-03-12T04:19:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.816+0000 7f6a84a35740 20 client.16683 trim_cache size 0 max 16384 2024-03-12T04:19:23.825 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:23.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6a640a1ba0 msgr2=0x7f6a640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6a640a1ba0 0x7f6a640a4060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a70005d00 tx=0x7f6a70009f90 comp rx=0 tx=0).stop 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c073f00 msgr2=0x7f6a7c1a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c073f00 0x7f6a7c1a5970 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400eaa0 tx=0x7f6a7400ef60 comp rx=0 tx=0).stop 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 shutdown_connections 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6a640a1ba0 0x7f6a640a4060 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a7c073f00 0x7f6a7c1a5970 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a7c106450 0x7f6a7c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 --2- 172.21.15.42:0/4259865982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a7c101380 0x7f6a7c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 >> 172.21.15.42:0/4259865982 conn(0x7f6a7c078580 msgr2=0x7f6a7c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 shutdown_connections 2024-03-12T04:19:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.817+0000 7f6a81d8a640 1 -- 172.21.15.42:0/4259865982 wait complete. 2024-03-12T04:19:23.829 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:23.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 -- 172.21.15.42:0/1163059011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 msgr2=0x7f91d010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 --2- 172.21.15.42:0/1163059011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d010e020 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009cd0 tx=0x7f91c802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 -- 172.21.15.42:0/1163059011 shutdown_connections 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 --2- 172.21.15.42:0/1163059011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d010e020 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 --2- 172.21.15.42:0/1163059011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0101d30 0x7f91d0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 --2- 172.21.15.42:0/1163059011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0101380 0x7f91d0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 -- 172.21.15.42:0/1163059011 >> 172.21.15.42:0/1163059011 conn(0x7f91d0078580 msgr2=0x7f91d0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 -- 172.21.15.42:0/1163059011 shutdown_connections 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.932+0000 7f91d5a83640 1 -- 172.21.15.42:0/1163059011 wait complete. 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 Processor -- start 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 -- start start 2024-03-12T04:19:23.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0101380 0x7f91d01976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d0197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d019bf70 0x7f91d019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d019c960 con 0x7f91d0106450 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d019cad0 con 0x7f91d019bf70 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91d5a83640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d019cc40 con 0x7f91d0101380 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d0197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d0197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57092/0 (socket says 172.21.15.42:57092) 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 -- 172.21.15.42:0/2110732414 learned_addr learned my addr 172.21.15.42:0/2110732414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91cffff640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d019bf70 0x7f91d019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 -- 172.21.15.42:0/2110732414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0101380 msgr2=0x7f91d01976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0101380 0x7f91d01976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0101380 0x7f91d01976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 -- 172.21.15.42:0/2110732414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d019bf70 msgr2=0x7f91d019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d019bf70 0x7f91d019c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91ceffd640 1 -- 172.21.15.42:0/2110732414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009900 con 0x7f91d0106450 2024-03-12T04:19:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.933+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0101380 0x7f91d01976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:23.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.934+0000 7f91ceffd640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d0197bf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005fd0 tx=0x7f91c400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.934+0000 7f91ccff9640 1 -- 172.21.15.42:0/2110732414 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4009c30 con 0x7f91d0106450 2024-03-12T04:19:23.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.934+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d01a4220 con 0x7f91d0106450 2024-03-12T04:19:23.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.934+0000 7f91ccff9640 1 -- 172.21.15.42:0/2110732414 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91c40044d0 con 0x7f91d0106450 2024-03-12T04:19:23.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.934+0000 7f91ccff9640 1 -- 172.21.15.42:0/2110732414 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4003e50 con 0x7f91d0106450 2024-03-12T04:19:23.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.934+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d01a4720 con 0x7f91d0106450 2024-03-12T04:19:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91ccff9640 1 -- 172.21.15.42:0/2110732414 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f91c4004000 con 0x7f91d0106450 2024-03-12T04:19:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 Processor -- start 2024-03-12T04:19:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 -- start start 2024-03-12T04:19:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a21428630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428b70 0x562a21431210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a21433b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a213e3960 con 0x562a21428250 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a21397310 con 0x562a21428b70 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91d864b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a21394840 con 0x562a21431750 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a21428630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a21428630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57104/0 (socket says 172.21.15.42:57104) 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 -- 172.21.15.42:0/3708545718 learned_addr learned my addr 172.21.15.42:0/3708545718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428b70 0x562a21431210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91ccff9640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f91b80a1ba0 0x7f91b80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91ceffd640 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a21433b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 -- 172.21.15.42:0/3708545718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 msgr2=0x562a21433b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a21433b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 -- 172.21.15.42:0/3708545718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428b70 msgr2=0x562a21431210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428b70 0x562a21431210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91ccff9640 1 -- 172.21.15.42:0/2110732414 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f91c40cd4e0 con 0x7f91d0106450 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91cffff640 1 -- 172.21.15.42:0/3708545718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a21352060 con 0x562a21428250 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.937+0000 7f91ceffd640 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a21433b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.938+0000 7f91cffff640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f91b80a1ba0 0x7f91b80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.938+0000 7f91cffff640 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a21428630 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f91c802c700 tx=0x7f91c8032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17c87f38794b0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.938+0000 7f91b5ffb640 1 -- 172.21.15.42:0/3708545718 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8034e20 con 0x562a21428250 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.938+0000 7f91b5ffb640 1 -- 172.21.15.42:0/3708545718 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91c8032a80 con 0x562a21428250 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.938+0000 7f91b5ffb640 1 -- 172.21.15.42:0/3708545718 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80042b0 con 0x562a21428250 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- 172.21.15.42:0/3708545718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 msgr2=0x562a21428630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91cffff640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f91b80a1ba0 0x7f91b80a4060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91c8005e30 tx=0x7f91c8045000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a21428630 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f91c802c700 tx=0x7f91c8032340 comp rx=0 tx=0).stop 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- 172.21.15.42:0/3708545718 shutdown_connections 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a21433b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428b70 0x562a21431210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 --2- 172.21.15.42:0/3708545718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a21428630 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- 172.21.15.42:0/3708545718 >> 172.21.15.42:0/3708545718 conn(0x562a21422980 msgr2=0x562a21422d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- 172.21.15.42:0/3708545718 shutdown_connections 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- 172.21.15.42:0/3708545718 wait complete. 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- start start 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 Processor -- start 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a2142e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428b70 0x562a2142ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a2142f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a21390020 con 0x562a21428250 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a21063430 con 0x562a21431750 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2101fe00 con 0x562a21428b70 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.939+0000 7f91d864b740 10 client.0 _ll_register_callbacks cb 0x562a21337cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a2142e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a2142e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57108/0 (socket says 172.21.15.42:57108) 2024-03-12T04:19:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 -- 172.21.15.42:0/1012138614 learned_addr learned my addr 172.21.15.42:0/1012138614 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91ceffd640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428b70 0x562a2142ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 -- 172.21.15.42:0/1012138614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428b70 msgr2=0x562a2142ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cffff640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a2142f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428b70 0x562a2142ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 -- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 msgr2=0x562a2142f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a2142f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 -- 172.21.15.42:0/1012138614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d010e830 con 0x562a21428250 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91ceffd640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428b70 0x562a2142ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a2142e740 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e90 tx=0x7f91c00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf89b1e153d6a158 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91aeffd640 1 -- 172.21.15.42:0/1012138614 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0012860 con 0x562a21428250 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91aeffd640 1 -- 172.21.15.42:0/1012138614 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f91c00129c0 con 0x562a21428250 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a2108f7e0 con 0x562a21428250 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91aeffd640 1 -- 172.21.15.42:0/1012138614 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91c0012b80 con 0x562a21428250 2024-03-12T04:19:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91d864b740 10 client.16695 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.940+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562a2138e6a0 con 0x562a21428250 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91aeffd640 1 -- 172.21.15.42:0/1012138614 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91c0027080 con 0x562a21428250 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91aeffd640 1 -- 172.21.15.42:0/1012138614 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91c000c070 con 0x562a21428250 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91d864b740 10 client.16695 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91d864b740 10 client.16695 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91d864b740 20 client.16695 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91d864b740 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a21441bc0 0x562a21444000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91d864b740 4 client.16695 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a210cc720 con 0x562a21441bc0 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.941+0000 7f91ceffd640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a21441bc0 0x562a21444000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91aeffd640 10 client.16695 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91ceffd640 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a21441bc0 0x562a21444000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcce9d270e2816e7 server_cookie=fe0d21a22805b74 in_seq=0 out_seq=0 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91aeffd640 1 -- 172.21.15.42:0/1012138614 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a210cc720 con 0x562a21441bc0 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91aeffd640 10 client.16695 handle_command_reply: tid=0 2024-03-12T04:19:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 client.16695 shutdown 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 msgr2=0x562a2142e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a2142e740 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e90 tx=0x7f91c00117b0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 shutdown_connections 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a21441bc0 0x562a21444000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a2142f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91aeffd640 0 client.16695 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428b70 0x562a2142ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 --2- 172.21.15.42:0/1012138614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428250 0x562a2142e740 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.942+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 >> 172.21.15.42:0/1012138614 conn(0x562a21422980 msgr2=0x562a214246d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.943+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 shutdown_connections 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.943+0000 7f91d864b740 1 -- 172.21.15.42:0/1012138614 wait complete. 2024-03-12T04:19:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.943+0000 7f91d864b740 20 client.16695 trim_cache size 0 max 16384 2024-03-12T04:19:23.951 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:23 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3441101841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:19:23.961 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:23 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3441101841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 Processor -- start 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 -- start start 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428250 0x562a21430e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a214389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a2142efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a21315130 con 0x562a21428b70 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2142a3a0 con 0x562a21428250 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91d864b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2142cd10 con 0x562a21431750 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.954+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a214389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a214389e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57110/0 (socket says 172.21.15.42:57110) 2024-03-12T04:19:23.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 -- 172.21.15.42:0/4147412682 learned_addr learned my addr 172.21.15.42:0/4147412682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428250 0x562a21430e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91cffff640 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a2142efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 -- 172.21.15.42:0/4147412682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 msgr2=0x562a2142efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a2142efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 -- 172.21.15.42:0/4147412682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428250 msgr2=0x562a21430e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428250 0x562a21430e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 -- 172.21.15.42:0/4147412682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a2132ac20 con 0x562a21428b70 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428250 0x562a21430e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91ceffd640 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a214389e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f91c409a0f0 tx=0x7f91c401f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3aef207d688684 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91b57fa640 1 -- 172.21.15.42:0/4147412682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4020ec0 con 0x562a21428b70 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91b57fa640 1 -- 172.21.15.42:0/4147412682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91c4009d90 con 0x562a21428b70 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.955+0000 7f91b57fa640 1 -- 172.21.15.42:0/4147412682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4098200 con 0x562a21428b70 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- 172.21.15.42:0/4147412682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 msgr2=0x562a214389e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a214389e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f91c409a0f0 tx=0x7f91c401f840 comp rx=0 tx=0).stop 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- 172.21.15.42:0/4147412682 shutdown_connections 2024-03-12T04:19:23.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21431750 0x562a2142efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a214389e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 --2- 172.21.15.42:0/4147412682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21428250 0x562a21430e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- 172.21.15.42:0/4147412682 >> 172.21.15.42:0/4147412682 conn(0x562a21422980 msgr2=0x562a2141da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- 172.21.15.42:0/4147412682 shutdown_connections 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- 172.21.15.42:0/4147412682 wait complete. 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- start start 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 Processor -- start 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428250 0x562a21496810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a21496d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a21490c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a214911e0 con 0x562a21428b70 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a21491350 con 0x562a21431750 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a21491510 con 0x562a21428250 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.956+0000 7f91d864b740 10 client.0 _ll_register_callbacks cb 0x7f91c8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a21496d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a21496d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57116/0 (socket says 172.21.15.42:57116) 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 -- 172.21.15.42:0/3942684625 learned_addr learned my addr 172.21.15.42:0/3942684625 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428250 0x562a21496810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91cffff640 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a21490c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 -- 172.21.15.42:0/3942684625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428250 msgr2=0x562a21496810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428250 0x562a21496810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 -- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 msgr2=0x562a21490c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a21490c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 -- 172.21.15.42:0/3942684625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d01a4720 con 0x562a21428b70 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91ceffd640 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a21496d50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f91c4096b30 tx=0x7f91c400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37441154ce8c3d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91aeffd640 1 -- 172.21.15.42:0/3942684625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c409e2b0 con 0x562a21428b70 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91aeffd640 1 -- 172.21.15.42:0/3942684625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f91c409a860 con 0x562a21428b70 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91c8032a80 con 0x562a21428b70 2024-03-12T04:19:23.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91aeffd640 1 -- 172.21.15.42:0/3942684625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91c4098460 con 0x562a21428b70 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91d864b740 10 client.16701 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562a213508c0 con 0x562a21428b70 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.957+0000 7f91aeffd640 1 -- 172.21.15.42:0/3942684625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91c40b2e90 con 0x562a21428b70 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91aeffd640 1 -- 172.21.15.42:0/3942684625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91c4022020 con 0x562a21428b70 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91d864b740 10 client.16701 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91d864b740 10 client.16701 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91d864b740 20 client.16701 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91d864b740 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a2149e3f0 0x562a214a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91d864b740 4 client.16701 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a2111b890 con 0x562a2149e3f0 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a2149e3f0 0x562a214a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91aeffd640 10 client.16701 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91cf7fe640 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a2149e3f0 0x562a214a0810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb069c71d1fb1085 server_cookie=efaccce0b4cde260 in_seq=0 out_seq=0 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91aeffd640 1 -- 172.21.15.42:0/3942684625 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a2111b890 con 0x562a2149e3f0 2024-03-12T04:19:23.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91aeffd640 10 client.16701 handle_command_reply: tid=0 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.958+0000 7f91d864b740 1 client.16701 shutdown 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 msgr2=0x562a21496d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a21496d50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f91c4096b30 tx=0x7f91c400bf60 comp rx=0 tx=0).stop 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 shutdown_connections 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x562a2149e3f0 0x562a214a0810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a21431750 0x562a21490c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a21428b70 0x562a21496d50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 --2- 172.21.15.42:0/3942684625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a21428250 0x562a21496810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91aeffd640 0 client.16701 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 >> 172.21.15.42:0/3942684625 conn(0x562a21422980 msgr2=0x562a214489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 shutdown_connections 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 1 -- 172.21.15.42:0/3942684625 wait complete. 2024-03-12T04:19:23.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d864b740 20 client.16701 trim_cache size 0 max 16384 2024-03-12T04:19:23.968 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:23.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f91b80a1ba0 msgr2=0x7f91b80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f91b80a1ba0 0x7f91b80a4060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91c8005e30 tx=0x7f91c8045000 comp rx=0 tx=0).stop 2024-03-12T04:19:23.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 msgr2=0x7f91d0197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:23.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d0197bf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005fd0 tx=0x7f91c400dde0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 shutdown_connections 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f91b80a1ba0 0x7f91b80a4060 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d019bf70 0x7f91d019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d0106450 0x7f91d0197bf0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 --2- 172.21.15.42:0/2110732414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0101380 0x7f91d01976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 >> 172.21.15.42:0/2110732414 conn(0x7f91d0078580 msgr2=0x7f91d00ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 shutdown_connections 2024-03-12T04:19:23.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:23.959+0000 7f91d5a83640 1 -- 172.21.15.42:0/2110732414 wait complete. 2024-03-12T04:19:23.971 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:23.972 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:19:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/2794289315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 msgr2=0x7f3fac100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/2794289315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac100c10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009900 tx=0x7f3f9802bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/2794289315 shutdown_connections 2024-03-12T04:19:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/2794289315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac101150 0x7f3fac10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/2794289315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac100c10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/2794289315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fac105070 0x7f3fac105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/2794289315 >> 172.21.15.42:0/2794289315 conn(0x7f3fac076640 msgr2=0x7f3fac0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/2794289315 shutdown_connections 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/2794289315 wait complete. 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 Processor -- start 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.075+0000 7f3fb2d8c640 1 -- start start 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb2d8c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb2d8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac101150 0x7f3fac196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb2d8c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fac105070 0x7f3fac19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb2d8c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac19b3f0 con 0x7f3fac100790 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb2d8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac19b560 con 0x7f3fac101150 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb2d8c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac19b720 con 0x7f3fac105070 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57126/0 (socket says 172.21.15.42:57126) 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 -- 172.21.15.42:0/4061716532 learned_addr learned my addr 172.21.15.42:0/4061716532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fabfff640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac101150 0x7f3fac196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 -- 172.21.15.42:0/4061716532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fac105070 msgr2=0x7f3fac19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb1329640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fac105070 0x7f3fac19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fac105070 0x7f3fac19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 -- 172.21.15.42:0/4061716532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac101150 msgr2=0x7f3fac196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac101150 0x7f3fac196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 -- 172.21.15.42:0/4061716532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f98009560 con 0x7f3fac100790 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fabfff640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac101150 0x7f3fac196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb1329640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fac105070 0x7f3fac19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac196530 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000d770 tx=0x7f3fa000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fa9ffb640 1 -- 172.21.15.42:0/4061716532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0009c30 con 0x7f3fac100790 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fa9ffb640 1 -- 172.21.15.42:0/4061716532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fa000de80 con 0x7f3fac100790 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.076+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac19ba00 con 0x7f3fac100790 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.077+0000 7f3fa9ffb640 1 -- 172.21.15.42:0/4061716532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0004e00 con 0x7f3fac100790 2024-03-12T04:19:24.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.077+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac1a3280 con 0x7f3fac100790 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fa9ffb640 1 -- 172.21.15.42:0/4061716532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3fa0004f60 con 0x7f3fac100790 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fa9ffb640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3f9c0a1c20 0x7f3f9c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 Processor -- start 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 -- start start 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fabfff640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3f9c0a1c20 0x7f3f9c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fa9ffb640 1 -- 172.21.15.42:0/4061716532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3fa00cd1c0 con 0x7f3fac100790 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeda2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ee95960 con 0x56247eee5960 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ee49310 con 0x56247eed9f10 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.079+0000 7f3fb5963740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ee46840 con 0x56247eeda830 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57140/0 (socket says 172.21.15.42:57140) 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 -- 172.21.15.42:0/927666224 learned_addr learned my addr 172.21.15.42:0/927666224 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeda2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fabfff640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3f9c0a1c20 0x7f3f9c0a40e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006010 tx=0x7f3f98005a20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fabfff640 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 -- 172.21.15.42:0/927666224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 msgr2=0x56247eee5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 -- 172.21.15.42:0/927666224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 msgr2=0x56247eeda2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeda2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 -- 172.21.15.42:0/927666224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247ee04060 con 0x56247eee5960 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3fb1329640 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee7d50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009f90 tx=0x7f3fa400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b23fe090366c990 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3f96ffd640 1 -- 172.21.15.42:0/927666224 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004d20 con 0x56247eee5960 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3f96ffd640 1 -- 172.21.15.42:0/927666224 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fa4004e80 con 0x56247eee5960 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.080+0000 7f3f96ffd640 1 -- 172.21.15.42:0/927666224 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4011930 con 0x56247eee5960 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 -- 172.21.15.42:0/927666224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 msgr2=0x56247eee7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee7d50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009f90 tx=0x7f3fa400e780 comp rx=0 tx=0).stop 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 -- 172.21.15.42:0/927666224 shutdown_connections 2024-03-12T04:19:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee7d50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 --2- 172.21.15.42:0/927666224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeda2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 -- 172.21.15.42:0/927666224 >> 172.21.15.42:0/927666224 conn(0x56247eed4980 msgr2=0x56247eed4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 -- 172.21.15.42:0/927666224 shutdown_connections 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 -- 172.21.15.42:0/927666224 wait complete. 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 -- start start 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.081+0000 7f3fb5963740 1 Processor -- start 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eed9f10 0x56247eee28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eeda830 0x56247eee2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ee42020 con 0x56247eee5960 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247eb15430 con 0x56247eeda830 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ead1e00 con 0x56247eed9f10 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 10 client.0 _ll_register_callbacks cb 0x56247ede9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57152/0 (socket says 172.21.15.42:57152) 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 -- 172.21.15.42:0/3213699305 learned_addr learned my addr 172.21.15.42:0/3213699305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fabfff640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eeda830 0x56247eee2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eed9f10 0x56247eee28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 -- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eed9f10 msgr2=0x56247eee28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eed9f10 0x56247eee28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 -- 172.21.15.42:0/3213699305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eeda830 msgr2=0x56247eee2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eeda830 0x56247eee2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 -- 172.21.15.42:0/3213699305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa4009bc0 con 0x56247eee5960 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eed9f10 0x56247eee28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb1329640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee3350 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011550 tx=0x7f3fa40118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a2062055fc32f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3f8bfff640 1 -- 172.21.15.42:0/3213699305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4021070 con 0x56247eee5960 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56247eb7e720 con 0x56247eee5960 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.082+0000 7f3f8bfff640 1 -- 172.21.15.42:0/3213699305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3fa4011be0 con 0x56247eee5960 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3f8bfff640 1 -- 172.21.15.42:0/3213699305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fa4011da0 con 0x56247eee5960 2024-03-12T04:19:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 10 client.16713 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56247ee406a0 con 0x56247eee5960 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3f8bfff640 1 -- 172.21.15.42:0/3213699305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fa4033020 con 0x56247eee5960 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3f8bfff640 1 -- 172.21.15.42:0/3213699305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3fa400c070 con 0x56247eee5960 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 10 client.16713 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 10 client.16713 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 20 client.16713 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247eef5d10 0x56247eef8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 4 client.16713 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.083+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56247ebcd890 con 0x56247eef5d10 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.084+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247eef5d10 0x56247eef8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.084+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247eef5d10 0x56247eef8130 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c95e20f760178d16 server_cookie=9b8f3d7b43e0634f in_seq=0 out_seq=0 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.084+0000 7f3f8bfff640 10 client.16713 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3f8bfff640 1 -- 172.21.15.42:0/3213699305 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56247ebcd890 con 0x56247eef5d10 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3f8bfff640 10 client.16713 handle_command_reply: tid=0 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 client.16713 shutdown 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 msgr2=0x56247eee3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee3350 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011550 tx=0x7f3fa40118e0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 shutdown_connections 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247eef5d10 0x56247eef8130 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247eee3350 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eeda830 0x56247eee2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 --2- 172.21.15.42:0/3213699305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eed9f10 0x56247eee28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3f8bfff640 0 client.16713 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 >> 172.21.15.42:0/3213699305 conn(0x56247eed4980 msgr2=0x56247eed66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 shutdown_connections 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 1 -- 172.21.15.42:0/3213699305 wait complete. 2024-03-12T04:19:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.085+0000 7f3fb5963740 20 client.16713 trim_cache size 0 max 16384 2024-03-12T04:19:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 Processor -- start 2024-03-12T04:19:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 -- start start 2024-03-12T04:19:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eed9f10 0x56247eee5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eee5960 0x56247eedfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247edc7130 con 0x56247eed9f10 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247eed4830 con 0x56247eee5960 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb5963740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247eee7dd0 con 0x56247eeda830 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eed9f10 0x56247eee5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eed9f10 0x56247eee5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57164/0 (socket says 172.21.15.42:57164) 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 -- 172.21.15.42:0/1128853540 learned_addr learned my addr 172.21.15.42:0/1128853540 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fabfff640 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb1329640 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eee5960 0x56247eedfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 -- 172.21.15.42:0/1128853540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 msgr2=0x56247eee3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 -- 172.21.15.42:0/1128853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eee5960 msgr2=0x56247eedfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eee5960 0x56247eedfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 -- 172.21.15.42:0/1128853540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247eddcc20 con 0x56247eed9f10 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.097+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eed9f10 0x56247eee5000 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0096810 tx=0x7f3fa00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a32a11c7b26fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3f967fc640 1 -- 172.21.15.42:0/1128853540 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00b6070 con 0x56247eed9f10 2024-03-12T04:19:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3f967fc640 1 -- 172.21.15.42:0/1128853540 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fa00ade90 con 0x56247eed9f10 2024-03-12T04:19:24.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3f967fc640 1 -- 172.21.15.42:0/1128853540 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00afea0 con 0x56247eed9f10 2024-03-12T04:19:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 -- 172.21.15.42:0/1128853540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eed9f10 msgr2=0x56247eee5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eed9f10 0x56247eee5000 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0096810 tx=0x7f3fa00047a0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 -- 172.21.15.42:0/1128853540 shutdown_connections 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eee5960 0x56247eedfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eee3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 --2- 172.21.15.42:0/1128853540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eed9f10 0x56247eee5000 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 -- 172.21.15.42:0/1128853540 >> 172.21.15.42:0/1128853540 conn(0x56247eed4980 msgr2=0x56247eeddc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 -- 172.21.15.42:0/1128853540 shutdown_connections 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 -- 172.21.15.42:0/1128853540 wait complete. 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.098+0000 7f3fb5963740 1 -- start start 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 1 Processor -- start 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeebc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eeec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247ef451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ef45720 con 0x56247eee5960 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ef45890 con 0x56247eed9f10 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247ef45a50 con 0x56247eeda830 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb5963740 10 client.0 _ll_register_callbacks cb 0x7f3fa4009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247ef451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247ef451e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57170/0 (socket says 172.21.15.42:57170) 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 -- 172.21.15.42:0/578524303 learned_addr learned my addr 172.21.15.42:0/578524303 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fabfff640 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eeec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 -- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 msgr2=0x56247eeec1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eeec1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 -- 172.21.15.42:0/578524303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 msgr2=0x56247eeebc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeebc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.099+0000 7f3fb1329640 1 -- 172.21.15.42:0/578524303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac1a3280 con 0x56247eee5960 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb1329640 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247ef451e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011520 tx=0x7f3fa4009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de542edd58572b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3f8bfff640 1 -- 172.21.15.42:0/578524303 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4021070 con 0x56247eee5960 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3f8bfff640 1 -- 172.21.15.42:0/578524303 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3fa4034080 con 0x56247eee5960 2024-03-12T04:19:24.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56247eb417e0 con 0x56247eee5960 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3f8bfff640 1 -- 172.21.15.42:0/578524303 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fa4005e30 con 0x56247eee5960 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 10 client.16719 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56247ee028c0 con 0x56247eee5960 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3f8bfff640 1 -- 172.21.15.42:0/578524303 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56247eb417e0 con 0x56247eee5960 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3f8bfff640 1 -- 172.21.15.42:0/578524303 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3fa400c040 con 0x56247eee5960 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 10 client.16719 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 10 client.16719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 20 client.16719 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247ef4e000 0x56247ef503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 4 client.16719 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.100+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56247eaed700 con 0x56247ef4e000 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247ef4e000 0x56247ef503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3f8bfff640 10 client.16719 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb0b28640 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247ef4e000 0x56247ef503f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eecd52c018b5f296 server_cookie=65101b9df4cb8595 in_seq=0 out_seq=0 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3f8bfff640 1 -- 172.21.15.42:0/578524303 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56247eaed700 con 0x56247ef4e000 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3f8bfff640 10 client.16719 handle_command_reply: tid=0 2024-03-12T04:19:24.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 client.16719 shutdown 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 msgr2=0x56247ef451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247ef451e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011520 tx=0x7f3fa4009820 comp rx=0 tx=0).stop 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 shutdown_connections 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56247ef4e000 0x56247ef503f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56247eee5960 0x56247ef451e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56247eeda830 0x56247eeec1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3f8bfff640 0 client.16719 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 --2- 172.21.15.42:0/578524303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56247eed9f10 0x56247eeebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 >> 172.21.15.42:0/578524303 conn(0x56247eed4980 msgr2=0x56247eed32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 shutdown_connections 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.101+0000 7f3fb5963740 1 -- 172.21.15.42:0/578524303 wait complete. 2024-03-12T04:19:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb5963740 20 client.16719 trim_cache size 0 max 16384 2024-03-12T04:19:24.109 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3f9c0a1c20 msgr2=0x7f3f9c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3f9c0a1c20 0x7f3f9c0a40e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006010 tx=0x7f3f98005a20 comp rx=0 tx=0).stop 2024-03-12T04:19:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 msgr2=0x7f3fac196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac196530 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000d770 tx=0x7f3fa000dc30 comp rx=0 tx=0).stop 2024-03-12T04:19:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 shutdown_connections 2024-03-12T04:19:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3f9c0a1c20 0x7f3f9c0a40e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fac105070 0x7f3fac19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac101150 0x7f3fac196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 --2- 172.21.15.42:0/4061716532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac100790 0x7f3fac196530 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 >> 172.21.15.42:0/4061716532 conn(0x7f3fac076640 msgr2=0x7f3fac103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 shutdown_connections 2024-03-12T04:19:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.102+0000 7f3fb2d8c640 1 -- 172.21.15.42:0/4061716532 wait complete. 2024-03-12T04:19:24.115 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:24.115 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1683263563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 msgr2=0x7f02bc10c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1683263563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc10c6b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f02b400adb0 tx=0x7f02b402cff0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1683263563 shutdown_connections 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1683263563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc10c6b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1683263563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02bc0ffa40 0x7f02bc0ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1683263563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02bc1047b0 0x7f02bc104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1683263563 >> 172.21.15.42:0/1683263563 conn(0x7f02bc0753c0 msgr2=0x7f02bc0757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1683263563 shutdown_connections 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1683263563 wait complete. 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 Processor -- start 2024-03-12T04:19:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.218+0000 7f02c3fd0640 1 -- start start 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02bc0ffa40 0x7f02bc1014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc1019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02bc1047b0 0x7f02bc101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc1a59c0 con 0x7f02bc1003e0 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc1a5b30 con 0x7f02bc1047b0 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc1a5cf0 con 0x7f02bc0ffa40 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc1019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc1019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57180/0 (socket says 172.21.15.42:57180) 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 -- 172.21.15.42:0/1273368995 learned_addr learned my addr 172.21.15.42:0/1273368995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02bc0ffa40 0x7f02bc1014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c256d640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02bc1047b0 0x7f02bc101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 -- 172.21.15.42:0/1273368995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02bc0ffa40 msgr2=0x7f02bc1014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02bc0ffa40 0x7f02bc1014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 -- 172.21.15.42:0/1273368995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02bc1047b0 msgr2=0x7f02bc101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02bc1047b0 0x7f02bc101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 -- 172.21.15.42:0/1273368995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b400aa10 con 0x7f02bc1003e0 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02bc0ffa40 0x7f02bc1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c256d640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02bc1047b0 0x7f02bc101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c156b640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc1019e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d950 tx=0x7f02ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02b2ffd640 1 -- 172.21.15.42:0/1273368995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac009c30 con 0x7f02bc1003e0 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc1a5fd0 con 0x7f02bc1003e0 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02b2ffd640 1 -- 172.21.15.42:0/1273368995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02ac009d90 con 0x7f02bc1003e0 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02b2ffd640 1 -- 172.21.15.42:0/1273368995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00f6b0 con 0x7f02bc1003e0 2024-03-12T04:19:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.219+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc1a6450 con 0x7f02bc1003e0 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.221+0000 7f02b2ffd640 1 -- 172.21.15.42:0/1273368995 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f02ac010aa0 con 0x7f02bc1003e0 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 Processor -- start 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 -- start start 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2250 0x5641cf1d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1db210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf18d960 con 0x5641cf1d2250 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf141310 con 0x5641cf1db750 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf13e840 con 0x5641cf1d2b70 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02b2ffd640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02a80a1bd0 0x7f02a80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c256d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2250 0x5641cf1d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c256d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2250 0x5641cf1d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57192/0 (socket says 172.21.15.42:57192) 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c256d640 1 -- 172.21.15.42:0/2335782040 learned_addr learned my addr 172.21.15.42:0/2335782040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c256d640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02a80a1bd0 0x7f02a80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.222+0000 7f02c156b640 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02b2ffd640 1 -- 172.21.15.42:0/1273368995 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f02ac0cd4f0 con 0x7f02bc1003e0 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c256d640 1 -- 172.21.15.42:0/2335782040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 msgr2=0x5641cf1db210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c256d640 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1db210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c256d640 1 -- 172.21.15.42:0/2335782040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 msgr2=0x5641cf1ddb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c256d640 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1ddb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c256d640 1 -- 172.21.15.42:0/2335782040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641cf0fc060 con 0x5641cf1d2250 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1db210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c156b640 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c256d640 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2250 0x5641cf1d2630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f02b400adb0 tx=0x7f02b402df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a465b9e395a5ab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02c256d640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02a80a1bd0 0x7f02a80a4090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02b4037000 tx=0x7f02b4042000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02a7fff640 1 -- 172.21.15.42:0/2335782040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40333c0 con 0x5641cf1d2250 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02a7fff640 1 -- 172.21.15.42:0/2335782040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02b402dcc0 con 0x5641cf1d2250 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.223+0000 7f02a7fff640 1 -- 172.21.15.42:0/2335782040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4003970 con 0x5641cf1d2250 2024-03-12T04:19:24.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 -- 172.21.15.42:0/2335782040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2250 msgr2=0x5641cf1d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2250 0x5641cf1d2630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f02b400adb0 tx=0x7f02b402df90 comp rx=0 tx=0).stop 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 -- 172.21.15.42:0/2335782040 shutdown_connections 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1db210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/2335782040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2250 0x5641cf1d2630 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 -- 172.21.15.42:0/2335782040 >> 172.21.15.42:0/2335782040 conn(0x5641cf1cc980 msgr2=0x5641cf1ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 -- 172.21.15.42:0/2335782040 shutdown_connections 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 -- 172.21.15.42:0/2335782040 wait complete. 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 -- start start 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.224+0000 7f02c6ba5740 1 Processor -- start 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2250 0x5641cf1d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf13a020 con 0x5641cf1d2b70 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cee0d430 con 0x5641cf1db750 2024-03-12T04:19:24.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cedc9e00 con 0x5641cf1d2250 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 10 client.0 _ll_register_callbacks cb 0x5641cf0e1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57194/0 (socket says 172.21.15.42:57194) 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 -- 172.21.15.42:0/3007253099 learned_addr learned my addr 172.21.15.42:0/3007253099 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c256d640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2250 0x5641cf1d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 -- 172.21.15.42:0/3007253099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2250 msgr2=0x5641cf1d8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2250 0x5641cf1d8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 -- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 msgr2=0x5641cf1d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1d91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 -- 172.21.15.42:0/3007253099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc10cf20 con 0x5641cf1d2b70 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c256d640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c156b640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d8cb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f02ac000c00 tx=0x7f02ac023f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39675bc27bd5cbe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02a57fa640 1 -- 172.21.15.42:0/3007253099 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac09a870 con 0x5641cf1d2b70 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02a57fa640 1 -- 172.21.15.42:0/3007253099 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f02ac09a170 con 0x5641cf1d2b70 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.225+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641cee397e0 con 0x5641cf1d2b70 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02a57fa640 1 -- 172.21.15.42:0/3007253099 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02ac09e2c0 con 0x5641cf1d2b70 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 10 client.16731 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5641cf1386a0 con 0x5641cf1d2b70 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02a57fa640 1 -- 172.21.15.42:0/3007253099 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5641cee397e0 con 0x5641cf1d2b70 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02a57fa640 1 -- 172.21.15.42:0/3007253099 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02ac0b1060 con 0x5641cf1d2b70 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 10 client.16731 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 20 client.16731 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf1ebbd0 0x5641cf1edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 4 client.16731 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641cee76720 con 0x5641cf1ebbd0 2024-03-12T04:19:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.226+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf1ebbd0 0x5641cf1edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.227+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf1ebbd0 0x5641cf1edfc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e3e31c9919e6c8 server_cookie=f5908bf7736bea11 in_seq=0 out_seq=0 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.227+0000 7f02a57fa640 10 client.16731 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.227+0000 7f02a57fa640 1 -- 172.21.15.42:0/3007253099 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641cee76720 con 0x5641cf1ebbd0 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.227+0000 7f02a57fa640 10 client.16731 handle_command_reply: tid=0 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.227+0000 7f02c6ba5740 1 client.16731 shutdown 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 msgr2=0x5641cf1d8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d8cb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f02ac000c00 tx=0x7f02ac023f90 comp rx=0 tx=0).stop 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 shutdown_connections 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf1ebbd0 0x5641cf1edfc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1db750 0x5641cf1d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d8cb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/3007253099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2250 0x5641cf1d8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02a57fa640 0 client.16731 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 >> 172.21.15.42:0/3007253099 conn(0x5641cf1cc980 msgr2=0x5641cf1ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 shutdown_connections 2024-03-12T04:19:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 1 -- 172.21.15.42:0/3007253099 wait complete. 2024-03-12T04:19:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.228+0000 7f02c6ba5740 20 client.16731 trim_cache size 0 max 16384 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.239+0000 7f02c6ba5740 1 Processor -- start 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.239+0000 7f02c6ba5740 1 -- start start 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 0x5641cf1daea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1db750 0x5641cf1d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf0bf130 con 0x5641cf1d2b70 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf1c85f0 con 0x5641cf1d2250 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf1cda00 con 0x5641cf1db750 2024-03-12T04:19:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57206/0 (socket says 172.21.15.42:57206) 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 -- 172.21.15.42:0/1031326044 learned_addr learned my addr 172.21.15.42:0/1031326044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 -- 172.21.15.42:0/1031326044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1db750 msgr2=0x5641cf1d59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 --2- 172.21.15.42:0/1031326044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1db750 0x5641cf1d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 -- 172.21.15.42:0/1031326044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 msgr2=0x5641cf1daea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 --2- 172.21.15.42:0/1031326044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 0x5641cf1daea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 -- 172.21.15.42:0/1031326044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641cf0d4c20 con 0x5641cf1d2b70 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02c156b640 1 --2- 172.21.15.42:0/1031326044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d9010 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f02ac096b40 tx=0x7f02ac0cd840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20adabf8515fdac server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02a77fe640 1 -- 172.21.15.42:0/1031326044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0b2ea0 con 0x5641cf1d2b70 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02a77fe640 1 -- 172.21.15.42:0/1031326044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02ac096410 con 0x5641cf1d2b70 2024-03-12T04:19:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.240+0000 7f02a77fe640 1 -- 172.21.15.42:0/1031326044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac01e120 con 0x5641cf1d2b70 2024-03-12T04:19:24.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 -- 172.21.15.42:0/1031326044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 msgr2=0x5641cf1d9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/1031326044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d9010 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f02ac096b40 tx=0x7f02ac0cd840 comp rx=0 tx=0).stop 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 -- 172.21.15.42:0/1031326044 shutdown_connections 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/1031326044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1db750 0x5641cf1d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/1031326044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1d2b70 0x5641cf1d9010 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/1031326044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 0x5641cf1daea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 -- 172.21.15.42:0/1031326044 >> 172.21.15.42:0/1031326044 conn(0x5641cf1cc980 msgr2=0x5641cf1d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 -- 172.21.15.42:0/1031326044 shutdown_connections 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 -- 172.21.15.42:0/1031326044 wait complete. 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 -- start start 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.241+0000 7f02c6ba5740 1 Processor -- start 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 0x5641cf1e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1e1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c6ba5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1db750 0x5641cf1e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf242290 con 0x5641cf1db750 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf242400 con 0x5641cf1d2250 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c6ba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641cf2425c0 con 0x5641cf1d2b70 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c6ba5740 10 client.0 _ll_register_callbacks cb 0x7f02b4038070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1db750 0x5641cf1e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1db750 0x5641cf1e1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57220/0 (socket says 172.21.15.42:57220) 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 -- 172.21.15.42:0/293126926 learned_addr learned my addr 172.21.15.42:0/293126926 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c156b640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 0x5641cf1e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 -- 172.21.15.42:0/293126926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 msgr2=0x5641cf1e1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1e1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 -- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 msgr2=0x5641cf1e12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 0x5641cf1e12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 -- 172.21.15.42:0/293126926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641cf1386a0 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c156b640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02c256d640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1db750 0x5641cf1e1d80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f02b4035c90 tx=0x7f02b4035cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1b62e6cf889145 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.242+0000 7f02a57fa640 1 -- 172.21.15.42:0/293126926 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4033990 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02a57fa640 1 -- 172.21.15.42:0/293126926 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f02b4003970 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02b402dcc0 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02a57fa640 1 -- 172.21.15.42:0/293126926 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b40333c0 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 10 client.16737 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5641cf0fa8c0 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02a57fa640 1 -- 172.21.15.42:0/293126926 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02b402dcc0 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02a57fa640 1 -- 172.21.15.42:0/293126926 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02b4004310 con 0x5641cf1db750 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 10 client.16737 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 20 client.16737 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf243d20 0x5641cf246140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 4 client.16737 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5641ceec5890 con 0x5641cf243d20 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.243+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf243d20 0x5641cf246140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02a57fa640 10 client.16737 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c1d6c640 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf243d20 0x5641cf246140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa4ae8802ea4c985 server_cookie=b0fc6eb1559f968b in_seq=0 out_seq=0 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02a57fa640 1 -- 172.21.15.42:0/293126926 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5641ceec5890 con 0x5641cf243d20 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02a57fa640 10 client.16737 handle_command_reply: tid=0 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 client.16737 shutdown 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1db750 msgr2=0x5641cf1e1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1db750 0x5641cf1e1d80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f02b4035c90 tx=0x7f02b4035cc0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 shutdown_connections 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5641cf243d20 0x5641cf246140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641cf1db750 0x5641cf1e1d80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641cf1d2b70 0x5641cf1e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02a57fa640 0 client.16737 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 --2- 172.21.15.42:0/293126926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641cf1d2250 0x5641cf1e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 >> 172.21.15.42:0/293126926 conn(0x5641cf1cc980 msgr2=0x5641cf1cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 shutdown_connections 2024-03-12T04:19:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 1 -- 172.21.15.42:0/293126926 wait complete. 2024-03-12T04:19:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.244+0000 7f02c6ba5740 20 client.16737 trim_cache size 0 max 16384 2024-03-12T04:19:24.252 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02a80a1bd0 msgr2=0x7f02a80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02a80a1bd0 0x7f02a80a4090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02b4037000 tx=0x7f02b4042000 comp rx=0 tx=0).stop 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 msgr2=0x7f02bc1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc1019e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d950 tx=0x7f02ac00de10 comp rx=0 tx=0).stop 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 shutdown_connections 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02a80a1bd0 0x7f02a80a4090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02bc1047b0 0x7f02bc101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02bc1003e0 0x7f02bc1019e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 --2- 172.21.15.42:0/1273368995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02bc0ffa40 0x7f02bc1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 >> 172.21.15.42:0/1273368995 conn(0x7f02bc0753c0 msgr2=0x7f02bc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 shutdown_connections 2024-03-12T04:19:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:24.245+0000 7f02c3fd0640 1 -- 172.21.15.42:0/1273368995 wait complete. 2024-03-12T04:19:24.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:24 smithi119 ceph-mon[36119]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:19:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:24 smithi136 ceph-mon[35571]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:19:25.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:24 smithi042 ceph-mon[32260]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:19:25.257 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:19:25.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:19:25.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:19:25.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 -- 172.21.15.42:0/1484187174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8101d30 msgr2=0x7fa2d8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 --2- 172.21.15.42:0/1484187174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8101d30 0x7fa2d8105d70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009930 tx=0x7fa2cc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:25.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 -- 172.21.15.42:0/1484187174 shutdown_connections 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 --2- 172.21.15.42:0/1484187174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2d8106450 0x7fa2d810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 --2- 172.21.15.42:0/1484187174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8101d30 0x7fa2d8105d70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 --2- 172.21.15.42:0/1484187174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8101380 0x7fa2d8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 -- 172.21.15.42:0/1484187174 >> 172.21.15.42:0/1484187174 conn(0x7fa2d8078580 msgr2=0x7fa2d8078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 -- 172.21.15.42:0/1484187174 shutdown_connections 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.375+0000 7fa2df840640 1 -- 172.21.15.42:0/1484187174 wait complete. 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 Processor -- start 2024-03-12T04:19:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 -- start start 2024-03-12T04:19:25.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2d8101380 0x7fa2d8073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8106450 0x7fa2d80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8073f00 0x7fa2d81a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81a5ec0 con 0x7fa2d8073f00 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81a6030 con 0x7fa2d8106450 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2df840640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81a61f0 con 0x7fa2d8101380 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2ddddd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8073f00 0x7fa2d81a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2ddddd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8073f00 0x7fa2d81a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57232/0 (socket says 172.21.15.42:57232) 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2ddddd640 1 -- 172.21.15.42:0/541064133 learned_addr learned my addr 172.21.15.42:0/541064133 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8106450 0x7fa2d80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.376+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2d8101380 0x7fa2d8073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2ddddd640 1 -- 172.21.15.42:0/541064133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2d8101380 msgr2=0x7fa2d8073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2ddddd640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2d8101380 0x7fa2d8073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2ddddd640 1 -- 172.21.15.42:0/541064133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8106450 msgr2=0x7fa2d80739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2ddddd640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8106450 0x7fa2d80739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2ddddd640 1 -- 172.21.15.42:0/541064133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc009560 con 0x7fa2d8073f00 2024-03-12T04:19:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2ddddd640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8073f00 0x7fa2d81a5950 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800eaa0 tx=0x7fa2c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2d67fc640 1 -- 172.21.15.42:0/541064133 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800cdb0 con 0x7fa2d8073f00 2024-03-12T04:19:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2d67fc640 1 -- 172.21.15.42:0/541064133 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2c8005020 con 0x7fa2d8073f00 2024-03-12T04:19:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d81a64d0 con 0x7fa2d8073f00 2024-03-12T04:19:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2d67fc640 1 -- 172.21.15.42:0/541064133 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80116b0 con 0x7fa2d8073f00 2024-03-12T04:19:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.377+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d81a6a00 con 0x7fa2d8073f00 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.379+0000 7fa2d67fc640 1 -- 172.21.15.42:0/541064133 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa2c8005190 con 0x7fa2d8073f00 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 Processor -- start 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 -- start start 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bb7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 0x55ba04bc02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04b25f90 con 0x55ba04bc08b0 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04b234c0 con 0x55ba04bb7380 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2e2412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04b1eca0 con 0x55ba04bb7d30 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2d67fc640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa2c00a1c20 0x7fa2c00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57236/0 (socket says 172.21.15.42:57236) 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3656494559 learned_addr learned my addr 172.21.15.42:0/3656494559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2ddddd640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa2c00a1c20 0x7fa2c00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 0x55ba04bc02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2d67fc640 1 -- 172.21.15.42:0/541064133 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa2c80cd850 con 0x7fa2d8073f00 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bb7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3656494559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 msgr2=0x55ba04bc02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.380+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 0x55ba04bc02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3656494559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 msgr2=0x55ba04bb7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bb7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3656494559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba04ae10b0 con 0x55ba04bc08b0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2ddddd640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa2c00a1c20 0x7fa2c00a40e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2c809a220 tx=0x7fa2c806c720 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc2d00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806aa20 tx=0x7fa2cc02c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3e477ffc9a8f0b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2bf7fe640 1 -- 172.21.15.42:0/3656494559 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc048070 con 0x55ba04bc08b0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2bf7fe640 1 -- 172.21.15.42:0/3656494559 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2cc02de90 con 0x55ba04bc08b0 2024-03-12T04:19:25.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.381+0000 7fa2bf7fe640 1 -- 172.21.15.42:0/3656494559 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003810 con 0x55ba04bc08b0 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 -- 172.21.15.42:0/3656494559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 msgr2=0x55ba04bc2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc2d00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806aa20 tx=0x7fa2cc02c610 comp rx=0 tx=0).stop 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 -- 172.21.15.42:0/3656494559 shutdown_connections 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc2d00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 0x55ba04bc02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3656494559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bb7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 -- 172.21.15.42:0/3656494559 >> 172.21.15.42:0/3656494559 conn(0x55ba04bb1840 msgr2=0x55ba04bb1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 -- 172.21.15.42:0/3656494559 shutdown_connections 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 -- 172.21.15.42:0/3656494559 wait complete. 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 -- start start 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 Processor -- start 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.382+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bb7380 0x55ba04bbd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bbddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bc08b0 0x55ba04bbe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04ac3f40 con 0x55ba04bb7380 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba047f24b0 con 0x55ba04bb7d30 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04aa4670 con 0x55ba04bc08b0 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 10 client.0 _ll_register_callbacks cb 0x55ba04b11f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bb7380 0x55ba04bbd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bb7380 0x55ba04bbd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57246/0 (socket says 172.21.15.42:57246) 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2246101234 learned_addr learned my addr 172.21.15.42:0/2246101234 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bc08b0 0x55ba04bbe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bbddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2246101234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bc08b0 msgr2=0x55ba04bbe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bc08b0 0x55ba04bbe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 msgr2=0x55ba04bbddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bbddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2246101234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d810e830 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bbddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bb7380 0x55ba04bbd880 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc036040 tx=0x7fa2cc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864a868574638862 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2bcff9640 1 -- 172.21.15.42:0/2246101234 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc048040 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2bcff9640 1 -- 172.21.15.42:0/2246101234 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa2cc000e40 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba04af8f40 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2bcff9640 1 -- 172.21.15.42:0/2246101234 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2cc005400 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 10 client.16749 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.383+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ba04b1d320 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2bcff9640 1 -- 172.21.15.42:0/2246101234 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ba04af8f40 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2bcff9640 1 -- 172.21.15.42:0/2246101234 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa2cc04e070 con 0x55ba04bb7380 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2e2412740 10 client.16749 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2e2412740 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2e2412740 20 client.16749 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04bd0bc0 0x55ba04bd2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2e2412740 4 client.16749 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba04b15310 con 0x55ba04bd0bc0 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.384+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04bd0bc0 0x55ba04bd2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04bd0bc0 0x55ba04bd2fe0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3518f2c61d7238ff server_cookie=19d594c742df7452 in_seq=0 out_seq=0 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2bcff9640 10 client.16749 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2bcff9640 1 -- 172.21.15.42:0/2246101234 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba04b15310 con 0x55ba04bd0bc0 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2bcff9640 10 client.16749 handle_command_reply: tid=0 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 client.16749 shutdown 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bb7380 msgr2=0x55ba04bbd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bb7380 0x55ba04bbd880 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc036040 tx=0x7fa2cc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 shutdown_connections 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04bd0bc0 0x55ba04bd2fe0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bc08b0 0x55ba04bbe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bbddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2246101234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bb7380 0x55ba04bbd880 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2bcff9640 0 client.16749 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 >> 172.21.15.42:0/2246101234 conn(0x55ba04bb1840 msgr2=0x55ba04bb37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 shutdown_connections 2024-03-12T04:19:25.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.385+0000 7fa2e2412740 1 -- 172.21.15.42:0/2246101234 wait complete. 2024-03-12T04:19:25.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.386+0000 7fa2e2412740 20 client.16749 trim_cache size 0 max 16384 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 Processor -- start 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 -- start start 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bbffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 0x55ba04bbc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bbc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04babda0 con 0x55ba04bc08b0 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04bb9490 con 0x55ba04bb7380 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2e2412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04bbcbd0 con 0x55ba04bb7d30 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bbc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bbc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57262/0 (socket says 172.21.15.42:57262) 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2240208773 learned_addr learned my addr 172.21.15.42:0/2240208773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bbffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2240208773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 msgr2=0x55ba04bbc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 0x55ba04bbc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2240208773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 msgr2=0x55ba04bbffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bbffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 -- 172.21.15.42:0/2240208773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba04ab9c00 con 0x55ba04bc08b0 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.401+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bbc690 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc034510 tx=0x7fa2cc043fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1674377ff629b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2beffd640 1 -- 172.21.15.42:0/2240208773 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc048040 con 0x55ba04bc08b0 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2beffd640 1 -- 172.21.15.42:0/2240208773 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2cc004a20 con 0x55ba04bc08b0 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2beffd640 1 -- 172.21.15.42:0/2240208773 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc034db0 con 0x55ba04bc08b0 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 -- 172.21.15.42:0/2240208773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 msgr2=0x55ba04bbc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bbc690 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc034510 tx=0x7fa2cc043fe0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 -- 172.21.15.42:0/2240208773 shutdown_connections 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bbc690 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7d30 0x55ba04bbc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 --2- 172.21.15.42:0/2240208773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7380 0x55ba04bbffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 -- 172.21.15.42:0/2240208773 >> 172.21.15.42:0/2240208773 conn(0x55ba04bb1840 msgr2=0x55ba04bd7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 -- 172.21.15.42:0/2240208773 shutdown_connections 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.402+0000 7fa2e2412740 1 -- 172.21.15.42:0/2240208773 wait complete. 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 -- start start 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 Processor -- start 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7380 0x55ba04bc7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bc5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04bc6920 con 0x55ba04bc08b0 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04bc6a90 con 0x55ba04bb7d30 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba04bc6c50 con 0x55ba04bb7380 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2e2412740 10 client.0 _ll_register_callbacks cb 0x7fa2cc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57266/0 (socket says 172.21.15.42:57266) 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3196195790 learned_addr learned my addr 172.21.15.42:0/3196195790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7380 0x55ba04bc7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bc5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3196195790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7380 msgr2=0x55ba04bc7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7380 0x55ba04bc7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 msgr2=0x55ba04bc5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bc5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.403+0000 7fa2dcddb640 1 -- 172.21.15.42:0/3196195790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba04b1d320 con 0x55ba04bc08b0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2dcddb640 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc63b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc02c170 tx=0x7fa2cc034590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2962801c5e084b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2bcff9640 1 -- 172.21.15.42:0/3196195790 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc048070 con 0x55ba04bc08b0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2bcff9640 1 -- 172.21.15.42:0/3196195790 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa2cc000e40 con 0x55ba04bc08b0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba04b44900 con 0x55ba04bc08b0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2bcff9640 1 -- 172.21.15.42:0/3196195790 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2cc034e20 con 0x55ba04bc08b0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 10 client.16755 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ba04adf910 con 0x55ba04bc08b0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2bcff9640 1 -- 172.21.15.42:0/3196195790 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ba04b44900 con 0x55ba04bc08b0 2024-03-12T04:19:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2bcff9640 1 -- 172.21.15.42:0/3196195790 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa2cc04e070 con 0x55ba04bc08b0 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 10 client.16755 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 20 client.16755 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04c281e0 0x55ba04c2a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 4 client.16755 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.404+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ba0481e860 con 0x55ba04c281e0 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.405+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04c281e0 0x55ba04c2a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.405+0000 7fa2bcff9640 10 client.16755 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.405+0000 7fa2dd5dc640 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04c281e0 0x55ba04c2a5d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7958a653f0bed4ff server_cookie=547ebf58c88f37ff in_seq=0 out_seq=0 2024-03-12T04:19:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2bcff9640 1 -- 172.21.15.42:0/3196195790 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ba0481e860 con 0x55ba04c281e0 2024-03-12T04:19:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2bcff9640 10 client.16755 handle_command_reply: tid=0 2024-03-12T04:19:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 client.16755 shutdown 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 msgr2=0x55ba04bc63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc63b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc02c170 tx=0x7fa2cc034590 comp rx=0 tx=0).stop 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 shutdown_connections 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ba04c281e0 0x55ba04c2a5d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba04bc08b0 0x55ba04bc63b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba04bb7d30 0x55ba04bc5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 --2- 172.21.15.42:0/3196195790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba04bb7380 0x55ba04bc7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 >> 172.21.15.42:0/3196195790 conn(0x55ba04bb1840 msgr2=0x55ba04bbc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.406+0000 7fa2bcff9640 0 client.16755 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 shutdown_connections 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2e2412740 1 -- 172.21.15.42:0/3196195790 wait complete. 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2e2412740 20 client.16755 trim_cache size 0 max 16384 2024-03-12T04:19:25.413 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:19:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:19:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "cce5fa24-7fef-4471-99d2-b955870ef2fd", 2024-03-12T04:19:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:19:25.414 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa2c00a1c20 msgr2=0x7fa2c00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa2c00a1c20 0x7fa2c00a40e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2c809a220 tx=0x7fa2c806c720 comp rx=0 tx=0).stop 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8073f00 msgr2=0x7fa2d81a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8073f00 0x7fa2d81a5950 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800eaa0 tx=0x7fa2c800ef60 comp rx=0 tx=0).stop 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 shutdown_connections 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa2c00a1c20 0x7fa2c00a40e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2d8073f00 0x7fa2d81a5950 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8106450 0x7fa2d80739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 --2- 172.21.15.42:0/541064133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2d8101380 0x7fa2d8073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 >> 172.21.15.42:0/541064133 conn(0x7fa2d8078580 msgr2=0x7fa2d80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 shutdown_connections 2024-03-12T04:19:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.407+0000 7fa2df840640 1 -- 172.21.15.42:0/541064133 wait complete. 2024-03-12T04:19:25.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:19:25.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:19:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.523+0000 7f4022464640 1 -- 172.21.15.42:0/956587569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c102cc0 msgr2=0x7f401c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.523+0000 7f4022464640 1 --2- 172.21.15.42:0/956587569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c102cc0 0x7f401c10b1a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4010009900 tx=0x7f401002bcd0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.523+0000 7f4022464640 1 -- 172.21.15.42:0/956587569 shutdown_connections 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.523+0000 7f4022464640 1 --2- 172.21.15.42:0/956587569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c10b6e0 0x7f401c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.523+0000 7f4022464640 1 --2- 172.21.15.42:0/956587569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c102cc0 0x7f401c10b1a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.523+0000 7f4022464640 1 --2- 172.21.15.42:0/956587569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f401c1023a0 0x7f401c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.523+0000 7f4022464640 1 -- 172.21.15.42:0/956587569 >> 172.21.15.42:0/956587569 conn(0x7f401c077230 msgr2=0x7f401c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 -- 172.21.15.42:0/956587569 shutdown_connections 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 -- 172.21.15.42:0/956587569 wait complete. 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 Processor -- start 2024-03-12T04:19:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 -- start start 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f401c1023a0 0x7f401c197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c102cc0 0x7f401c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c10b6e0 0x7f401c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c19c550 con 0x7f401c10b6e0 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c19c6c0 con 0x7f401c1023a0 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.524+0000 7f4022464640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c19c880 con 0x7f401c102cc0 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c10b6e0 0x7f401c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c10b6e0 0x7f401c19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57278/0 (socket says 172.21.15.42:57278) 2024-03-12T04:19:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 -- 172.21.15.42:0/1607126622 learned_addr learned my addr 172.21.15.42:0/1607126622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f401bfff640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f401c1023a0 0x7f401c197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f401b7fe640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c102cc0 0x7f401c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 -- 172.21.15.42:0/1607126622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c102cc0 msgr2=0x7f401c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c102cc0 0x7f401c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 -- 172.21.15.42:0/1607126622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f401c1023a0 msgr2=0x7f401c197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f401c1023a0 0x7f401c197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 -- 172.21.15.42:0/1607126622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4010009560 con 0x7f401c10b6e0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4020a01640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c10b6e0 0x7f401c19bf50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f401400cad0 tx=0x7f401400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f40197fa640 1 -- 172.21.15.42:0/1607126622 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401400ae20 con 0x7f401c10b6e0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f40197fa640 1 -- 172.21.15.42:0/1607126622 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4014005020 con 0x7f401c10b6e0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c19cb60 con 0x7f401c10b6e0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f40197fa640 1 -- 172.21.15.42:0/1607126622 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401400f690 con 0x7f401c10b6e0 2024-03-12T04:19:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.525+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c1a43e0 con 0x7f401c10b6e0 2024-03-12T04:19:25.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f40197fa640 1 -- 172.21.15.42:0/1607126622 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f4014005190 con 0x7f401c10b6e0 2024-03-12T04:19:25.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 Processor -- start 2024-03-12T04:19:25.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 -- start start 2024-03-12T04:19:25.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x560126586690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560126586bd0 0x56012658f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126591ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f40197fa640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40040a1ba0 0x7f40040a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601264f5280 con 0x5601265862b0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601264f27b0 con 0x560126586bd0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f402503e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601264edf90 con 0x56012658f7b0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x560126586690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.528+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x560126586690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57286/0 (socket says 172.21.15.42:57286) 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 -- 172.21.15.42:0/1769225726 learned_addr learned my addr 172.21.15.42:0/1769225726 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f4020a01640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40040a1ba0 0x7f40040a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401bfff640 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560126586bd0 0x56012658f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f40197fa640 1 -- 172.21.15.42:0/1607126622 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f40140cd7b0 con 0x7f401c10b6e0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126591ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 -- 172.21.15.42:0/1769225726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 msgr2=0x560126591ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126591ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 -- 172.21.15.42:0/1769225726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560126586bd0 msgr2=0x56012658f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560126586bd0 0x56012658f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 -- 172.21.15.42:0/1769225726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601264ac570 con 0x5601265862b0 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f4020a01640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40040a1ba0 0x7f40040a4060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f40140cb550 tx=0x7f4014005c50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f401b7fe640 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x560126586690 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4010005940 tx=0x7f40100058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4220b73bdb7159e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f40027fc640 1 -- 172.21.15.42:0/1769225726 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401002ca90 con 0x5601265862b0 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f40027fc640 1 -- 172.21.15.42:0/1769225726 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40100044e0 con 0x5601265862b0 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.529+0000 7f40027fc640 1 -- 172.21.15.42:0/1769225726 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010003e00 con 0x5601265862b0 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 -- 172.21.15.42:0/1769225726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 msgr2=0x560126586690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x560126586690 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4010005940 tx=0x7f40100058b0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 -- 172.21.15.42:0/1769225726 shutdown_connections 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126591ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560126586bd0 0x56012658f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 --2- 172.21.15.42:0/1769225726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x560126586690 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 -- 172.21.15.42:0/1769225726 >> 172.21.15.42:0/1769225726 conn(0x560126580990 msgr2=0x560126580d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 -- 172.21.15.42:0/1769225726 shutdown_connections 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 -- 172.21.15.42:0/1769225726 wait complete. 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 -- start start 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.530+0000 7f402503e740 1 Processor -- start 2024-03-12T04:19:25.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601265862b0 0x56012658f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x56012658c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 0x56012658cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601261c1430 con 0x560126586bd0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56012617de00 con 0x56012658f7b0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560126472fd0 con 0x5601265862b0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 10 client.0 _ll_register_callbacks cb 0x5601264c8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x56012658c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x56012658c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57298/0 (socket says 172.21.15.42:57298) 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 -- 172.21.15.42:0/325337469 learned_addr learned my addr 172.21.15.42:0/325337469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401b7fe640 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601265862b0 0x56012658f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 -- 172.21.15.42:0/325337469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601265862b0 msgr2=0x56012658f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601265862b0 0x56012658f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 -- 172.21.15.42:0/325337469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 msgr2=0x56012658cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 0x56012658cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 -- 172.21.15.42:0/325337469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c10e800 con 0x560126586bd0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f401bfff640 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x56012658c550 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f400c00bad0 tx=0x7f400c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4d0397ab1c3fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f3ffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/325337469 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c016070 con 0x560126586bd0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/325337469 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f400c005020 con 0x560126586bd0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.531+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601261ed7e0 con 0x560126586bd0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/325337469 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f400c003e80 con 0x560126586bd0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 10 client.16767 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5601264ec610 con 0x560126586bd0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/325337469 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5601261ed7e0 con 0x560126586bd0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/325337469 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f400c01b470 con 0x560126586bd0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 10 client.16767 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 20 client.16767 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56012659f9e0 0x5601265a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 4 client.16767 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56012622a720 con 0x56012659f9e0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.532+0000 7f401b7fe640 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56012659f9e0 0x5601265a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.533+0000 7f401b7fe640 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56012659f9e0 0x5601265a1e00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e459f5478a883989 server_cookie=bc96608176dcf891 in_seq=0 out_seq=0 2024-03-12T04:19:25.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.533+0000 7f3ffb7fe640 10 client.16767 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.533+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/325337469 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56012622a720 con 0x56012659f9e0 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.533+0000 7f3ffb7fe640 10 client.16767 handle_command_reply: tid=0 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.533+0000 7f402503e740 1 client.16767 shutdown 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 msgr2=0x56012658c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x56012658c550 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f400c00bad0 tx=0x7f400c00bed0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 shutdown_connections 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56012659f9e0 0x5601265a1e00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 0x56012658cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x56012658c550 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f3ffb7fe640 0 client.16767 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 --2- 172.21.15.42:0/325337469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601265862b0 0x56012658f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 >> 172.21.15.42:0/325337469 conn(0x560126580990 msgr2=0x560126582730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 shutdown_connections 2024-03-12T04:19:25.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 1 -- 172.21.15.42:0/325337469 wait complete. 2024-03-12T04:19:25.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.534+0000 7f402503e740 20 client.16767 trim_cache size 0 max 16384 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 Processor -- start 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 -- start start 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x56012658ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560126586bd0 0x56012658f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 0x56012658bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56012659bde0 con 0x5601265862b0 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560126580240 con 0x56012658f7b0 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f402503e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560126585cf0 con 0x560126586bd0 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x56012658ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x56012658ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57302/0 (socket says 172.21.15.42:57302) 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 -- 172.21.15.42:0/2856703687 learned_addr learned my addr 172.21.15.42:0/2856703687 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 -- 172.21.15.42:0/2856703687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560126586bd0 msgr2=0x56012658f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 --2- 172.21.15.42:0/2856703687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560126586bd0 0x56012658f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 -- 172.21.15.42:0/2856703687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 msgr2=0x56012658bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 --2- 172.21.15.42:0/2856703687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 0x56012658bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 -- 172.21.15.42:0/2856703687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560126488ac0 con 0x5601265862b0 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.549+0000 7f401b7fe640 1 --2- 172.21.15.42:0/2856703687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x56012658ec80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4010036040 tx=0x7f401002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef91e51ee892850 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f4001ffb640 1 -- 172.21.15.42:0/2856703687 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010054070 con 0x5601265862b0 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f4001ffb640 1 -- 172.21.15.42:0/2856703687 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4010005d50 con 0x5601265862b0 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f4001ffb640 1 -- 172.21.15.42:0/2856703687 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010057de0 con 0x5601265862b0 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 -- 172.21.15.42:0/2856703687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 msgr2=0x56012658ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 --2- 172.21.15.42:0/2856703687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x56012658ec80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4010036040 tx=0x7f401002bcd0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 -- 172.21.15.42:0/2856703687 shutdown_connections 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 --2- 172.21.15.42:0/2856703687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56012658f7b0 0x56012658bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 --2- 172.21.15.42:0/2856703687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560126586bd0 0x56012658f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 --2- 172.21.15.42:0/2856703687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601265862b0 0x56012658ec80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 -- 172.21.15.42:0/2856703687 >> 172.21.15.42:0/2856703687 conn(0x560126580990 msgr2=0x56012658ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 -- 172.21.15.42:0/2856703687 shutdown_connections 2024-03-12T04:19:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 -- 172.21.15.42:0/2856703687 wait complete. 2024-03-12T04:19:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.550+0000 7f402503e740 1 -- start start 2024-03-12T04:19:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 1 Processor -- start 2024-03-12T04:19:25.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601265862b0 0x560126593250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x560126593790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126593cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601265f2030 con 0x560126586bd0 2024-03-12T04:19:25.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601265fead0 con 0x5601265862b0 2024-03-12T04:19:25.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601265fec10 con 0x56012658f7b0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f402503e740 10 client.0 _ll_register_callbacks cb 0x7f4010009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x560126593790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x560126593790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57316/0 (socket says 172.21.15.42:57316) 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 -- 172.21.15.42:0/591247636 learned_addr learned my addr 172.21.15.42:0/591247636 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401b7fe640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601265862b0 0x560126593250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401b7fe640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126593cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 -- 172.21.15.42:0/591247636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 msgr2=0x560126593cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126593cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 -- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601265862b0 msgr2=0x560126593250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601265862b0 0x560126593250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401bfff640 1 -- 172.21.15.42:0/591247636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601264ac570 con 0x560126586bd0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.551+0000 7f401b7fe640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601265862b0 0x560126593250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f401bfff640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x560126593790 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x56012658bdb0 tx=0x7f400c005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8b8f7320c05244 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f3ffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/591247636 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c016040 con 0x560126586bd0 2024-03-12T04:19:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/591247636 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f400c029030 con 0x560126586bd0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40100044e0 con 0x560126586bd0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/591247636 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f400c011440 con 0x560126586bd0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 10 client.16773 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5601264ae6a0 con 0x560126586bd0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/591247636 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40100044e0 con 0x560126586bd0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/591247636 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f400c037550 con 0x560126586bd0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 10 client.16773 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 20 client.16773 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601266002c0 0x5601266026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 4 client.16773 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.552+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560126279890 con 0x5601266002c0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f401b7fe640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601266002c0 0x5601266026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f401b7fe640 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601266002c0 0x5601266026b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c99e0b5629fa17 server_cookie=5763157f76a5d597 in_seq=0 out_seq=0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f3ffb7fe640 10 client.16773 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f3ffb7fe640 1 -- 172.21.15.42:0/591247636 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560126279890 con 0x5601266002c0 2024-03-12T04:19:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f3ffb7fe640 10 client.16773 handle_command_reply: tid=0 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f402503e740 1 client.16773 shutdown 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 msgr2=0x560126593790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f402503e740 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x560126593790 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x56012658bdb0 tx=0x7f400c005690 comp rx=0 tx=0).stop 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 shutdown_connections 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.553+0000 7f402503e740 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5601266002c0 0x5601266026b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f402503e740 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56012658f7b0 0x560126593cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f402503e740 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560126586bd0 0x560126593790 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f3ffb7fe640 0 client.16773 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f402503e740 1 --2- 172.21.15.42:0/591247636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601265862b0 0x560126593250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 >> 172.21.15.42:0/591247636 conn(0x560126580990 msgr2=0x56012658ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 shutdown_connections 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f402503e740 1 -- 172.21.15.42:0/591247636 wait complete. 2024-03-12T04:19:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f402503e740 20 client.16773 trim_cache size 0 max 16384 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cce5fa24-7fef-4471-99d2-b955870ef2fd": { 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "cce5fa24-7fef-4471-99d2-b955870ef2fd", 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:19:25.561 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40040a1ba0 msgr2=0x7f40040a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40040a1ba0 0x7f40040a4060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f40140cb550 tx=0x7f4014005c50 comp rx=0 tx=0).stop 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c10b6e0 msgr2=0x7f401c19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c10b6e0 0x7f401c19bf50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f401400cad0 tx=0x7f401400cf90 comp rx=0 tx=0).stop 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 shutdown_connections 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40040a1ba0 0x7f40040a4060 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f401c10b6e0 0x7f401c19bf50 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c102cc0 0x7f401c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 --2- 172.21.15.42:0/1607126622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f401c1023a0 0x7f401c197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 >> 172.21.15.42:0/1607126622 conn(0x7f401c077230 msgr2=0x7f401c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:25.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 shutdown_connections 2024-03-12T04:19:25.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:25.554+0000 7f4022464640 1 -- 172.21.15.42:0/1607126622 wait complete. 2024-03-12T04:19:25.567 INFO:tasks.cephfs.filesystem:scrub status for tag:cce5fa24-7fef-4471-99d2-b955870ef2fd - {'path': '/', 'tag': 'cce5fa24-7fef-4471-99d2-b955870ef2fd', 'options': 'recursive,force'} 2024-03-12T04:19:26.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:26 smithi119 ceph-mon[36119]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:26.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:26 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:19:26.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:26 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:26.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:26 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:19:26.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:26 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:26.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:26 smithi136 ceph-mon[35571]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:26.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:26 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:19:26.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:26 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:26.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:26 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:19:26.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:26 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:26 smithi042 ceph-mon[32260]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:26 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:19:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:26 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:26 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:19:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:26 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:28.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:28 smithi119 ceph-mon[36119]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:28.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:28 smithi136 ceph-mon[35571]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:29.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:28 smithi042 ceph-mon[32260]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:29.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:29 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:19:29.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:29 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:19:30.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:29 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:19:30.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:30 smithi119 ceph-mon[36119]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:30.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:30 smithi136 ceph-mon[35571]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:30 smithi042 ceph-mon[32260]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:32.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:32 smithi119 ceph-mon[36119]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:32.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:32 smithi136 ceph-mon[35571]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:33.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:32 smithi042 ceph-mon[32260]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:34.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:34 smithi119 ceph-mon[36119]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:34.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:34 smithi136 ceph-mon[35571]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:34 smithi042 ceph-mon[32260]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:36.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:36 smithi119 ceph-mon[36119]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:36.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:36 smithi136 ceph-mon[35571]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:36 smithi042 ceph-mon[32260]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:19:38.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:38 smithi119 ceph-mon[36119]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:38.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:38 smithi136 ceph-mon[35571]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:38 smithi042 ceph-mon[32260]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:19:40.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:40 smithi119 ceph-mon[36119]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:40.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:40 smithi136 ceph-mon[35571]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:41.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:40 smithi042 ceph-mon[32260]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:42.868 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:42 smithi042 ceph-mon[32260]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:42.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:42 smithi119 ceph-mon[36119]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:42.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:42 smithi136 ceph-mon[35571]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:44.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:44 smithi119 ceph-mon[36119]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:44.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:44 smithi136 ceph-mon[35571]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:44 smithi042 ceph-mon[32260]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:46.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:46 smithi119 ceph-mon[36119]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:46.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:46 smithi136 ceph-mon[35571]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:47.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:46 smithi042 ceph-mon[32260]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:48.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:48 smithi119 ceph-mon[36119]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:48.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:48 smithi136 ceph-mon[35571]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:49.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:48 smithi042 ceph-mon[32260]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:50.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:50 smithi119 ceph-mon[36119]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:50.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:50 smithi136 ceph-mon[35571]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:51.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:50 smithi042 ceph-mon[32260]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:52.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:52 smithi119 ceph-mon[36119]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:52.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:52 smithi136 ceph-mon[35571]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:53.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:52 smithi042 ceph-mon[32260]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:54.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:54 smithi119 ceph-mon[36119]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:54.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:54 smithi136 ceph-mon[35571]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:55.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:54 smithi042 ceph-mon[32260]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:55.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:19:55.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:19:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 -- 172.21.15.42:0/3307930044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8100270 msgr2=0x7f29e8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3307930044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8100270 0x7f29e8100650 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009930 tx=0x7f29e402bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 -- 172.21.15.42:0/3307930044 shutdown_connections 2024-03-12T04:19:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3307930044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e8105340 0x7f29e810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3307930044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e8100c20 0x7f29e8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3307930044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8100270 0x7f29e8100650 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 -- 172.21.15.42:0/3307930044 >> 172.21.15.42:0/3307930044 conn(0x7f29e8077230 msgr2=0x7f29e8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 -- 172.21.15.42:0/3307930044 shutdown_connections 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.684+0000 7f29ef89d640 1 -- 172.21.15.42:0/3307930044 wait complete. 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 Processor -- start 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 -- start start 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e8100270 0x7f29e8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8105340 0x7f29e8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e819c010 0x7f29e819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e819ca00 con 0x7f29e8105340 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e819cb70 con 0x7f29e8100270 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ef89d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e819cce0 con 0x7f29e819c010 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8105340 0x7f29e8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8105340 0x7f29e8197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41082/0 (socket says 172.21.15.42:41082) 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 -- 172.21.15.42:0/3544223903 learned_addr learned my addr 172.21.15.42:0/3544223903 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ede3a640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e819c010 0x7f29e819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 -- 172.21.15.42:0/3544223903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e819c010 msgr2=0x7f29e819c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ed639640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e8100270 0x7f29e8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e819c010 0x7f29e819c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 -- 172.21.15.42:0/3544223903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e8100270 msgr2=0x7f29e8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e8100270 0x7f29e8197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ece38640 1 -- 172.21.15.42:0/3544223903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4009560 con 0x7f29e8105340 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.685+0000 7f29ede3a640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e819c010 0x7f29e819c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.686+0000 7f29ece38640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8105340 0x7f29e8197c90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f29d800d740 tx=0x7f29d800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.686+0000 7f29de7fc640 1 -- 172.21.15.42:0/3544223903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8009c30 con 0x7f29e8105340 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.686+0000 7f29de7fc640 1 -- 172.21.15.42:0/3544223903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29d800de50 con 0x7f29e8105340 2024-03-12T04:19:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.686+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e81a42c0 con 0x7f29e8105340 2024-03-12T04:19:55.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.686+0000 7f29de7fc640 1 -- 172.21.15.42:0/3544223903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8004e40 con 0x7f29e8105340 2024-03-12T04:19:55.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.686+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e81a4720 con 0x7f29e8105340 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.688+0000 7f29de7fc640 1 -- 172.21.15.42:0/3544223903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f29d8004fa0 con 0x7f29e8105340 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 Processor -- start 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 -- start start 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bfea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bf59280 con 0x55f74bfeabd0 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bf567b0 con 0x55f74bfea2b0 2024-03-12T04:19:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29f2477740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bf51f90 con 0x55f74bff37b0 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29de7fc640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d40a1ba0 0x7f29d40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41096/0 (socket says 172.21.15.42:41096) 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 -- 172.21.15.42:0/3641370019 learned_addr learned my addr 172.21.15.42:0/3641370019 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ece38640 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ede3a640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d40a1ba0 0x7f29d40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ede3a640 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bfea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29de7fc640 1 -- 172.21.15.42:0/3544223903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f29d80cd290 con 0x7f29e8105340 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 -- 172.21.15.42:0/3641370019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 msgr2=0x55f74bff5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 -- 172.21.15.42:0/3641370019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 msgr2=0x55f74bfea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bfea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.689+0000 7f29ed639640 1 -- 172.21.15.42:0/3641370019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f74bf10570 con 0x55f74bfeabd0 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.690+0000 7f29ed639640 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3270 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f29e4005e70 tx=0x7f29e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3059cbffed94a81 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.690+0000 7f29ede3a640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d40a1ba0 0x7f29d40a4060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005c40 tx=0x7f29e0009530 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.690+0000 7f29d37fe640 1 -- 172.21.15.42:0/3641370019 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4004ae0 con 0x55f74bfeabd0 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.690+0000 7f29d37fe640 1 -- 172.21.15.42:0/3641370019 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29e4004030 con 0x55f74bfeabd0 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.690+0000 7f29d37fe640 1 -- 172.21.15.42:0/3641370019 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4003bb0 con 0x55f74bfeabd0 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 -- 172.21.15.42:0/3641370019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 msgr2=0x55f74bff3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3270 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f29e4005e70 tx=0x7f29e40046c0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 -- 172.21.15.42:0/3641370019 shutdown_connections 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3270 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 --2- 172.21.15.42:0/3641370019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bfea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 -- 172.21.15.42:0/3641370019 >> 172.21.15.42:0/3641370019 conn(0x55f74bfe4990 msgr2=0x55f74bfe4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 -- 172.21.15.42:0/3641370019 shutdown_connections 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 -- 172.21.15.42:0/3641370019 wait complete. 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 -- start start 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 Processor -- start 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.691+0000 7f29f2477740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29f2477740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29f2477740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29f2477740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bc25430 con 0x55f74bfeabd0 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29f2477740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bbe1e00 con 0x55f74bfea2b0 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29f2477740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bed6fd0 con 0x55f74bff37b0 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29f2477740 10 client.0 _ll_register_callbacks cb 0x55f74bf2cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41110/0 (socket says 172.21.15.42:41110) 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 -- 172.21.15.42:0/367248266 learned_addr learned my addr 172.21.15.42:0/367248266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ede3a640 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ed639640 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 -- 172.21.15.42:0/367248266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 msgr2=0x55f74bff0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 -- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 msgr2=0x55f74bff3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 -- 172.21.15.42:0/367248266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e80682e0 con 0x55f74bfeabd0 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29ece38640 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff04d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f29d8099e40 tx=0x7f29d8069290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5037a838ab2ce2b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29d0ff9640 1 -- 172.21.15.42:0/367248266 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d809e000 con 0x55f74bfeabd0 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29d0ff9640 1 -- 172.21.15.42:0/367248266 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f29d8096520 con 0x55f74bfeabd0 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f74bc517e0 con 0x55f74bfeabd0 2024-03-12T04:19:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.692+0000 7f29d0ff9640 1 -- 172.21.15.42:0/367248266 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d805d050 con 0x55f74bfeabd0 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 10 client.16785 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f74bf50610 con 0x55f74bfeabd0 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29d0ff9640 1 -- 172.21.15.42:0/367248266 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f74bc517e0 con 0x55f74bfeabd0 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29d0ff9640 1 -- 172.21.15.42:0/367248266 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29d80cac50 con 0x55f74bfeabd0 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 10 client.16785 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 20 client.16785 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c0037b0 0x55f74c005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 4 client.16785 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f74bc8e720 con 0x55f74c0037b0 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.693+0000 7f29ed639640 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c0037b0 0x55f74c005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29d0ff9640 10 client.16785 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29ed639640 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c0037b0 0x55f74c005bd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf0f94fad40394 server_cookie=570a728e81a56892 in_seq=0 out_seq=0 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29d0ff9640 1 -- 172.21.15.42:0/367248266 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f74bc8e720 con 0x55f74c0037b0 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29d0ff9640 10 client.16785 handle_command_reply: tid=0 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29f2477740 1 client.16785 shutdown 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 msgr2=0x55f74bff04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29f2477740 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff04d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f29d8099e40 tx=0x7f29d8069290 comp rx=0 tx=0).stop 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 shutdown_connections 2024-03-12T04:19:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.694+0000 7f29f2477740 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c0037b0 0x55f74c005bd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29f2477740 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bff0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29f2477740 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff04d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29d0ff9640 0 client.16785 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29f2477740 1 --2- 172.21.15.42:0/367248266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 >> 172.21.15.42:0/367248266 conn(0x55f74bfe4990 msgr2=0x55f74bfe6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 shutdown_connections 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29f2477740 1 -- 172.21.15.42:0/367248266 wait complete. 2024-03-12T04:19:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.695+0000 7f29f2477740 20 client.16785 trim_cache size 0 max 16384 2024-03-12T04:19:55.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 Processor -- start 2024-03-12T04:19:55.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 -- start start 2024-03-12T04:19:55.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bfefd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bfe0120 con 0x55f74bfeabd0 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bfec160 con 0x55f74bfea2b0 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29f2477740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74bfe0320 con 0x55f74bff37b0 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41116/0 (socket says 172.21.15.42:41116) 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 -- 172.21.15.42:0/3914573506 learned_addr learned my addr 172.21.15.42:0/3914573506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 -- 172.21.15.42:0/3914573506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 msgr2=0x55f74bfefd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bfefd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ed639640 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 -- 172.21.15.42:0/3914573506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 msgr2=0x55f74bff2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.710+0000 7f29ece38640 1 -- 172.21.15.42:0/3914573506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f74beecac0 con 0x55f74bfeabd0 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29ece38640 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3140 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f29d801d040 tx=0x7f29d80999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1530c00e125fca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29d2ffd640 1 -- 172.21.15.42:0/3914573506 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d805d070 con 0x55f74bfeabd0 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29d2ffd640 1 -- 172.21.15.42:0/3914573506 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29d80055d0 con 0x55f74bfeabd0 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29d2ffd640 1 -- 172.21.15.42:0/3914573506 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80966c0 con 0x55f74bfeabd0 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 -- 172.21.15.42:0/3914573506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 msgr2=0x55f74bff3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3140 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f29d801d040 tx=0x7f29d80999f0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 -- 172.21.15.42:0/3914573506 shutdown_connections 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bff37b0 0x55f74bfefd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bfeabd0 0x55f74bff3140 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 --2- 172.21.15.42:0/3914573506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 -- 172.21.15.42:0/3914573506 >> 172.21.15.42:0/3914573506 conn(0x55f74bfe4990 msgr2=0x55f74c00a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.711+0000 7f29f2477740 1 -- 172.21.15.42:0/3914573506 shutdown_connections 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 -- 172.21.15.42:0/3914573506 wait complete. 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 -- start start 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 Processor -- start 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bfeabd0 0x55f74bff77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bff37b0 0x55f74bff7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74c059df0 con 0x55f74bff37b0 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74c059f60 con 0x55f74bfea2b0 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74c05a120 con 0x55f74bfeabd0 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29f2477740 10 client.0 _ll_register_callbacks cb 0x7f29e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29ede3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bff37b0 0x55f74bff7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29ede3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bff37b0 0x55f74bff7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41126/0 (socket says 172.21.15.42:41126) 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29ede3a640 1 -- 172.21.15.42:0/2244727695 learned_addr learned my addr 172.21.15.42:0/2244727695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.712+0000 7f29ed639640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ece38640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bfeabd0 0x55f74bff77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ede3a640 1 -- 172.21.15.42:0/2244727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bfeabd0 msgr2=0x55f74bff77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ede3a640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bfeabd0 0x55f74bff77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ede3a640 1 -- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 msgr2=0x55f74bff72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ede3a640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ede3a640 1 -- 172.21.15.42:0/2244727695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f74bf50610 con 0x55f74bff37b0 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ed639640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29ede3a640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bff37b0 0x55f74bff7d20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f29e0010b10 tx=0x7f29e0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7f054b3a70b07a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29d0ff9640 1 -- 172.21.15.42:0/2244727695 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0023070 con 0x55f74bff37b0 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29d0ff9640 1 -- 172.21.15.42:0/2244727695 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f29e001db80 con 0x55f74bff37b0 2024-03-12T04:19:55.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29e4004030 con 0x55f74bff37b0 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29d0ff9640 1 -- 172.21.15.42:0/2244727695 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e001dd40 con 0x55f74bff37b0 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29f2477740 10 client.16791 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f74bf126a0 con 0x55f74bff37b0 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.713+0000 7f29d0ff9640 1 -- 172.21.15.42:0/2244727695 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29e0035080 con 0x55f74bff37b0 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29d0ff9640 1 -- 172.21.15.42:0/2244727695 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29e000c070 con 0x55f74bff37b0 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 10 client.16791 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 20 client.16791 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c05b9f0 0x55f74c05de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 4 client.16791 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:19:55.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f74bcdd890 con 0x55f74c05b9f0 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29ed639640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c05b9f0 0x55f74c05de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29d0ff9640 10 client.16791 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29ed639640 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c05b9f0 0x55f74c05de10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ddfc896267cdd3 server_cookie=e3e46edbc4c619d8 in_seq=0 out_seq=0 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29d0ff9640 1 -- 172.21.15.42:0/2244727695 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f74bcdd890 con 0x55f74c05b9f0 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29d0ff9640 10 client.16791 handle_command_reply: tid=0 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 1 client.16791 shutdown 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.714+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bff37b0 msgr2=0x55f74bff7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bff37b0 0x55f74bff7d20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f29e0010b10 tx=0x7f29e0009d50 comp rx=0 tx=0).stop 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 shutdown_connections 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f74c05b9f0 0x55f74c05de10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f74bff37b0 0x55f74bff7d20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74bfeabd0 0x55f74bff77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29d0ff9640 0 client.16791 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:55.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 --2- 172.21.15.42:0/2244727695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74bfea2b0 0x55f74bff72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 >> 172.21.15.42:0/2244727695 conn(0x55f74bfe4990 msgr2=0x55f74bfefd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:55.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 shutdown_connections 2024-03-12T04:19:55.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 1 -- 172.21.15.42:0/2244727695 wait complete. 2024-03-12T04:19:55.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29f2477740 20 client.16791 trim_cache size 0 max 16384 2024-03-12T04:19:55.728 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:19:55.729 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:19:55.729 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:19:55.729 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:19:55.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d40a1ba0 msgr2=0x7f29d40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d40a1ba0 0x7f29d40a4060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005c40 tx=0x7f29e0009530 comp rx=0 tx=0).stop 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8105340 msgr2=0x7f29e8197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8105340 0x7f29e8197c90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f29d800d740 tx=0x7f29d800dc00 comp rx=0 tx=0).stop 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 shutdown_connections 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29d40a1ba0 0x7f29d40a4060 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e819c010 0x7f29e819c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29e8105340 0x7f29e8197c90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 --2- 172.21.15.42:0/3544223903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29e8100270 0x7f29e8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 >> 172.21.15.42:0/3544223903 conn(0x7f29e8077230 msgr2=0x7f29e80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 shutdown_connections 2024-03-12T04:19:55.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.715+0000 7f29ef89d640 1 -- 172.21.15.42:0/3544223903 wait complete. 2024-03-12T04:19:55.732 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:19:55.732 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:19:55.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.832+0000 7fe02bfe2640 1 -- 172.21.15.42:0/494708991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024101d40 msgr2=0x7fe024105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.832+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/494708991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024101d40 0x7fe024105d80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe018009930 tx=0x7fe01802bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:55.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- 172.21.15.42:0/494708991 shutdown_connections 2024-03-12T04:19:55.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/494708991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe024106460 0x7fe02410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/494708991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024101d40 0x7fe024105d80 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/494708991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe024101390 0x7fe024101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- 172.21.15.42:0/494708991 >> 172.21.15.42:0/494708991 conn(0x7fe0240fcc10 msgr2=0x7fe0240ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- 172.21.15.42:0/494708991 shutdown_connections 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- 172.21.15.42:0/494708991 wait complete. 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 Processor -- start 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- start start 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe024101390 0x7fe024197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024106460 0x7fe024197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe02419bfd0 0x7fe02419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02419c9c0 con 0x7fe024106460 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02419cb30 con 0x7fe02419bfd0 2024-03-12T04:19:55.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.833+0000 7fe02bfe2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02419cca0 con 0x7fe024101390 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024106460 0x7fe024197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024106460 0x7fe024197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41146/0 (socket says 172.21.15.42:41146) 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 -- 172.21.15.42:0/623067393 learned_addr learned my addr 172.21.15.42:0/623067393 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe029d7e640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe024101390 0x7fe024197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02a57f640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe02419bfd0 0x7fe02419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 -- 172.21.15.42:0/623067393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe024101390 msgr2=0x7fe024197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe024101390 0x7fe024197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 -- 172.21.15.42:0/623067393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe02419bfd0 msgr2=0x7fe02419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe02419bfd0 0x7fe02419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 -- 172.21.15.42:0/623067393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018009560 con 0x7fe024106460 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02a57f640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe02419bfd0 0x7fe02419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02957d640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024106460 0x7fe024197c50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe018009230 tx=0x7fe018005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe022ffd640 1 -- 172.21.15.42:0/623067393 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180043d0 con 0x7fe024106460 2024-03-12T04:19:55.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe022ffd640 1 -- 172.21.15.42:0/623067393 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe01802de90 con 0x7fe024106460 2024-03-12T04:19:55.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0241a4280 con 0x7fe024106460 2024-03-12T04:19:55.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe022ffd640 1 -- 172.21.15.42:0/623067393 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180052b0 con 0x7fe024106460 2024-03-12T04:19:55.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.834+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0241a4730 con 0x7fe024106460 2024-03-12T04:19:55.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.836+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe024067f70 con 0x7fe024106460 2024-03-12T04:19:55.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.836+0000 7fe022ffd640 1 -- 172.21.15.42:0/623067393 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe018003cc0 con 0x7fe024106460 2024-03-12T04:19:55.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.837+0000 7fe022ffd640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe00c0a1c20 0x7fe00c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:55.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.837+0000 7fe022ffd640 1 -- 172.21.15.42:0/623067393 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe0180ef6b0 con 0x7fe024106460 2024-03-12T04:19:55.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.837+0000 7fe029d7e640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe00c0a1c20 0x7fe00c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:55.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.838+0000 7fe029d7e640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe00c0a1c20 0x7fe00c0a40e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009b60 tx=0x7fe01c005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:55.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:55.842+0000 7fe022ffd640 1 -- 172.21.15.42:0/623067393 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0180b86e0 con 0x7fe024106460 2024-03-12T04:19:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.061+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe024101d40 con 0x7fe024106460 2024-03-12T04:19:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.062+0000 7fe022ffd640 1 -- 172.21.15.42:0/623067393 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fe018005410 con 0x7fe024106460 2024-03-12T04:19:56.064 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:19:56.065 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:19:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:19:56.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe00c0a1c20 msgr2=0x7fe00c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe00c0a1c20 0x7fe00c0a40e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009b60 tx=0x7fe01c005f90 comp rx=0 tx=0).stop 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024106460 msgr2=0x7fe024197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024106460 0x7fe024197c50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe018009230 tx=0x7fe018005e20 comp rx=0 tx=0).stop 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 shutdown_connections 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe00c0a1c20 0x7fe00c0a40e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe02419bfd0 0x7fe02419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe024106460 0x7fe024197c50 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 --2- 172.21.15.42:0/623067393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe024101390 0x7fe024197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 >> 172.21.15.42:0/623067393 conn(0x7fe0240fcc10 msgr2=0x7fe0240fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 shutdown_connections 2024-03-12T04:19:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.064+0000 7fe02bfe2640 1 -- 172.21.15.42:0/623067393 wait complete. 2024-03-12T04:19:56.077 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:56.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:19:56.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 -- 172.21.15.42:0/791650368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b41002a0 msgr2=0x7f98b4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/791650368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b41002a0 0x7f98b4105dd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009930 tx=0x7f98a802bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:56.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 -- 172.21.15.42:0/791650368 shutdown_connections 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/791650368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b4106500 0x7f98b410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/791650368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b41002a0 0x7f98b4105dd0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/791650368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b40ff8f0 0x7f98b40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 -- 172.21.15.42:0/791650368 >> 172.21.15.42:0/791650368 conn(0x7f98b4076640 msgr2=0x7f98b40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 -- 172.21.15.42:0/791650368 shutdown_connections 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.179+0000 7f98ba3bc640 1 -- 172.21.15.42:0/791650368 wait complete. 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 Processor -- start 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 -- start start 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b40ff8f0 0x7f98b41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b4106500 0x7f98b4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b419bfa0 0x7f98b419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b419c990 con 0x7f98b40ff8f0 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b419cb00 con 0x7f98b419bfa0 2024-03-12T04:19:56.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98ba3bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b419cc70 con 0x7f98b4106500 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b40ff8f0 0x7f98b41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b40ff8f0 0x7f98b41976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41156/0 (socket says 172.21.15.42:41156) 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.180+0000 7f98b3fff640 1 -- 172.21.15.42:0/3436897291 learned_addr learned my addr 172.21.15.42:0/3436897291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b8959640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b419bfa0 0x7f98b419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b37fe640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b4106500 0x7f98b4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b3fff640 1 -- 172.21.15.42:0/3436897291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b4106500 msgr2=0x7f98b4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b3fff640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b4106500 0x7f98b4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b3fff640 1 -- 172.21.15.42:0/3436897291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b419bfa0 msgr2=0x7f98b419c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b3fff640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b419bfa0 0x7f98b419c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b3fff640 1 -- 172.21.15.42:0/3436897291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8009560 con 0x7f98b40ff8f0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b3fff640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b40ff8f0 0x7f98b41976e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f98a400d950 tx=0x7f98a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b17fa640 1 -- 172.21.15.42:0/3436897291 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4009c30 con 0x7f98b40ff8f0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b41a4250 con 0x7f98b40ff8f0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b17fa640 1 -- 172.21.15.42:0/3436897291 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98a4009d90 con 0x7f98b40ff8f0 2024-03-12T04:19:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98b17fa640 1 -- 172.21.15.42:0/3436897291 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400f6b0 con 0x7f98b40ff8f0 2024-03-12T04:19:56.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.181+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b41a4720 con 0x7f98b40ff8f0 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.183+0000 7f98b17fa640 1 -- 172.21.15.42:0/3436897291 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f98a4010aa0 con 0x7f98b40ff8f0 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 Processor -- start 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 -- start start 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b17fa640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f989c0a1ba0 0x7f989c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437850630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437859210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x55843785bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843780b960 con 0x558437859750 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584377bf310 con 0x558437850b70 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584377bc840 con 0x558437850250 2024-03-12T04:19:56.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f989c0a1ba0 0x7f989c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b17fa640 1 -- 172.21.15.42:0/3436897291 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f98a40cd4e0 con 0x7f98b40ff8f0 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x55843785bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x55843785bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41172/0 (socket says 172.21.15.42:41172) 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 -- 172.21.15.42:0/209618433 learned_addr learned my addr 172.21.15.42:0/209618433 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b3fff640 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437850630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b37fe640 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437859210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 -- 172.21.15.42:0/209618433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 msgr2=0x558437850630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437850630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 -- 172.21.15.42:0/209618433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 msgr2=0x558437859210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437859210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.184+0000 7f98b8959640 1 -- 172.21.15.42:0/209618433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55843777a060 con 0x558437859750 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.185+0000 7f98b8959640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f989c0a1ba0 0x7f989c0a4060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0077f0 tx=0x7f98ac007e40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.185+0000 7f98b8959640 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x55843785bb40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f98ac011c40 tx=0x7f98ac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9b3c5468bf6108 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.185+0000 7f989a7fc640 1 -- 172.21.15.42:0/209618433 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac014270 con 0x558437859750 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.185+0000 7f989a7fc640 1 -- 172.21.15.42:0/209618433 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98ac01a040 con 0x558437859750 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.185+0000 7f989a7fc640 1 -- 172.21.15.42:0/209618433 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0055e0 con 0x558437859750 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.185+0000 7f98bcf8f740 1 -- 172.21.15.42:0/209618433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 msgr2=0x55843785bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.185+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x55843785bb40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f98ac011c40 tx=0x7f98ac01b000 comp rx=0 tx=0).stop 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- 172.21.15.42:0/209618433 shutdown_connections 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x55843785bb40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437859210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/209618433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437850630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- 172.21.15.42:0/209618433 >> 172.21.15.42:0/209618433 conn(0x55843784a980 msgr2=0x55843784ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- 172.21.15.42:0/209618433 shutdown_connections 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- 172.21.15.42:0/209618433 wait complete. 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- start start 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 Processor -- start 2024-03-12T04:19:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437856760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850b70 0x558437856ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437859750 0x5584378571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584377b8020 con 0x558437850b70 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843748b430 con 0x558437859750 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558437447e00 con 0x558437850250 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98bcf8f740 10 client.0 _ll_register_callbacks cb 0x55843775fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98b37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850b70 0x558437856ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98b37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850b70 0x558437856ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41174/0 (socket says 172.21.15.42:41174) 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.186+0000 7f98b37fe640 1 -- 172.21.15.42:0/2150353898 learned_addr learned my addr 172.21.15.42:0/2150353898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b8959640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437859750 0x5584378571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b3fff640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437856760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b37fe640 1 -- 172.21.15.42:0/2150353898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 msgr2=0x558437856760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b37fe640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437856760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b37fe640 1 -- 172.21.15.42:0/2150353898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437859750 msgr2=0x5584378571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b37fe640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437859750 0x5584378571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b37fe640 1 -- 172.21.15.42:0/2150353898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac014c60 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b8959640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437859750 0x5584378571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b3fff640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437856760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98b37fe640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850b70 0x558437856ca0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009900 tx=0x7f98a802c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5909d255c3ef21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98937fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98937fe640 1 -- 172.21.15.42:0/2150353898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8004240 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98937fe640 1 -- 172.21.15.42:0/2150353898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f98a80043a0 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584374b77e0 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98937fe640 1 -- 172.21.15.42:0/2150353898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98a80035f0 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98bcf8f740 10 client.16809 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5584377b66a0 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.187+0000 7f98937fe640 1 -- 172.21.15.42:0/2150353898 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5584374b77e0 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98937fe640 1 -- 172.21.15.42:0/2150353898 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f98a80051f0 con 0x558437850b70 2024-03-12T04:19:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98bcf8f740 10 client.16809 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98bcf8f740 10 client.16809 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98bcf8f740 20 client.16809 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558437869a90 0x55843786beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98bcf8f740 4 client.16809 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584374f4720 con 0x558437869a90 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98b3fff640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558437869a90 0x55843786beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98937fe640 10 client.16809 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98b3fff640 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558437869a90 0x55843786beb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=576728d38db767e3 server_cookie=b518c85607b0b1cf in_seq=0 out_seq=0 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98937fe640 1 -- 172.21.15.42:0/2150353898 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584374f4720 con 0x558437869a90 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98937fe640 10 client.16809 handle_command_reply: tid=0 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.188+0000 7f98bcf8f740 1 client.16809 shutdown 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850b70 msgr2=0x558437856ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850b70 0x558437856ca0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009900 tx=0x7f98a802c9a0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 shutdown_connections 2024-03-12T04:19:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437859750 0x5584378571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558437869a90 0x55843786beb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850b70 0x558437856ca0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2150353898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850250 0x558437856760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98937fe640 0 client.16809 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 >> 172.21.15.42:0/2150353898 conn(0x55843784a980 msgr2=0x55843784c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 shutdown_connections 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2150353898 wait complete. 2024-03-12T04:19:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.189+0000 7f98bcf8f740 20 client.16809 trim_cache size 0 max 16384 2024-03-12T04:19:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 Processor -- start 2024-03-12T04:19:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 -- start start 2024-03-12T04:19:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850250 0x558437858e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437857000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437859750 0x5584378539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843773d130 con 0x558437850250 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558437871100 con 0x558437850b70 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843784bb40 con 0x558437859750 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850250 0x558437858e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850250 0x558437858e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41188/0 (socket says 172.21.15.42:41188) 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 -- 172.21.15.42:0/3575172988 learned_addr learned my addr 172.21.15.42:0/3575172988 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 -- 172.21.15.42:0/3575172988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437859750 msgr2=0x5584378539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437859750 0x5584378539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 -- 172.21.15.42:0/3575172988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 msgr2=0x558437857000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b37fe640 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437857000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437857000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 -- 172.21.15.42:0/3575172988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558437752c20 con 0x558437850250 2024-03-12T04:19:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f98b3fff640 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850250 0x558437858e90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f98a4096ad0 tx=0x7f98a4099b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4381f4f283ea3171 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.201+0000 7f9899ffb640 1 -- 172.21.15.42:0/3575172988 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a409e250 con 0x558437850250 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f9899ffb640 1 -- 172.21.15.42:0/3575172988 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98a409ef10 con 0x558437850250 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f9899ffb640 1 -- 172.21.15.42:0/3575172988 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a409f170 con 0x558437850250 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 -- 172.21.15.42:0/3575172988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850250 msgr2=0x558437858e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850250 0x558437858e90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f98a4096ad0 tx=0x7f98a4099b80 comp rx=0 tx=0).stop 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 -- 172.21.15.42:0/3575172988 shutdown_connections 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437859750 0x5584378539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850b70 0x558437857000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/3575172988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437850250 0x558437858e90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 -- 172.21.15.42:0/3575172988 >> 172.21.15.42:0/3575172988 conn(0x55843784a980 msgr2=0x558437851a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 -- 172.21.15.42:0/3575172988 shutdown_connections 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 -- 172.21.15.42:0/3575172988 wait complete. 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.202+0000 7f98bcf8f740 1 -- start start 2024-03-12T04:19:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 1 Processor -- start 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850250 0x55843785f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850b70 0x55843785fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x5584378bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584378602e0 con 0x558437859750 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584378c0520 con 0x558437850250 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584378c06b0 con 0x558437850b70 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98bcf8f740 10 client.0 _ll_register_callbacks cb 0x7f98ac0147b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x5584378bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x5584378bffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41202/0 (socket says 172.21.15.42:41202) 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 -- 172.21.15.42:0/2505339966 learned_addr learned my addr 172.21.15.42:0/2505339966 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b37fe640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850b70 0x55843785fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 -- 172.21.15.42:0/2505339966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850b70 msgr2=0x55843785fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850b70 0x55843785fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b3fff640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850250 0x55843785f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 -- 172.21.15.42:0/2505339966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850250 msgr2=0x55843785f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850250 0x55843785f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b8959640 1 -- 172.21.15.42:0/2505339966 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a407ea90 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.203+0000 7f98b37fe640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850b70 0x55843785fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98b8959640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x5584378bffe0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x55843784bb10 tx=0x7f98ac004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844410378ffb69c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98937fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98937fe640 1 -- 172.21.15.42:0/2505339966 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac012860 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98937fe640 1 -- 172.21.15.42:0/2505339966 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f98ac0129c0 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98ac01a040 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98937fe640 1 -- 172.21.15.42:0/2505339966 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98ac011ed0 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 10 client.16815 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5584377788c0 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98937fe640 1 -- 172.21.15.42:0/2505339966 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98ac01a040 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98937fe640 1 -- 172.21.15.42:0/2505339966 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f98ac0054f0 con 0x558437859750 2024-03-12T04:19:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 10 client.16815 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 10 client.16815 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 20 client.16815 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5584378c1e20 0x5584378c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 4 client.16815 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558437543890 con 0x5584378c1e20 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.204+0000 7f98b3fff640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5584378c1e20 0x5584378c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98937fe640 10 client.16815 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98b3fff640 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5584378c1e20 0x5584378c4240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4786bfeb212a0ae5 server_cookie=29cac0885521ed7a in_seq=0 out_seq=0 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98937fe640 1 -- 172.21.15.42:0/2505339966 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558437543890 con 0x5584378c1e20 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98937fe640 10 client.16815 handle_command_reply: tid=0 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 client.16815 shutdown 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 msgr2=0x5584378bffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x5584378bffe0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x55843784bb10 tx=0x7f98ac004c70 comp rx=0 tx=0).stop 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 shutdown_connections 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5584378c1e20 0x5584378c4240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558437859750 0x5584378bffe0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558437850b70 0x55843785fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 --2- 172.21.15.42:0/2505339966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558437850250 0x55843785f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98937fe640 0 client.16815 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 >> 172.21.15.42:0/2505339966 conn(0x55843784a980 msgr2=0x55843784bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 shutdown_connections 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 1 -- 172.21.15.42:0/2505339966 wait complete. 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.205+0000 7f98bcf8f740 20 client.16815 trim_cache size 0 max 16384 2024-03-12T04:19:56.214 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f989c0a1ba0 msgr2=0x7f989c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f989c0a1ba0 0x7f989c0a4060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0077f0 tx=0x7f98ac007e40 comp rx=0 tx=0).stop 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b40ff8f0 msgr2=0x7f98b41976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b40ff8f0 0x7f98b41976e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f98a400d950 tx=0x7f98a400de10 comp rx=0 tx=0).stop 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 shutdown_connections 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f989c0a1ba0 0x7f989c0a4060 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b419bfa0 0x7f98b419c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b4106500 0x7f98b4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 --2- 172.21.15.42:0/3436897291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98b40ff8f0 0x7f98b41976e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 >> 172.21.15.42:0/3436897291 conn(0x7f98b4076640 msgr2=0x7f98b4104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 shutdown_connections 2024-03-12T04:19:56.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.206+0000 7f98ba3bc640 1 -- 172.21.15.42:0/3436897291 wait complete. 2024-03-12T04:19:56.219 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:56.219 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:19:56.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/754367255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 msgr2=0x7fe9b0100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/754367255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b0100680 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009930 tx=0x7fe9ac02bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:56.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/754367255 shutdown_connections 2024-03-12T04:19:56.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/754367255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b0105370 0x7fe9b010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/754367255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b0100c50 0x7fe9b0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/754367255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b0100680 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/754367255 >> 172.21.15.42:0/754367255 conn(0x7fe9b0076400 msgr2=0x7fe9b0076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/754367255 shutdown_connections 2024-03-12T04:19:56.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.321+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/754367255 wait complete. 2024-03-12T04:19:56.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 Processor -- start 2024-03-12T04:19:56.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 -- start start 2024-03-12T04:19:56.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b0105370 0x7fe9b0197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b019afb0 0x7fe9b019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b019b9a0 con 0x7fe9b01002a0 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b019bb10 con 0x7fe9b019afb0 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b7f2e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b019bcd0 con 0x7fe9b0105370 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b5cca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b5cca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b01977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41206/0 (socket says 172.21.15.42:41206) 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b5cca640 1 -- 172.21.15.42:0/242405874 learned_addr learned my addr 172.21.15.42:0/242405874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b0105370 0x7fe9b0197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.322+0000 7fe9b64cb640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b019afb0 0x7fe9b019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b5cca640 1 -- 172.21.15.42:0/242405874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b0105370 msgr2=0x7fe9b0197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b0105370 0x7fe9b0197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b5cca640 1 -- 172.21.15.42:0/242405874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b019afb0 msgr2=0x7fe9b019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b019afb0 0x7fe9b019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b5cca640 1 -- 172.21.15.42:0/242405874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac009560 con 0x7fe9b01002a0 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b64cb640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b019afb0 0x7fe9b019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b01977a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009230 tx=0x7fe9ac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9a6ffd640 1 -- 172.21.15.42:0/242405874 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac004440 con 0x7fe9b01002a0 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9a6ffd640 1 -- 172.21.15.42:0/242405874 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe9ac02de90 con 0x7fe9b01002a0 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9a6ffd640 1 -- 172.21.15.42:0/242405874 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0052e0 con 0x7fe9b01002a0 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b01a4340 con 0x7fe9b01002a0 2024-03-12T04:19:56.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.323+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b01a4770 con 0x7fe9b01002a0 2024-03-12T04:19:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.325+0000 7fe9a6ffd640 1 -- 172.21.15.42:0/242405874 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe9ac003cf0 con 0x7fe9b01002a0 2024-03-12T04:19:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.325+0000 7fe9babd7740 1 Processor -- start 2024-03-12T04:19:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.325+0000 7fe9babd7740 1 -- start start 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503d2750 0x5604503d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9babd7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450384960 con 0x5604503d2750 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9babd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450338310 con 0x5604503c9b70 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9babd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450335840 con 0x5604503c9250 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9a6ffd640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe99c0a1c20 0x7fe99c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503d2750 0x5604503d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503d2750 0x5604503d4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41214/0 (socket says 172.21.15.42:41214) 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 -- 172.21.15.42:0/1783373778 learned_addr learned my addr 172.21.15.42:0/1783373778 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b64cb640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe99c0a1c20 0x7fe99c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9a6ffd640 1 -- 172.21.15.42:0/242405874 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe9ac0ef650 con 0x7fe9b01002a0 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b64cb640 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 -- 172.21.15.42:0/1783373778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 msgr2=0x5604503c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 -- 172.21.15.42:0/1783373778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 msgr2=0x5604503d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 -- 172.21.15.42:0/1783373778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604502f3060 con 0x5604503d2750 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b64cb640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe99c0a1c20 0x7fe99c0a40e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8005fd0 tx=0x7fe9a8012b10 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.326+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503d2750 0x5604503d4b40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000bc30 tx=0x7fe9a00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c3525dcb9f4fcd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe99bfff640 1 -- 172.21.15.42:0/1783373778 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a001c480 con 0x5604503d2750 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe99bfff640 1 -- 172.21.15.42:0/1783373778 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe9a0015070 con 0x5604503d2750 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe99bfff640 1 -- 172.21.15.42:0/1783373778 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0011860 con 0x5604503d2750 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 -- 172.21.15.42:0/1783373778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503d2750 msgr2=0x5604503d4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503d2750 0x5604503d4b40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000bc30 tx=0x7fe9a00107a0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 -- 172.21.15.42:0/1783373778 shutdown_connections 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503d2750 0x5604503d4b40 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1783373778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 -- 172.21.15.42:0/1783373778 >> 172.21.15.42:0/1783373778 conn(0x5604503c3980 msgr2=0x5604503c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 -- 172.21.15.42:0/1783373778 shutdown_connections 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.327+0000 7fe9babd7740 1 -- 172.21.15.42:0/1783373778 wait complete. 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 -- start start 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 Processor -- start 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9250 0x5604503cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450331020 con 0x5604503c9b70 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450004430 con 0x5604503c9250 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044ffc0e00 con 0x5604503d2750 2024-03-12T04:19:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9babd7740 10 client.0 _ll_register_callbacks cb 0x5604502d8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9b54c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9b54c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41218/0 (socket says 172.21.15.42:41218) 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9b54c9640 1 -- 172.21.15.42:0/253238919 learned_addr learned my addr 172.21.15.42:0/253238919 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9250 0x5604503cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.328+0000 7fe9b64cb640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9b54c9640 1 -- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 msgr2=0x5604503d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9b54c9640 1 -- 172.21.15.42:0/253238919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9250 msgr2=0x5604503cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9250 0x5604503cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9b54c9640 1 -- 172.21.15.42:0/253238919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a000b860 con 0x5604503c9b70 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9250 0x5604503cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503cfca0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0011480 tx=0x7fe9a000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc89353f560fea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9997fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9997fa640 1 -- 172.21.15.42:0/253238919 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0021070 con 0x5604503c9b70 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9997fa640 1 -- 172.21.15.42:0/253238919 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe9a0017070 con 0x5604503c9b70 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604500307e0 con 0x5604503c9b70 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9997fa640 1 -- 172.21.15.42:0/253238919 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9a0005410 con 0x5604503c9b70 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 10 client.16827 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56045032f6a0 con 0x5604503c9b70 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9997fa640 1 -- 172.21.15.42:0/253238919 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5604500307e0 con 0x5604503c9b70 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9997fa640 1 -- 172.21.15.42:0/253238919 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe9a001c9e0 con 0x5604503c9b70 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 10 client.16827 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 10 client.16827 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 20 client.16827 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5604503e2a40 0x5604503e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 4 client.16827 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.329+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56045006d720 con 0x5604503e2a40 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.330+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5604503e2a40 0x5604503e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.330+0000 7fe9997fa640 10 client.16827 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.330+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5604503e2a40 0x5604503e4e80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8c3059b9441f257 server_cookie=f421b4c5d1435fca in_seq=0 out_seq=0 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9997fa640 1 -- 172.21.15.42:0/253238919 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56045006d720 con 0x5604503e2a40 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9997fa640 10 client.16827 handle_command_reply: tid=0 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 client.16827 shutdown 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 msgr2=0x5604503cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503cfca0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0011480 tx=0x7fe9a000bf50 comp rx=0 tx=0).stop 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 shutdown_connections 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5604503e2a40 0x5604503e4e80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503cfca0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9997fa640 0 client.16827 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 --2- 172.21.15.42:0/253238919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9250 0x5604503cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 >> 172.21.15.42:0/253238919 conn(0x5604503c3980 msgr2=0x5604503c56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 shutdown_connections 2024-03-12T04:19:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 1 -- 172.21.15.42:0/253238919 wait complete. 2024-03-12T04:19:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.331+0000 7fe9babd7740 20 client.16827 trim_cache size 0 max 16384 2024-03-12T04:19:56.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 Processor -- start 2024-03-12T04:19:56.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 -- start start 2024-03-12T04:19:56.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503d2750 0x5604503cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604502b6130 con 0x5604503c9b70 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604503c4780 con 0x5604503d2750 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9babd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604503c8c90 con 0x5604503c9250 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41232/0 (socket says 172.21.15.42:41232) 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 -- 172.21.15.42:0/869392462 learned_addr learned my addr 172.21.15.42:0/869392462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 -- 172.21.15.42:0/869392462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 msgr2=0x5604503d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503d1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 -- 172.21.15.42:0/869392462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503d2750 msgr2=0x5604503cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503d2750 0x5604503cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.343+0000 7fe9b54c9640 1 -- 172.21.15.42:0/869392462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604502cbc20 con 0x5604503c9b70 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503d0000 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0005d00 tx=0x7fe9a0004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328ce4f735e58d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe99b7fe640 1 -- 172.21.15.42:0/869392462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0021070 con 0x5604503c9b70 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe99b7fe640 1 -- 172.21.15.42:0/869392462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe9a0005410 con 0x5604503c9b70 2024-03-12T04:19:56.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe99b7fe640 1 -- 172.21.15.42:0/869392462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00035f0 con 0x5604503c9b70 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 -- 172.21.15.42:0/869392462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 msgr2=0x5604503d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503d0000 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0005d00 tx=0x7fe9a0004900 comp rx=0 tx=0).stop 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 -- 172.21.15.42:0/869392462 shutdown_connections 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503d2750 0x5604503cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9b70 0x5604503d0000 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 --2- 172.21.15.42:0/869392462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503c9250 0x5604503d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 -- 172.21.15.42:0/869392462 >> 172.21.15.42:0/869392462 conn(0x5604503c3980 msgr2=0x5604503caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 -- 172.21.15.42:0/869392462 shutdown_connections 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.344+0000 7fe9babd7740 1 -- 172.21.15.42:0/869392462 wait complete. 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 -- start start 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 Processor -- start 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9250 0x5604503d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450438fe0 con 0x5604503c9250 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450439150 con 0x5604503c9b70 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560450439310 con 0x5604503d2750 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9babd7740 10 client.0 _ll_register_callbacks cb 0x7fe9a000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9250 0x5604503d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9250 0x5604503d8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41238/0 (socket says 172.21.15.42:41238) 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 -- 172.21.15.42:0/1165800425 learned_addr learned my addr 172.21.15.42:0/1165800425 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b64cb640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 -- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 msgr2=0x5604503d8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 -- 172.21.15.42:0/1165800425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 msgr2=0x5604503d8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b5cca640 1 -- 172.21.15.42:0/1165800425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56045032f6a0 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.345+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9b5cca640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9250 0x5604503d8140 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac005c40 tx=0x7fe9ac0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe49ff90df61b05 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9997fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9997fa640 1 -- 172.21.15.42:0/1165800425 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0a0ea0 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9997fa640 1 -- 172.21.15.42:0/1165800425 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe9ac0c4b20 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9a0015070 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9997fa640 1 -- 172.21.15.42:0/1165800425 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9ac0b89f0 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 10 client.16833 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5604502f18c0 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9997fa640 1 -- 172.21.15.42:0/1165800425 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe9a0015070 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9997fa640 1 -- 172.21.15.42:0/1165800425 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe9ac03c070 con 0x5604503c9250 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 10 client.16833 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 10 client.16833 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 20 client.16833 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56045043a940 0x56045043cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 4 client.16833 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604500bc890 con 0x56045043a940 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.346+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56045043a940 0x56045043cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9997fa640 10 client.16833 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9b54c9640 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56045043a940 0x56045043cd30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3b8e5b6d1027590 server_cookie=dbfb13ee11ca4094 in_seq=0 out_seq=0 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9997fa640 1 -- 172.21.15.42:0/1165800425 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604500bc890 con 0x56045043a940 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9997fa640 10 client.16833 handle_command_reply: tid=0 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 client.16833 shutdown 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9250 msgr2=0x5604503d8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9250 0x5604503d8140 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac005c40 tx=0x7fe9ac0dc040 comp rx=0 tx=0).stop 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 shutdown_connections 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56045043a940 0x56045043cd30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5604503d2750 0x5604503d8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9997fa640 0 client.16833 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:19:56.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604503c9b70 0x5604503d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 --2- 172.21.15.42:0/1165800425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604503c9250 0x5604503d8140 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 >> 172.21.15.42:0/1165800425 conn(0x5604503c3980 msgr2=0x5604503c22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 shutdown_connections 2024-03-12T04:19:56.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 1 -- 172.21.15.42:0/1165800425 wait complete. 2024-03-12T04:19:56.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.347+0000 7fe9babd7740 20 client.16833 trim_cache size 0 max 16384 2024-03-12T04:19:56.357 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:56.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe99c0a1c20 msgr2=0x7fe99c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe99c0a1c20 0x7fe99c0a40e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8005fd0 tx=0x7fe9a8012b10 comp rx=0 tx=0).stop 2024-03-12T04:19:56.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 msgr2=0x7fe9b01977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b01977a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009230 tx=0x7fe9ac004040 comp rx=0 tx=0).stop 2024-03-12T04:19:56.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 shutdown_connections 2024-03-12T04:19:56.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe99c0a1c20 0x7fe99c0a40e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b019afb0 0x7fe9b019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b0105370 0x7fe9b0197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 --2- 172.21.15.42:0/242405874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b01002a0 0x7fe9b01977a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 >> 172.21.15.42:0/242405874 conn(0x7fe9b0076400 msgr2=0x7fe9b0069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 shutdown_connections 2024-03-12T04:19:56.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.348+0000 7fe9b7f2e640 1 -- 172.21.15.42:0/242405874 wait complete. 2024-03-12T04:19:56.361 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:56.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.463+0000 7f40458f1640 1 -- 172.21.15.42:0/1416684702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 msgr2=0x7f40401006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.463+0000 7f40458f1640 1 --2- 172.21.15.42:0/1416684702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f40401006f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4030009930 tx=0x7f403002bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.463+0000 7f40458f1640 1 -- 172.21.15.42:0/1416684702 shutdown_connections 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.463+0000 7f40458f1640 1 --2- 172.21.15.42:0/1416684702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40401053e0 0x7f404010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.463+0000 7f40458f1640 1 --2- 172.21.15.42:0/1416684702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040100cc0 0x7f4040104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.463+0000 7f40458f1640 1 --2- 172.21.15.42:0/1416684702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f40401006f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 -- 172.21.15.42:0/1416684702 >> 172.21.15.42:0/1416684702 conn(0x7f40400771e0 msgr2=0x7f40400775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 -- 172.21.15.42:0/1416684702 shutdown_connections 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 -- 172.21.15.42:0/1416684702 wait complete. 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 Processor -- start 2024-03-12T04:19:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 -- start start 2024-03-12T04:19:56.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f4040198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40401053e0 0x7f4040198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.464+0000 7f40458f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4040199020 0x7f40401a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40458f1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40401a3c70 con 0x7f4040100310 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40458f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40401a3de0 con 0x7f4040199020 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40458f1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40401a3fa0 con 0x7f40401053e0 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f4040198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f4040198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41264/0 (socket says 172.21.15.42:41264) 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 -- 172.21.15.42:0/412905253 learned_addr learned my addr 172.21.15.42:0/412905253 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 -- 172.21.15.42:0/412905253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40401053e0 msgr2=0x7f4040198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403e7fc640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40401053e0 0x7f4040198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40401053e0 0x7f4040198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 -- 172.21.15.42:0/412905253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4040199020 msgr2=0x7f40401a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4040199020 0x7f40401a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 -- 172.21.15.42:0/412905253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030009560 con 0x7f4040100310 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403e7fc640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40401053e0 0x7f4040198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f403effd640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f4040198550 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4030009230 tx=0x7f40300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40448ef640 1 -- 172.21.15.42:0/412905253 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300044e0 con 0x7f4040100310 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40401a4220 con 0x7f4040100310 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40448ef640 1 -- 172.21.15.42:0/412905253 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f403002de90 con 0x7f4040100310 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40448ef640 1 -- 172.21.15.42:0/412905253 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030005400 con 0x7f4040100310 2024-03-12T04:19:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.465+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40401a4730 con 0x7f4040100310 2024-03-12T04:19:56.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.467+0000 7f40448ef640 1 -- 172.21.15.42:0/412905253 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f4030005560 con 0x7f4040100310 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 Processor -- start 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 -- start start 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8512630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b851b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b851db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b84cd960 con 0x5556b8512b70 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b8481310 con 0x5556b851b750 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f404859e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b847e840 con 0x5556b8512250 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b851b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b851b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41280/0 (socket says 172.21.15.42:41280) 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 -- 172.21.15.42:0/3059323232 learned_addr learned my addr 172.21.15.42:0/3059323232 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403f7fe640 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8512630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403e7fc640 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b851db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f40448ef640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40280a1ba0 0x7f40280a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 -- 172.21.15.42:0/3059323232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 msgr2=0x5556b8512630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8512630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 -- 172.21.15.42:0/3059323232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 msgr2=0x5556b851db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b851db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403effd640 1 -- 172.21.15.42:0/3059323232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556b843c060 con 0x5556b8512b70 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403e7fc640 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b851db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f403f7fe640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40280a1ba0 0x7f40280a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.468+0000 7f40448ef640 1 -- 172.21.15.42:0/412905253 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f40300efa00 con 0x7f4040100310 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f403effd640 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b851b210 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f40300ed220 tx=0x7f4030087a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa02d0651576a7c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f4025ffb640 1 -- 172.21.15.42:0/3059323232 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300f9040 con 0x5556b8512b70 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f4025ffb640 1 -- 172.21.15.42:0/3059323232 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40300bc2d0 con 0x5556b8512b70 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f4025ffb640 1 -- 172.21.15.42:0/3059323232 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300f3220 con 0x5556b8512b70 2024-03-12T04:19:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 -- 172.21.15.42:0/3059323232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 msgr2=0x5556b851b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f403f7fe640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40280a1ba0 0x7f40280a4060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4038005cc0 tx=0x7f4038004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b851b210 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f40300ed220 tx=0x7f4030087a20 comp rx=0 tx=0).stop 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 -- 172.21.15.42:0/3059323232 shutdown_connections 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b851db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b851b210 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 --2- 172.21.15.42:0/3059323232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8512630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 -- 172.21.15.42:0/3059323232 >> 172.21.15.42:0/3059323232 conn(0x5556b850c980 msgr2=0x5556b850cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.469+0000 7f404859e740 1 -- 172.21.15.42:0/3059323232 shutdown_connections 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 -- 172.21.15.42:0/3059323232 wait complete. 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 -- start start 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 Processor -- start 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b85187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8518ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b851b750 0x5556b8519220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b847a020 con 0x5556b851b750 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b814d430 con 0x5556b8512b70 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b8109e00 con 0x5556b8512250 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f404859e740 10 client.0 _ll_register_callbacks cb 0x5556b8421cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f403f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b851b750 0x5556b8519220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f403f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b851b750 0x5556b8519220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41294/0 (socket says 172.21.15.42:41294) 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f403f7fe640 1 -- 172.21.15.42:0/220013659 learned_addr learned my addr 172.21.15.42:0/220013659 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f403e7fc640 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8518ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.470+0000 7f403effd640 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b85187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f403f7fe640 1 -- 172.21.15.42:0/220013659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 msgr2=0x5556b85187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f403f7fe640 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b85187a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f403f7fe640 1 -- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 msgr2=0x5556b8518ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f403f7fe640 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8518ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f403f7fe640 1 -- 172.21.15.42:0/220013659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40401a4730 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f403f7fe640 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b851b750 0x5556b8519220 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4038011940 tx=0x7f40380074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4290c30b51a84d0e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f401effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f401effd640 1 -- 172.21.15.42:0/220013659 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403801cd80 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f401effd640 1 -- 172.21.15.42:0/220013659 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4038015070 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556b81797e0 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f401effd640 1 -- 172.21.15.42:0/220013659 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f403801a450 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f404859e740 10 client.16845 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556b84786a0 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f401effd640 1 -- 172.21.15.42:0/220013659 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5556b81797e0 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.471+0000 7f401effd640 1 -- 172.21.15.42:0/220013659 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4038025580 con 0x5556b851b750 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f404859e740 10 client.16845 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f404859e740 10 client.16845 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f404859e740 20 client.16845 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f404859e740 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b852b820 0x5556b852dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f404859e740 4 client.16845 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556b81b6720 con 0x5556b852b820 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f403effd640 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b852b820 0x5556b852dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f403effd640 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b852b820 0x5556b852dc40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f75938062791f38 server_cookie=70a7bd53e6607215 in_seq=0 out_seq=0 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.472+0000 7f401effd640 10 client.16845 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f401effd640 1 -- 172.21.15.42:0/220013659 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556b81b6720 con 0x5556b852b820 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f401effd640 10 client.16845 handle_command_reply: tid=0 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 client.16845 shutdown 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b851b750 msgr2=0x5556b8519220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b851b750 0x5556b8519220 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4038011940 tx=0x7f40380074a0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 shutdown_connections 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b852b820 0x5556b852dc40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b851b750 0x5556b8519220 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8518ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 --2- 172.21.15.42:0/220013659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b85187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 >> 172.21.15.42:0/220013659 conn(0x5556b850c980 msgr2=0x5556b850e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f401effd640 0 client.16845 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:56.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 shutdown_connections 2024-03-12T04:19:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 1 -- 172.21.15.42:0/220013659 wait complete. 2024-03-12T04:19:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.473+0000 7f404859e740 20 client.16845 trim_cache size 0 max 16384 2024-03-12T04:19:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 Processor -- start 2024-03-12T04:19:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 -- start start 2024-03-12T04:19:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512250 0x5556b851aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8519040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b851b750 0x5556b8513a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b83ff130 con 0x5556b8512250 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b85283d0 con 0x5556b8512b70 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f404859e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b851db40 con 0x5556b851b750 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f403effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512250 0x5556b851aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f403effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512250 0x5556b851aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41302/0 (socket says 172.21.15.42:41302) 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.485+0000 7f403effd640 1 -- 172.21.15.42:0/2544322589 learned_addr learned my addr 172.21.15.42:0/2544322589 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403f7fe640 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b851b750 0x5556b8513a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403e7fc640 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8519040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403effd640 1 -- 172.21.15.42:0/2544322589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b851b750 msgr2=0x5556b8513a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403effd640 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b851b750 0x5556b8513a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403effd640 1 -- 172.21.15.42:0/2544322589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 msgr2=0x5556b8519040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403effd640 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8519040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403effd640 1 -- 172.21.15.42:0/2544322589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556b8414c20 con 0x5556b8512250 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f403effd640 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512250 0x5556b851aed0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4030005ec0 tx=0x7f403002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f27c9ba3df025e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f40257fa640 1 -- 172.21.15.42:0/2544322589 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300c6220 con 0x5556b8512250 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f40257fa640 1 -- 172.21.15.42:0/2544322589 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40300bc0f0 con 0x5556b8512250 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f40257fa640 1 -- 172.21.15.42:0/2544322589 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300c08e0 con 0x5556b8512250 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f404859e740 1 -- 172.21.15.42:0/2544322589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512250 msgr2=0x5556b851aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.486+0000 7f404859e740 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512250 0x5556b851aed0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4030005ec0 tx=0x7f403002bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- 172.21.15.42:0/2544322589 shutdown_connections 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b851b750 0x5556b8513a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b8512b70 0x5556b8519040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 --2- 172.21.15.42:0/2544322589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512250 0x5556b851aed0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- 172.21.15.42:0/2544322589 >> 172.21.15.42:0/2544322589 conn(0x5556b850c980 msgr2=0x5556b85325a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- 172.21.15.42:0/2544322589 shutdown_connections 2024-03-12T04:19:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- 172.21.15.42:0/2544322589 wait complete. 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- start start 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 Processor -- start 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8522450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b8520840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b8520d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b8521310 con 0x5556b8512b70 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b8521480 con 0x5556b851b750 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b8521640 con 0x5556b8512250 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f404859e740 10 client.0 _ll_register_callbacks cb 0x7f40300f34c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f403e7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b8520840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f403e7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b8520840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41306/0 (socket says 172.21.15.42:41306) 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.487+0000 7f403e7fc640 1 -- 172.21.15.42:0/2829927249 learned_addr learned my addr 172.21.15.42:0/2829927249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403f7fe640 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b8520d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403effd640 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8522450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403e7fc640 1 -- 172.21.15.42:0/2829927249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 msgr2=0x5556b8522450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403e7fc640 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8522450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403e7fc640 1 -- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 msgr2=0x5556b8520d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403e7fc640 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b8520d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403e7fc640 1 -- 172.21.15.42:0/2829927249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403008de70 con 0x5556b8512b70 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f403e7fc640 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b8520840 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f40340058c0 tx=0x7f40340117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e9e93453f65e7a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f401effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f401effd640 1 -- 172.21.15.42:0/2829927249 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034016070 con 0x5556b8512b70 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f401effd640 1 -- 172.21.15.42:0/2829927249 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f403400f420 con 0x5556b8512b70 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40300bc2d0 con 0x5556b8512b70 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f401effd640 1 -- 172.21.15.42:0/2829927249 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40340112e0 con 0x5556b8512b70 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f404859e740 10 client.16851 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556b843a8c0 con 0x5556b8512b70 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.488+0000 7f401effd640 1 -- 172.21.15.42:0/2829927249 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40300bc2d0 con 0x5556b8512b70 2024-03-12T04:19:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f401effd640 1 -- 172.21.15.42:0/2829927249 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f403400e860 con 0x5556b8512b70 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f404859e740 10 client.16851 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f404859e740 10 client.16851 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f404859e740 20 client.16851 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f404859e740 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b85834b0 0x5556b85858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f404859e740 4 client.16851 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5556b8205890 con 0x5556b85834b0 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f403effd640 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b85834b0 0x5556b85858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f403effd640 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b85834b0 0x5556b85858d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c61bc1e7f84320c3 server_cookie=f5a92444907a6b4 in_seq=0 out_seq=0 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.489+0000 7f401effd640 10 client.16851 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f401effd640 1 -- 172.21.15.42:0/2829927249 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5556b8205890 con 0x5556b85834b0 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f401effd640 10 client.16851 handle_command_reply: tid=0 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 client.16851 shutdown 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 msgr2=0x5556b8520840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b8520840 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f40340058c0 tx=0x7f40340117f0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 shutdown_connections 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556b85834b0 0x5556b85858d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556b851b750 0x5556b8520d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f401effd640 0 client.16851 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b8512b70 0x5556b8520840 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 --2- 172.21.15.42:0/2829927249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556b8512250 0x5556b8522450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 >> 172.21.15.42:0/2829927249 conn(0x5556b850c980 msgr2=0x5556b8513a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 shutdown_connections 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 1 -- 172.21.15.42:0/2829927249 wait complete. 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f404859e740 20 client.16851 trim_cache size 0 max 16384 2024-03-12T04:19:56.499 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:56.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40280a1ba0 msgr2=0x7f40280a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40280a1ba0 0x7f40280a4060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4038005cc0 tx=0x7f4038004a40 comp rx=0 tx=0).stop 2024-03-12T04:19:56.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 msgr2=0x7f4040198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f4040198550 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4030009230 tx=0x7f40300042f0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 shutdown_connections 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40280a1ba0 0x7f40280a4060 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4040199020 0x7f40401a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40401053e0 0x7f4040198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 --2- 172.21.15.42:0/412905253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4040100310 0x7f4040198550 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.490+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 >> 172.21.15.42:0/412905253 conn(0x7f40400771e0 msgr2=0x7f40400fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.491+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 shutdown_connections 2024-03-12T04:19:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.491+0000 7f40458f1640 1 -- 172.21.15.42:0/412905253 wait complete. 2024-03-12T04:19:56.504 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:56.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 -- 172.21.15.42:0/3401125380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b4105330 msgr2=0x7fb0b410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/3401125380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b4105330 0x7fb0b410cf00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009ca0 tx=0x7fb0a402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 -- 172.21.15.42:0/3401125380 shutdown_connections 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/3401125380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b4105330 0x7fb0b410cf00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/3401125380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b4100c10 0x7fb0b4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/3401125380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0b4100260 0x7fb0b4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 -- 172.21.15.42:0/3401125380 >> 172.21.15.42:0/3401125380 conn(0x7fb0b4077180 msgr2=0x7fb0b4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 -- 172.21.15.42:0/3401125380 shutdown_connections 2024-03-12T04:19:56.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 -- 172.21.15.42:0/3401125380 wait complete. 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.607+0000 7fb0b8af4640 1 Processor -- start 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b8af4640 1 -- start start 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b8af4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0b4100260 0x7fb0b4197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b8af4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b4105330 0x7fb0b4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b8af4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b419bf50 0x7fb0b419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b8af4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b419c940 con 0x7fb0b419bf50 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b8af4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b419cab0 con 0x7fb0b4105330 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b8af4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b419cc20 con 0x7fb0b4100260 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b419bf50 0x7fb0b419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b419bf50 0x7fb0b419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41320/0 (socket says 172.21.15.42:41320) 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 -- 172.21.15.42:0/2725041384 learned_addr learned my addr 172.21.15.42:0/2725041384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b259c640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0b4100260 0x7fb0b4197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b4105330 0x7fb0b4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 -- 172.21.15.42:0/2725041384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0b4100260 msgr2=0x7fb0b4197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0b4100260 0x7fb0b4197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 -- 172.21.15.42:0/2725041384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b4105330 msgr2=0x7fb0b4197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b4105330 0x7fb0b4197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 -- 172.21.15.42:0/2725041384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a4009900 con 0x7fb0b419bf50 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b259c640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0b4100260 0x7fb0b4197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b4105330 0x7fb0b4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.608+0000 7fb0b2d9d640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b419bf50 0x7fb0b419c400 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb0a402c700 tx=0x7fb0a4035d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.609+0000 7fb0a37fe640 1 -- 172.21.15.42:0/2725041384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40329e0 con 0x7fb0b419bf50 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.609+0000 7fb0a37fe640 1 -- 172.21.15.42:0/2725041384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0a4032410 con 0x7fb0b419bf50 2024-03-12T04:19:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.609+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b41a4200 con 0x7fb0b419bf50 2024-03-12T04:19:56.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.609+0000 7fb0a37fe640 1 -- 172.21.15.42:0/2725041384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4004840 con 0x7fb0b419bf50 2024-03-12T04:19:56.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.609+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b41a4620 con 0x7fb0b419bf50 2024-03-12T04:19:56.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.611+0000 7fb0a37fe640 1 -- 172.21.15.42:0/2725041384 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fb0a403b040 con 0x7fb0b419bf50 2024-03-12T04:19:56.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 Processor -- start 2024-03-12T04:19:56.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 -- start start 2024-03-12T04:19:56.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856f250 0x558f5856f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58578210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f58578750 0x558f5857ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5852a960 con 0x558f5856f250 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f584de310 con 0x558f5856fb70 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f584db840 con 0x558f58578750 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856f250 0x558f5856f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0a37fe640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb09c0a1ba0 0x7fb09c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856f250 0x558f5856f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41334/0 (socket says 172.21.15.42:41334) 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2843973309 learned_addr learned my addr 172.21.15.42:0/2843973309 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b2d9d640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb09c0a1ba0 0x7fb09c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b259c640 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58578210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f58578750 0x558f5857ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2843973309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f58578750 msgr2=0x558f5857ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0a37fe640 1 -- 172.21.15.42:0/2725041384 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb0a40f02c0 con 0x7fb0b419bf50 2024-03-12T04:19:56.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f58578750 0x558f5857ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2843973309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 msgr2=0x558f58578210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58578210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.612+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2843973309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f58499060 con 0x558f5856f250 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.613+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856f250 0x558f5856f630 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005d80 tx=0x7fb0a80095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504528a897cd5324 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.613+0000 7fb0b2d9d640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb09c0a1ba0 0x7fb09c0a4060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40b99c0 tx=0x7fb0a408f1f0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.613+0000 7fb0a0ff9640 1 -- 172.21.15.42:0/2843973309 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8005020 con 0x558f5856f250 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.613+0000 7fb0a0ff9640 1 -- 172.21.15.42:0/2843973309 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0a8005630 con 0x558f5856f250 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.613+0000 7fb0a0ff9640 1 -- 172.21.15.42:0/2843973309 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004ea0 con 0x558f5856f250 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.613+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2843973309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856f250 msgr2=0x558f5856f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.613+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856f250 0x558f5856f630 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005d80 tx=0x7fb0a80095b0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2843973309 shutdown_connections 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f58578750 0x558f5857ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58578210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2843973309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856f250 0x558f5856f630 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2843973309 >> 172.21.15.42:0/2843973309 conn(0x558f58569980 msgr2=0x558f58569d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2843973309 shutdown_connections 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2843973309 wait complete. 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- start start 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 Processor -- start 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f585757a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58575ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f58576220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f584d7020 con 0x558f58578750 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f581aa430 con 0x558f5856fb70 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f58166e00 con 0x558f5856f250 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0bb7a2740 10 client.0 _ll_register_callbacks cb 0x558f5847ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0b1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f58576220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0b1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f58576220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41344/0 (socket says 172.21.15.42:41344) 2024-03-12T04:19:56.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.614+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/797435067 learned_addr learned my addr 172.21.15.42:0/797435067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b259c640 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58575ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f585757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 msgr2=0x558f585757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f585757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/797435067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 msgr2=0x558f58575ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58575ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/797435067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8009970 con 0x558f58578750 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f58576220 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8010f00 tx=0x7fb0a8004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63b6384311aabfb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb099ffb640 1 -- 172.21.15.42:0/797435067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004020 con 0x558f58578750 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb099ffb640 1 -- 172.21.15.42:0/797435067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb0a8004180 con 0x558f58578750 2024-03-12T04:19:56.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f581d67e0 con 0x558f58578750 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb099ffb640 1 -- 172.21.15.42:0/797435067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0a8004340 con 0x558f58578750 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0bb7a2740 10 client.16863 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f584d56a0 con 0x558f58578750 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.615+0000 7fb099ffb640 1 -- 172.21.15.42:0/797435067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558f581d67e0 con 0x558f58578750 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb099ffb640 1 -- 172.21.15.42:0/797435067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0a800c070 con 0x558f58578750 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0bb7a2740 10 client.16863 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0bb7a2740 10 client.16863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0bb7a2740 20 client.16863 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f5858ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0bb7a2740 4 client.16863 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f58213720 con 0x558f58588a10 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0b259c640 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f5858ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb099ffb640 10 client.16863 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.616+0000 7fb0b259c640 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f5858ae30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=baf0287eee2a0f3 server_cookie=f77e12b58e0818f7 in_seq=0 out_seq=0 2024-03-12T04:19:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb099ffb640 1 -- 172.21.15.42:0/797435067 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f58213720 con 0x558f58588a10 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb099ffb640 10 client.16863 handle_command_reply: tid=0 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 client.16863 shutdown 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 msgr2=0x558f58576220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f58576220 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8010f00 tx=0x7fb0a8004d80 comp rx=0 tx=0).stop 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 shutdown_connections 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f5858ae30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f58576220 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb099ffb640 0 client.16863 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856fb70 0x558f58575ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/797435067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f585757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 >> 172.21.15.42:0/797435067 conn(0x558f58569980 msgr2=0x558f5856b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 shutdown_connections 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.617+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/797435067 wait complete. 2024-03-12T04:19:56.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.618+0000 7fb0bb7a2740 20 client.16863 trim_cache size 0 max 16384 2024-03-12T04:19:56.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 Processor -- start 2024-03-12T04:19:56.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 -- start start 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f58577ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856fb70 0x558f58572a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f58578750 0x558f58572f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5845c130 con 0x558f5856fb70 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5857b110 con 0x558f58578750 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f58585680 con 0x558f5856f250 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0b259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856fb70 0x558f58572a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0b259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856fb70 0x558f58572a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41360/0 (socket says 172.21.15.42:41360) 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.629+0000 7fb0b259c640 1 -- 172.21.15.42:0/373876235 learned_addr learned my addr 172.21.15.42:0/373876235 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f58578750 0x558f58572f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f58577ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b259c640 1 -- 172.21.15.42:0/373876235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 msgr2=0x558f58577ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b259c640 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f58577ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b259c640 1 -- 172.21.15.42:0/373876235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f58578750 msgr2=0x558f58572f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b259c640 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f58578750 0x558f58572f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b259c640 1 -- 172.21.15.42:0/373876235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f58471c20 con 0x558f5856fb70 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb0b259c640 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856fb70 0x558f58572a40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005fd0 tx=0x7fb0ac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a4205b3c019ee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb09bfff640 1 -- 172.21.15.42:0/373876235 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac018040 con 0x558f5856fb70 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb09bfff640 1 -- 172.21.15.42:0/373876235 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0ac01e070 con 0x558f5856fb70 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.630+0000 7fb09bfff640 1 -- 172.21.15.42:0/373876235 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac01a2b0 con 0x558f5856fb70 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/373876235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856fb70 msgr2=0x558f58572a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856fb70 0x558f58572a40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005fd0 tx=0x7fb0ac004040 comp rx=0 tx=0).stop 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/373876235 shutdown_connections 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f58578750 0x558f58572f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f5856fb70 0x558f58572a40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/373876235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856f250 0x558f58577ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/373876235 >> 172.21.15.42:0/373876235 conn(0x558f58569980 msgr2=0x558f5858f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/373876235 shutdown_connections 2024-03-12T04:19:56.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/373876235 wait complete. 2024-03-12T04:19:56.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- start start 2024-03-12T04:19:56.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 Processor -- start 2024-03-12T04:19:56.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856f250 0x558f5857e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856fb70 0x558f5857eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f585d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f585d9460 con 0x558f58578750 2024-03-12T04:19:56.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f585d95a0 con 0x558f5856f250 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f585d9760 con 0x558f5856fb70 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.631+0000 7fb0bb7a2740 10 client.0 _ll_register_callbacks cb 0x7fb0a8009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f585d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f585d8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41374/0 (socket says 172.21.15.42:41374) 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2641865209 learned_addr learned my addr 172.21.15.42:0/2641865209 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856f250 0x558f5857e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b259c640 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856fb70 0x558f5857eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856fb70 msgr2=0x558f5857eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856fb70 0x558f5857eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2641865209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856f250 msgr2=0x558f5857e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856f250 0x558f5857e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 -- 172.21.15.42:0/2641865209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac01ab70 con 0x558f58578750 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb0b1d9b640 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f585d8f20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80093e0 tx=0x7fb0a80134c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf863337a937a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.632+0000 7fb099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb099ffb640 1 -- 172.21.15.42:0/2641865209 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8010da0 con 0x558f58578750 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb099ffb640 1 -- 172.21.15.42:0/2641865209 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb0a8000e40 con 0x558f58578750 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0a8005630 con 0x558f58578750 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb099ffb640 1 -- 172.21.15.42:0/2641865209 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0a8013740 con 0x558f58578750 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 10 client.16869 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f584978c0 con 0x558f58578750 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb099ffb640 1 -- 172.21.15.42:0/2641865209 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0a8046020 con 0x558f58578750 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb099ffb640 1 -- 172.21.15.42:0/2641865209 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0a800c070 con 0x558f58578750 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 10 client.16869 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 10 client.16869 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 20 client.16869 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f585e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 4 client.16869 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.634+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558f58262890 con 0x558f58588a10 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.635+0000 7fb0b259c640 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f585e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.635+0000 7fb0b259c640 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f585e2c30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fba869c9ecd4c6aa server_cookie=3d223702ae0b31e0 in_seq=0 out_seq=0 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.635+0000 7fb099ffb640 10 client.16869 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.635+0000 7fb099ffb640 1 -- 172.21.15.42:0/2641865209 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558f58262890 con 0x558f58588a10 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.635+0000 7fb099ffb640 10 client.16869 handle_command_reply: tid=0 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.635+0000 7fb0bb7a2740 1 client.16869 shutdown 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 msgr2=0x558f585d8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f585d8f20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80093e0 tx=0x7fb0a80134c0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 shutdown_connections 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x558f58588a10 0x558f585e2c30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f58578750 0x558f585d8f20 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f5856fb70 0x558f5857eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb099ffb640 0 client.16869 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 --2- 172.21.15.42:0/2641865209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f5856f250 0x558f5857e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 >> 172.21.15.42:0/2641865209 conn(0x558f58569980 msgr2=0x558f58575760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 shutdown_connections 2024-03-12T04:19:56.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 1 -- 172.21.15.42:0/2641865209 wait complete. 2024-03-12T04:19:56.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0bb7a2740 20 client.16869 trim_cache size 0 max 16384 2024-03-12T04:19:56.643 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb09c0a1ba0 msgr2=0x7fb09c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb09c0a1ba0 0x7fb09c0a4060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40b99c0 tx=0x7fb0a408f1f0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b419bf50 msgr2=0x7fb0b419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b419bf50 0x7fb0b419c400 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb0a402c700 tx=0x7fb0a4035d20 comp rx=0 tx=0).stop 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 shutdown_connections 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.636+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb09c0a1ba0 0x7fb09c0a4060 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.637+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b419bf50 0x7fb0b419c400 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.637+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b4105330 0x7fb0b4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.637+0000 7fb0b8af4640 1 --2- 172.21.15.42:0/2725041384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0b4100260 0x7fb0b4197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.637+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 >> 172.21.15.42:0/2725041384 conn(0x7fb0b4077180 msgr2=0x7fb0b40fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.637+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 shutdown_connections 2024-03-12T04:19:56.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.637+0000 7fb0b8af4640 1 -- 172.21.15.42:0/2725041384 wait complete. 2024-03-12T04:19:56.650 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:19:56.650 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:19:56.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 -- 172.21.15.42:0/1989229441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88105030 msgr2=0x7faa88105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 --2- 172.21.15.42:0/1989229441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88105030 0x7faa88105410 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7faa78009ca0 tx=0x7faa7802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 -- 172.21.15.42:0/1989229441 shutdown_connections 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 --2- 172.21.15.42:0/1989229441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa88101170 0x7faa8810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 --2- 172.21.15.42:0/1989229441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa881007d0 0x7faa88100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 --2- 172.21.15.42:0/1989229441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88105030 0x7faa88105410 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 -- 172.21.15.42:0/1989229441 >> 172.21.15.42:0/1989229441 conn(0x7faa88076640 msgr2=0x7faa880fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 -- 172.21.15.42:0/1989229441 shutdown_connections 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 -- 172.21.15.42:0/1989229441 wait complete. 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.752+0000 7faa8cafb640 1 Processor -- start 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 -- start start 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa881007d0 0x7faa8819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88101170 0x7faa8819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa88105030 0x7faa881975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88197c00 con 0x7faa88101170 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88197d70 con 0x7faa881007d0 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88197f30 con 0x7faa88105030 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88101170 0x7faa8819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88101170 0x7faa8819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41386/0 (socket says 172.21.15.42:41386) 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 -- 172.21.15.42:0/853439186 learned_addr learned my addr 172.21.15.42:0/853439186 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8659c640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa881007d0 0x7faa8819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 -- 172.21.15.42:0/853439186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa88105030 msgr2=0x7faa881975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa88105030 0x7faa881975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 -- 172.21.15.42:0/853439186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa881007d0 msgr2=0x7faa8819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa881007d0 0x7faa8819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 -- 172.21.15.42:0/853439186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa78009900 con 0x7faa88101170 2024-03-12T04:19:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8659c640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa881007d0 0x7faa8819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa85d9b640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88101170 0x7faa8819d940 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00d950 tx=0x7faa7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa777fe640 1 -- 172.21.15.42:0/853439186 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c009c30 con 0x7faa88101170 2024-03-12T04:19:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa777fe640 1 -- 172.21.15.42:0/853439186 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa7c009d90 con 0x7faa88101170 2024-03-12T04:19:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa88198210 con 0x7faa88101170 2024-03-12T04:19:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa777fe640 1 -- 172.21.15.42:0/853439186 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00f6b0 con 0x7faa88101170 2024-03-12T04:19:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.753+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa881a44a0 con 0x7faa88101170 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.755+0000 7faa777fe640 1 -- 172.21.15.42:0/853439186 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7faa7c010aa0 con 0x7faa88101170 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 Processor -- start 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 -- start start 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5e250 0x5600d0c5e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa777fe640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faa700a1c20 0x7faa700a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5eb70 0x5600d0c67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c69b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0c19960 con 0x5600d0c5eb70 2024-03-12T04:19:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0bcd310 con 0x5600d0c67750 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.756+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0bca840 con 0x5600d0c5e250 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5eb70 0x5600d0c67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa8659c640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faa700a1c20 0x7faa700a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5eb70 0x5600d0c67210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41390/0 (socket says 172.21.15.42:41390) 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 -- 172.21.15.42:0/941369284 learned_addr learned my addr 172.21.15.42:0/941369284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa86d9d640 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5e250 0x5600d0c5e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa777fe640 1 -- 172.21.15.42:0/853439186 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faa7c0cd500 con 0x7faa88101170 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa86d9d640 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c69b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 -- 172.21.15.42:0/941369284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5e250 msgr2=0x5600d0c5e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5e250 0x5600d0c5e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 -- 172.21.15.42:0/941369284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 msgr2=0x5600d0c69b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c69b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 -- 172.21.15.42:0/941369284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600d0b88060 con 0x5600d0c5eb70 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa86d9d640 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c69b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:56.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa8659c640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faa700a1c20 0x7faa700a40e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa78037040 tx=0x7faa78005d80 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa85d9b640 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5eb70 0x5600d0c67210 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faa7c096af0 tx=0x7faa7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1f042d6a1e4304 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa74ff9640 1 -- 172.21.15.42:0/941369284 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0a2220 con 0x5600d0c5eb70 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa74ff9640 1 -- 172.21.15.42:0/941369284 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa7c0a2030 con 0x5600d0c5eb70 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.757+0000 7faa74ff9640 1 -- 172.21.15.42:0/941369284 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c09e2e0 con 0x5600d0c5eb70 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 -- 172.21.15.42:0/941369284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5eb70 msgr2=0x5600d0c67210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5eb70 0x5600d0c67210 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faa7c096af0 tx=0x7faa7c005880 comp rx=0 tx=0).stop 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 -- 172.21.15.42:0/941369284 shutdown_connections 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c69b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5eb70 0x5600d0c67210 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/941369284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5e250 0x5600d0c5e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 -- 172.21.15.42:0/941369284 >> 172.21.15.42:0/941369284 conn(0x5600d0c58980 msgr2=0x5600d0c58d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 -- 172.21.15.42:0/941369284 shutdown_connections 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 -- 172.21.15.42:0/941369284 wait complete. 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 -- start start 2024-03-12T04:19:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.758+0000 7faa8f7ac740 1 Processor -- start 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5eb70 0x5600d0c64d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c67750 0x5600d0c65270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0bc6020 con 0x5600d0c5e250 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0899430 con 0x5600d0c5eb70 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0855e00 con 0x5600d0c67750 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 10 client.0 _ll_register_callbacks cb 0x5600d0b6dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41396/0 (socket says 172.21.15.42:41396) 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 -- 172.21.15.42:0/3404351109 learned_addr learned my addr 172.21.15.42:0/3404351109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 -- 172.21.15.42:0/3404351109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c67750 msgr2=0x5600d0c65270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c67750 0x5600d0c65270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 -- 172.21.15.42:0/3404351109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5eb70 msgr2=0x5600d0c64d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5eb70 0x5600d0c64d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 -- 172.21.15.42:0/3404351109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa881a44a0 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8659c640 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c647f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7faa78036000 tx=0x7faa780095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd71cfbe2037aa25 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa6dffb640 1 -- 172.21.15.42:0/3404351109 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78032a30 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa6dffb640 1 -- 172.21.15.42:0/3404351109 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faa78045040 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.759+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600d08c57e0 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa6dffb640 1 -- 172.21.15.42:0/3404351109 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa78003970 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 10 client.16881 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5600d0bc46a0 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa6dffb640 1 -- 172.21.15.42:0/3404351109 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5600d08c57e0 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa6dffb640 1 -- 172.21.15.42:0/3404351109 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faa78004710 con 0x5600d0c5e250 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 10 client.16881 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 20 client.16881 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0c77b20 0x5600d0c79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 4 client.16881 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600d0902720 con 0x5600d0c77b20 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.760+0000 7faa85d9b640 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0c77b20 0x5600d0c79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa85d9b640 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0c77b20 0x5600d0c79f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69bd58eaf53da9a3 server_cookie=f62a802f67e52adf in_seq=0 out_seq=0 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa6dffb640 10 client.16881 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa6dffb640 1 -- 172.21.15.42:0/3404351109 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600d0902720 con 0x5600d0c77b20 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa6dffb640 10 client.16881 handle_command_reply: tid=0 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa8f7ac740 1 client.16881 shutdown 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 msgr2=0x5600d0c647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c647f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7faa78036000 tx=0x7faa780095c0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.761+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 shutdown_connections 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0c77b20 0x5600d0c79f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c67750 0x5600d0c65270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5eb70 0x5600d0c64d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa6dffb640 0 client.16881 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:56.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/3404351109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c647f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 >> 172.21.15.42:0/3404351109 conn(0x5600d0c58980 msgr2=0x5600d0c5a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 shutdown_connections 2024-03-12T04:19:56.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 1 -- 172.21.15.42:0/3404351109 wait complete. 2024-03-12T04:19:56.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.762+0000 7faa8f7ac740 20 client.16881 trim_cache size 0 max 16384 2024-03-12T04:19:56.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 Processor -- start 2024-03-12T04:19:56.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 -- start start 2024-03-12T04:19:56.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5e250 0x5600d0c66f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 0x5600d0c65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c67750 0x5600d0c61a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0b4b130 con 0x5600d0c67750 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0c65c50 con 0x5600d0c5e250 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.773+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0c599d0 con 0x5600d0c5eb70 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c67750 0x5600d0c61a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c67750 0x5600d0c61a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41402/0 (socket says 172.21.15.42:41402) 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 -- 172.21.15.42:0/2725812028 learned_addr learned my addr 172.21.15.42:0/2725812028 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa8659c640 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5e250 0x5600d0c66f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa85d9b640 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 0x5600d0c65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 -- 172.21.15.42:0/2725812028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 msgr2=0x5600d0c65090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 0x5600d0c65090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 -- 172.21.15.42:0/2725812028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5e250 msgr2=0x5600d0c66f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5e250 0x5600d0c66f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 -- 172.21.15.42:0/2725812028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600d0b60c20 con 0x5600d0c67750 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa86d9d640 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c67750 0x5600d0c61a50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7faa80005e00 tx=0x7faa800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60713a8951b0b1c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa6ffff640 1 -- 172.21.15.42:0/2725812028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80010e90 con 0x5600d0c67750 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa6ffff640 1 -- 172.21.15.42:0/2725812028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa800044d0 con 0x5600d0c67750 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.774+0000 7faa6ffff640 1 -- 172.21.15.42:0/2725812028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa800126b0 con 0x5600d0c67750 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2725812028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c67750 msgr2=0x5600d0c61a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c67750 0x5600d0c61a50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7faa80005e00 tx=0x7faa800107a0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2725812028 shutdown_connections 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c67750 0x5600d0c61a50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 0x5600d0c65090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2725812028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c5e250 0x5600d0c66f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2725812028 >> 172.21.15.42:0/2725812028 conn(0x5600d0c58980 msgr2=0x5600d0c5fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2725812028 shutdown_connections 2024-03-12T04:19:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2725812028 wait complete. 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- start start 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 Processor -- start 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c6d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 0x5600d0c6d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c6dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0cce130 con 0x5600d0c5e250 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0cce2a0 con 0x5600d0c67750 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d0cce460 con 0x5600d0c5eb70 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8f7ac740 10 client.0 _ll_register_callbacks cb 0x7faa7c095d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c6d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c6d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41404/0 (socket says 172.21.15.42:41404) 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa8659c640 1 -- 172.21.15.42:0/2714678634 learned_addr learned my addr 172.21.15.42:0/2714678634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.775+0000 7faa86d9d640 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c6dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8659c640 1 -- 172.21.15.42:0/2714678634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 msgr2=0x5600d0c6d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8659c640 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 0x5600d0c6d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8659c640 1 -- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 msgr2=0x5600d0c6dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8659c640 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c6dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8659c640 1 -- 172.21.15.42:0/2714678634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8000e9e0 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8659c640 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c6d230 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faa7802c860 tx=0x7faa78040950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb64444119ec5ee8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa6dffb640 1 -- 172.21.15.42:0/2714678634 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78040cd0 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa6dffb640 1 -- 172.21.15.42:0/2714678634 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faa78040e30 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600d0c540f0 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa6dffb640 1 -- 172.21.15.42:0/2714678634 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa78032ac0 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 10 client.16887 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5600d0b868c0 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa6dffb640 1 -- 172.21.15.42:0/2714678634 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5600d0c540f0 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa6dffb640 1 -- 172.21.15.42:0/2714678634 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faa78053630 con 0x5600d0c5e250 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 10 client.16887 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 20 client.16887 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0ccf8e0 0x5600d0cd1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 4 client.16887 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.776+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7faa7c0a2030 con 0x5600d0ccf8e0 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa85d9b640 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0ccf8e0 0x5600d0cd1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa85d9b640 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0ccf8e0 0x5600d0cd1d00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caec317c859c6420 server_cookie=119741197c093191 in_seq=0 out_seq=0 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa6dffb640 10 client.16887 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa6dffb640 1 -- 172.21.15.42:0/2714678634 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7faa7c0a2030 con 0x5600d0ccf8e0 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa6dffb640 10 client.16887 handle_command_reply: tid=0 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 client.16887 shutdown 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 msgr2=0x5600d0c6d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c6d230 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faa7802c860 tx=0x7faa78040950 comp rx=0 tx=0).stop 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 shutdown_connections 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5600d0ccf8e0 0x5600d0cd1d00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600d0c67750 0x5600d0c6dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600d0c5eb70 0x5600d0c6d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa6dffb640 0 client.16887 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 --2- 172.21.15.42:0/2714678634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600d0c5e250 0x5600d0c6d230 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.777+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 >> 172.21.15.42:0/2714678634 conn(0x5600d0c58980 msgr2=0x5600d0c572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 shutdown_connections 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8f7ac740 1 -- 172.21.15.42:0/2714678634 wait complete. 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8f7ac740 20 client.16887 trim_cache size 0 max 16384 2024-03-12T04:19:56.787 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faa700a1c20 msgr2=0x7faa700a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faa700a1c20 0x7faa700a40e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa78037040 tx=0x7faa78005d80 comp rx=0 tx=0).stop 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88101170 msgr2=0x7faa8819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88101170 0x7faa8819d940 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00d950 tx=0x7faa7c00de10 comp rx=0 tx=0).stop 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 shutdown_connections 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faa700a1c20 0x7faa700a40e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa88105030 0x7faa881975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa88101170 0x7faa8819d940 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 --2- 172.21.15.42:0/853439186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa881007d0 0x7faa8819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:56.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 >> 172.21.15.42:0/853439186 conn(0x7faa88076640 msgr2=0x7faa88103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:56.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 shutdown_connections 2024-03-12T04:19:56.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:56.778+0000 7faa8cafb640 1 -- 172.21.15.42:0/853439186 wait complete. 2024-03-12T04:19:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:56 smithi119 ceph-mon[36119]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:56.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:56 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/623067393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:19:56.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:56 smithi136 ceph-mon[35571]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:56.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:56 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/623067393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:19:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:56 smithi042 ceph-mon[32260]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:19:57.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:56 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/623067393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:19:57.792 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:19:57.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:19:57.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:19:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 -- 172.21.15.42:0/596956808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c101390 msgr2=0x7f3c0c101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 --2- 172.21.15.42:0/596956808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c101390 0x7f3c0c101770 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009930 tx=0x7f3bfc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 -- 172.21.15.42:0/596956808 shutdown_connections 2024-03-12T04:19:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 --2- 172.21.15.42:0/596956808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c0c106460 0x7f3c0c10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 --2- 172.21.15.42:0/596956808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c0c101d40 0x7f3c0c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 --2- 172.21.15.42:0/596956808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c101390 0x7f3c0c101770 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 -- 172.21.15.42:0/596956808 >> 172.21.15.42:0/596956808 conn(0x7f3c0c0fcbb0 msgr2=0x7f3c0c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 -- 172.21.15.42:0/596956808 shutdown_connections 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.908+0000 7f3c1193e640 1 -- 172.21.15.42:0/596956808 wait complete. 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 Processor -- start 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 -- start start 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c0c101390 0x7f3c0c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c106460 0x7f3c0c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c0c1975b0 0x7f3c0c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c197fa0 con 0x7f3c0c106460 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c198110 con 0x7f3c0c1975b0 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c1193e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c1982d0 con 0x7f3c0c101390 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c106460 0x7f3c0c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c106460 0x7f3c0c19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41432/0 (socket says 172.21.15.42:41432) 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c03fff640 1 -- 172.21.15.42:0/1528421368 learned_addr learned my addr 172.21.15.42:0/1528421368 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c0c1975b0 0x7f3c0c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.909+0000 7f3c0affd640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c0c101390 0x7f3c0c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c03fff640 1 -- 172.21.15.42:0/1528421368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c0c101390 msgr2=0x7f3c0c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c03fff640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c0c101390 0x7f3c0c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c03fff640 1 -- 172.21.15.42:0/1528421368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c0c1975b0 msgr2=0x7f3c0c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c03fff640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c0c1975b0 0x7f3c0c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c03fff640 1 -- 172.21.15.42:0/1528421368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc009560 con 0x7f3c0c106460 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c03fff640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c106460 0x7f3c0c19d940 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c970 tx=0x7f3bf400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c1093c640 1 -- 172.21.15.42:0/1528421368 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4007bb0 con 0x7f3c0c106460 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c1093c640 1 -- 172.21.15.42:0/1528421368 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3bf4007d10 con 0x7f3c0c106460 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c1a42c0 con 0x7f3c0c106460 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c1093c640 1 -- 172.21.15.42:0/1528421368 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4018660 con 0x7f3c0c106460 2024-03-12T04:19:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.910+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c1a4780 con 0x7f3c0c106460 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.912+0000 7f3c1093c640 1 -- 172.21.15.42:0/1528421368 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3bf400f450 con 0x7f3c0c106460 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.912+0000 7f3c145ec740 1 Processor -- start 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c145ec740 1 -- start start 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82a3a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82a476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c145ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e829a8f90 con 0x559e82a3a100 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c145ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e829a64c0 con 0x559e82a3aab0 2024-03-12T04:19:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c145ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e829a1ca0 con 0x559e82a47cc0 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c1093c640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bf80a1ba0 0x7f3bf80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82a3a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82a3a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41444/0 (socket says 172.21.15.42:41444) 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 -- 172.21.15.42:0/1500349958 learned_addr learned my addr 172.21.15.42:0/1500349958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0affd640 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82a476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bf80a1ba0 0x7f3bf80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c1093c640 1 -- 172.21.15.42:0/1528421368 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3bf40cd8a0 con 0x7f3c0c106460 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c03fff640 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 -- 172.21.15.42:0/1500349958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 msgr2=0x559e82a4a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 -- 172.21.15.42:0/1500349958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 msgr2=0x559e82a476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82a476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 -- 172.21.15.42:0/1500349958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e829640b0 con 0x559e82a3a100 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0affd640 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82a476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.913+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82a3a4e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3c04013940 tx=0x7f3c04013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9d25500f8714e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c01ffb640 1 -- 172.21.15.42:0/1500349958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04003ec0 con 0x559e82a3a100 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c01ffb640 1 -- 172.21.15.42:0/1500349958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c04019070 con 0x559e82a3a100 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c01ffb640 1 -- 172.21.15.42:0/1500349958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0401d780 con 0x559e82a3a100 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 -- 172.21.15.42:0/1500349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 msgr2=0x559e82a3a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bf80a1ba0 0x7f3bf80a4060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c04021000 tx=0x7f3c04020000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82a3a4e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3c04013940 tx=0x7f3c04013e00 comp rx=0 tx=0).stop 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 -- 172.21.15.42:0/1500349958 shutdown_connections 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82a476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 --2- 172.21.15.42:0/1500349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82a3a4e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 -- 172.21.15.42:0/1500349958 >> 172.21.15.42:0/1500349958 conn(0x559e82a34840 msgr2=0x559e82a34c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.914+0000 7f3c145ec740 1 -- 172.21.15.42:0/1500349958 shutdown_connections 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 -- 172.21.15.42:0/1500349958 wait complete. 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 -- start start 2024-03-12T04:19:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 Processor -- start 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a44c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a45700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82946f40 con 0x559e82a3aab0 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e826754b0 con 0x559e82a3a100 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82927670 con 0x559e82a47cc0 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c145ec740 10 client.0 _ll_register_callbacks cb 0x559e82994f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41446/0 (socket says 172.21.15.42:41446) 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.915+0000 7f3c03fff640 1 -- 172.21.15.42:0/3954419475 learned_addr learned my addr 172.21.15.42:0/3954419475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a45700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c0affd640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a44c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c03fff640 1 -- 172.21.15.42:0/3954419475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 msgr2=0x559e82a45700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c03fff640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a45700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c03fff640 1 -- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 msgr2=0x559e82a44c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c03fff640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a44c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c03fff640 1 -- 172.21.15.42:0/3954419475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c04013580 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c0affd640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a44c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c03fff640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a451c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c940 tx=0x7f3bf409a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fcb7df6324dc49 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3beaffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3beaffd640 1 -- 172.21.15.42:0/3954419475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40b2070 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3beaffd640 1 -- 172.21.15.42:0/3954419475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3bf40ba0b0 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e8297bf40 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3beaffd640 1 -- 172.21.15.42:0/3954419475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bf409a4c0 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c145ec740 10 client.16899 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559e829a0320 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.916+0000 7f3beaffd640 1 -- 172.21.15.42:0/3954419475 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x559e8297bf40 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3beaffd640 1 -- 172.21.15.42:0/3954419475 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3bf4016020 con 0x559e82a3aab0 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c145ec740 10 client.16899 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c145ec740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c145ec740 20 client.16899 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82a58120 0x559e82a5a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c145ec740 4 client.16899 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e82998310 con 0x559e82a58120 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c0affd640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82a58120 0x559e82a5a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3beaffd640 10 client.16899 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.917+0000 7f3c0affd640 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82a58120 0x559e82a5a540 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc34757e7f80afd server_cookie=9fd455d8db514b37 in_seq=0 out_seq=0 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3beaffd640 1 -- 172.21.15.42:0/3954419475 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e82998310 con 0x559e82a58120 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3beaffd640 10 client.16899 handle_command_reply: tid=0 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 client.16899 shutdown 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 msgr2=0x559e82a451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a451c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c940 tx=0x7f3bf409a0c0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 shutdown_connections 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82a58120 0x559e82a5a540 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a45700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a451c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3beaffd640 0 client.16899 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:57.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3954419475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a44c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 >> 172.21.15.42:0/3954419475 conn(0x559e82a34840 msgr2=0x559e82a367a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:57.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 shutdown_connections 2024-03-12T04:19:57.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 1 -- 172.21.15.42:0/3954419475 wait complete. 2024-03-12T04:19:57.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.918+0000 7f3c145ec740 20 client.16899 trim_cache size 0 max 16384 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 Processor -- start 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 -- start start 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a45520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a46ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82a2eda0 con 0x559e82a3aab0 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82a39660 con 0x559e82a3a100 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c145ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82a3b0c0 con 0x559e82a47cc0 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a46ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a46ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41460/0 (socket says 172.21.15.42:41460) 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 -- 172.21.15.42:0/2030737688 learned_addr learned my addr 172.21.15.42:0/2030737688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 -- 172.21.15.42:0/2030737688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 msgr2=0x559e82a4ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 -- 172.21.15.42:0/2030737688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 msgr2=0x559e82a45520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a45520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 -- 172.21.15.42:0/2030737688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e8293cc00 con 0x559e82a3aab0 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.934+0000 7f3c03fff640 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a46ab0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3bf409a1c0 tx=0x7f3bf40989d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba0b2c317efd022 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c017fa640 1 -- 172.21.15.42:0/2030737688 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40b2070 con 0x559e82a3aab0 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c017fa640 1 -- 172.21.15.42:0/2030737688 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3bf40cb260 con 0x559e82a3aab0 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c017fa640 1 -- 172.21.15.42:0/2030737688 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf409e3d0 con 0x559e82a3aab0 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 -- 172.21.15.42:0/2030737688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 msgr2=0x559e82a46ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a46ab0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3bf409a1c0 tx=0x7f3bf40989d0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 -- 172.21.15.42:0/2030737688 shutdown_connections 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3aab0 0x559e82a46ab0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 --2- 172.21.15.42:0/2030737688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3a100 0x559e82a45520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 -- 172.21.15.42:0/2030737688 >> 172.21.15.42:0/2030737688 conn(0x559e82a34840 msgr2=0x559e82a463a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 -- 172.21.15.42:0/2030737688 shutdown_connections 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.935+0000 7f3c145ec740 1 -- 172.21.15.42:0/2030737688 wait complete. 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 -- start start 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 Processor -- start 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82aa79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82aa7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82a4e5f0 con 0x559e82a3a100 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82a4e760 con 0x559e82a3aab0 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e82a4e920 con 0x559e82a47cc0 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c145ec740 10 client.0 _ll_register_callbacks cb 0x7f3c0400a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c0affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82aa79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c0affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82aa79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41464/0 (socket says 172.21.15.42:41464) 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c0affd640 1 -- 172.21.15.42:0/3127514346 learned_addr learned my addr 172.21.15.42:0/3127514346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:57.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c03fff640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82aa7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.936+0000 7f3c0affd640 1 -- 172.21.15.42:0/3127514346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 msgr2=0x559e82a4df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c0affd640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c0affd640 1 -- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 msgr2=0x559e82aa7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c0affd640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82aa7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c0affd640 1 -- 172.21.15.42:0/3127514346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e829a0320 con 0x559e82a3a100 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c0b7fe640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c0affd640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82aa79e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc004c20 tx=0x7f3bfc02c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9090af19536f09b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3beaffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3beaffd640 1 -- 172.21.15.42:0/3127514346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc047df0 con 0x559e82a3a100 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3beaffd640 1 -- 172.21.15.42:0/3127514346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3bfc04b030 con 0x559e82a3a100 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e82a351c0 con 0x559e82a3a100 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3beaffd640 1 -- 172.21.15.42:0/3127514346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bfc0040c0 con 0x559e82a3a100 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 10 client.16905 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559e82962910 con 0x559e82a3a100 2024-03-12T04:19:57.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3beaffd640 1 -- 172.21.15.42:0/3127514346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x559e82a351c0 con 0x559e82a3a100 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3beaffd640 1 -- 172.21.15.42:0/3127514346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3bfc03c040 con 0x559e82a3a100 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 10 client.16905 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 20 client.16905 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82ab0860 0x559e82ab2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 4 client.16905 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.937+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559e829c7900 con 0x559e82ab0860 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.938+0000 7f3c03fff640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82ab0860 0x559e82ab2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.938+0000 7f3beaffd640 10 client.16905 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:57.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.938+0000 7f3c03fff640 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82ab0860 0x559e82ab2c80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3302555e705cc070 server_cookie=898f9b7f4017485b in_seq=0 out_seq=0 2024-03-12T04:19:57.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.939+0000 7f3beaffd640 1 -- 172.21.15.42:0/3127514346 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559e829c7900 con 0x559e82ab0860 2024-03-12T04:19:57.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.939+0000 7f3beaffd640 10 client.16905 handle_command_reply: tid=0 2024-03-12T04:19:57.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.939+0000 7f3c145ec740 1 client.16905 shutdown 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 msgr2=0x559e82aa79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82aa79e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc004c20 tx=0x7f3bfc02c3c0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 shutdown_connections 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x559e82ab0860 0x559e82ab2c80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e82a47cc0 0x559e82a4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e82a3aab0 0x559e82aa7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 --2- 172.21.15.42:0/3127514346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e82a3a100 0x559e82aa79e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3beaffd640 0 client.16905 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 >> 172.21.15.42:0/3127514346 conn(0x559e82a34840 msgr2=0x559e82a338e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 shutdown_connections 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 1 -- 172.21.15.42:0/3127514346 wait complete. 2024-03-12T04:19:57.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c145ec740 20 client.16905 trim_cache size 0 max 16384 2024-03-12T04:19:57.948 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:19:57.948 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:19:57.948 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "4aaaffb5-b764-4c98-8c92-d24fcff005c5", 2024-03-12T04:19:57.948 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:19:57.948 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bf80a1ba0 msgr2=0x7f3bf80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bf80a1ba0 0x7f3bf80a4060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c04021000 tx=0x7f3c04020000 comp rx=0 tx=0).stop 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c106460 msgr2=0x7f3c0c19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c106460 0x7f3c0c19d940 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c970 tx=0x7f3bf400ce30 comp rx=0 tx=0).stop 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 shutdown_connections 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bf80a1ba0 0x7f3bf80a4060 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c0c1975b0 0x7f3c0c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c0c106460 0x7f3c0c19d940 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.940+0000 7f3c1193e640 1 --2- 172.21.15.42:0/1528421368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c0c101390 0x7f3c0c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.941+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 >> 172.21.15.42:0/1528421368 conn(0x7f3c0c0fcbb0 msgr2=0x7f3c0c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.941+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 shutdown_connections 2024-03-12T04:19:57.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:57.941+0000 7f3c1193e640 1 -- 172.21.15.42:0/1528421368 wait complete. 2024-03-12T04:19:57.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:19:57.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:19:58.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 -- 172.21.15.42:0/2811477437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14101380 msgr2=0x7f3c14101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2811477437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14101380 0x7f3c14101760 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009930 tx=0x7f3c0c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:19:58.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 -- 172.21.15.42:0/2811477437 shutdown_connections 2024-03-12T04:19:58.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2811477437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c14106450 0x7f3c1410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2811477437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c14101d30 0x7f3c14105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2811477437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14101380 0x7f3c14101760 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 -- 172.21.15.42:0/2811477437 >> 172.21.15.42:0/2811477437 conn(0x7f3c140fcc20 msgr2=0x7f3c140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 -- 172.21.15.42:0/2811477437 shutdown_connections 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.056+0000 7f3c18de9640 1 -- 172.21.15.42:0/2811477437 wait complete. 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 Processor -- start 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 -- start start 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c14101380 0x7f3c14073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14106450 0x7f3c140739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c14075120 0x7f3c14073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141aa3e0 con 0x7f3c14106450 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141aa550 con 0x7f3c14101380 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c18de9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141aa710 con 0x7f3c14075120 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c11d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14106450 0x7f3c140739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c11d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14106450 0x7f3c140739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41492/0 (socket says 172.21.15.42:41492) 2024-03-12T04:19:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.057+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2827790765 learned_addr learned my addr 172.21.15.42:0/2827790765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c1259c640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c14101380 0x7f3c14073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c14075120 0x7f3c14073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2827790765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c14075120 msgr2=0x7f3c14073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c14075120 0x7f3c14073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2827790765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c14101380 msgr2=0x7f3c14073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c14101380 0x7f3c14073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2827790765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c009560 con 0x7f3c14106450 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c1259c640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c14101380 0x7f3c14073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14106450 0x7f3c140739c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c970 tx=0x7f3c0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c037fe640 1 -- 172.21.15.42:0/2827790765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08007bb0 con 0x7f3c14106450 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c037fe640 1 -- 172.21.15.42:0/2827790765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c08007d10 con 0x7f3c14106450 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c141aa9f0 con 0x7f3c14106450 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c037fe640 1 -- 172.21.15.42:0/2827790765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08018660 con 0x7f3c14106450 2024-03-12T04:19:58.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.058+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c141aaeb0 con 0x7f3c14106450 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.060+0000 7f3c037fe640 1 -- 172.21.15.42:0/2827790765 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3c0800f450 con 0x7f3c14106450 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 Processor -- start 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 -- start start 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c037fe640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bfc0a1ba0 0x7f3bfc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173cebd0 0x560a173d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c037fe640 1 -- 172.21.15.42:0/2827790765 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3c080cd9f0 con 0x7f3c14106450 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bfc0a1ba0 0x7f3bfc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1733d280 con 0x560a173d77b0 2024-03-12T04:19:58.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1733a7b0 con 0x560a173ce2b0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a17335f90 con 0x560a173cebd0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c12d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c12d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41496/0 (socket says 172.21.15.42:41496) 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c12d9d640 1 -- 172.21.15.42:0/281602974 learned_addr learned my addr 172.21.15.42:0/281602974 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c1259c640 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.061+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173cebd0 0x560a173d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bfc0a1ba0 0x7f3bfc0a4060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3c04015690 tx=0x7f3c04015a90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c12d9d640 1 -- 172.21.15.42:0/281602974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173cebd0 msgr2=0x560a173d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173cebd0 0x560a173d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c12d9d640 1 -- 172.21.15.42:0/281602974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 msgr2=0x560a173ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c12d9d640 1 -- 172.21.15.42:0/281602974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a172f4570 con 0x560a173d77b0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c1259c640 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d9ba0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3c04012800 tx=0x7f3c0401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7360a243cd754f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c00ff9640 1 -- 172.21.15.42:0/281602974 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04015230 con 0x560a173d77b0 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c00ff9640 1 -- 172.21.15.42:0/281602974 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c04012e40 con 0x560a173d77b0 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.062+0000 7f3c00ff9640 1 -- 172.21.15.42:0/281602974 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04005280 con 0x560a173d77b0 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/281602974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 msgr2=0x560a173d9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d9ba0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3c04012800 tx=0x7f3c0401c000 comp rx=0 tx=0).stop 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/281602974 shutdown_connections 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d9ba0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173cebd0 0x560a173d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/281602974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/281602974 >> 172.21.15.42:0/281602974 conn(0x560a173c8990 msgr2=0x560a173c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/281602974 shutdown_connections 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/281602974 wait complete. 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 -- start start 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 Processor -- start 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.063+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a173d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a17009430 con 0x560a173cebd0 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a16fc5e00 con 0x560a173ce2b0 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a172bafd0 con 0x560a173d77b0 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1b9c4740 10 client.0 _ll_register_callbacks cb 0x560a17310fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173d45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41512/0 (socket says 172.21.15.42:41512) 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2886854420 learned_addr learned my addr 172.21.15.42:0/2886854420 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a173d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1259c640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2886854420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 msgr2=0x560a173d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a173d4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 msgr2=0x560a173d7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173d7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 -- 172.21.15.42:0/2886854420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0401b070 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a173d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173d45c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c940 tx=0x7f3c0809a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c49d22673e1d7e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3bf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/2886854420 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08083ea0 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/2886854420 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3c080a2e10 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a170357e0 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/2886854420 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c0800f990 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1b9c4740 10 client.16917 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.064+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560a17334610 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/2886854420 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c0807d080 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/2886854420 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c08016020 con 0x560a173cebd0 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1b9c4740 10 client.16917 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1b9c4740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1b9c4740 20 client.16917 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a173e79d0 0x560a173e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1b9c4740 4 client.16917 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a17072720 con 0x560a173e79d0 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1259c640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a173e79d0 0x560a173e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3c1259c640 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a173e79d0 0x560a173e9dc0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7460a19516cc10e server_cookie=19711952baa726be in_seq=0 out_seq=0 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.065+0000 7f3bf9ffb640 10 client.16917 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/2886854420 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a17072720 con 0x560a173e79d0 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3bf9ffb640 10 client.16917 handle_command_reply: tid=0 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 client.16917 shutdown 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 msgr2=0x560a173d45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173d45c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c940 tx=0x7f3c0809a1b0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 shutdown_connections 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a173e79d0 0x560a173e9dc0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a173d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173d45c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/2886854420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173d7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3bf9ffb640 0 client.16917 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 >> 172.21.15.42:0/2886854420 conn(0x560a173c8990 msgr2=0x560a173ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 shutdown_connections 2024-03-12T04:19:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/2886854420 wait complete. 2024-03-12T04:19:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.066+0000 7f3c1b9c4740 20 client.16917 trim_cache size 0 max 16384 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 Processor -- start 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 -- start start 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173ce2b0 0x560a173d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173cebd0 0x560a173d6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a173c99e0 con 0x560a173d77b0 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a173d55d0 con 0x560a173cebd0 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.081+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a173cda50 con 0x560a173ce2b0 2024-03-12T04:19:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41516/0 (socket says 172.21.15.42:41516) 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 -- 172.21.15.42:0/1857613130 learned_addr learned my addr 172.21.15.42:0/1857613130 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173cebd0 0x560a173d6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 -- 172.21.15.42:0/1857613130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173ce2b0 msgr2=0x560a173d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173ce2b0 0x560a173d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 -- 172.21.15.42:0/1857613130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173cebd0 msgr2=0x560a173d6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173cebd0 0x560a173d6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 -- 172.21.15.42:0/1857613130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a172d0ac0 con 0x560a173d77b0 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3c12d9d640 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d7230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009370 tx=0x7f3c040097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187b10848d06ee90 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3bfbfff640 1 -- 172.21.15.42:0/1857613130 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040150c0 con 0x560a173d77b0 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3bfbfff640 1 -- 172.21.15.42:0/1857613130 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c04015220 con 0x560a173d77b0 2024-03-12T04:19:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.082+0000 7f3bfbfff640 1 -- 172.21.15.42:0/1857613130 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04005040 con 0x560a173d77b0 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/1857613130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 msgr2=0x560a173d7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d7230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009370 tx=0x7f3c040097c0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/1857613130 shutdown_connections 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173d77b0 0x560a173d7230 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173cebd0 0x560a173d6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/1857613130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173ce2b0 0x560a173d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/1857613130 >> 172.21.15.42:0/1857613130 conn(0x560a173c8990 msgr2=0x560a173d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/1857613130 shutdown_connections 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/1857613130 wait complete. 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 -- start start 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 Processor -- start 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.083+0000 7f3c1b9c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a17437f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a174384b0 con 0x560a173cebd0 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a17438620 con 0x560a173ce2b0 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1b9c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a174387e0 con 0x560a173d77b0 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1b9c4740 10 client.0 _ll_register_callbacks cb 0x7f3c04009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173dbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41518/0 (socket says 172.21.15.42:41518) 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 -- 172.21.15.42:0/3696468122 learned_addr learned my addr 172.21.15.42:0/3696468122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1259c640 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 -- 172.21.15.42:0/3696468122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 msgr2=0x560a17437f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a17437f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 -- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 msgr2=0x560a173db8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173db8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 -- 172.21.15.42:0/3696468122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a172f4570 con 0x560a173cebd0 2024-03-12T04:19:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c11d9b640 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173dbdf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3c080989d0 tx=0x7f3c0809e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cac3cdaed0728f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3bf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/3696468122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08083ea0 con 0x560a173cebd0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/3696468122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3c080a2e10 con 0x560a173cebd0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c04012e40 con 0x560a173cebd0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/3696468122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c080969c0 con 0x560a173cebd0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1b9c4740 10 client.16923 fetch_fsmap learned FSMap version 27 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560a172f66a0 con 0x560a173cebd0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.084+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/3696468122 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c04012e40 con 0x560a173cebd0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/3696468122 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c080995e0 con 0x560a173cebd0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1b9c4740 10 client.16923 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1b9c4740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1b9c4740 20 client.16923 populate_metadata read hostname 'smithi042' 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a1743fcc0 0x560a174420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1b9c4740 4 client.16923 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560a170c1890 con 0x560a1743fcc0 2024-03-12T04:19:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1259c640 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a1743fcc0 0x560a174420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3bf9ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3c1259c640 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a1743fcc0 0x560a174420b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebc6a360cd15ae8 server_cookie=5913041dfa9beca3 in_seq=0 out_seq=0 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3bf9ffb640 1 -- 172.21.15.42:0/3696468122 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560a170c1890 con 0x560a1743fcc0 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.085+0000 7f3bf9ffb640 10 client.16923 handle_command_reply: tid=0 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 client.16923 shutdown 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 msgr2=0x560a173dbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173dbdf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3c080989d0 tx=0x7f3c0809e940 comp rx=0 tx=0).stop 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 shutdown_connections 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560a1743fcc0 0x560a174420b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a173d77b0 0x560a17437f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3bf9ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560a173cebd0 0x560a173dbdf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 --2- 172.21.15.42:0/3696468122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a173ce2b0 0x560a173db8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 >> 172.21.15.42:0/3696468122 conn(0x560a173c8990 msgr2=0x560a173d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:58.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 shutdown_connections 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 1 -- 172.21.15.42:0/3696468122 wait complete. 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c1b9c4740 20 client.16923 trim_cache size 0 max 16384 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: "4aaaffb5-b764-4c98-8c92-d24fcff005c5": { 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "4aaaffb5-b764-4c98-8c92-d24fcff005c5", 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:19:58.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:19:58.096 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:19:58.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bfc0a1ba0 msgr2=0x7f3bfc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bfc0a1ba0 0x7f3bfc0a4060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3c04015690 tx=0x7f3c04015a90 comp rx=0 tx=0).stop 2024-03-12T04:19:58.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14106450 msgr2=0x7f3c140739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14106450 0x7f3c140739c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c970 tx=0x7f3c0800ce30 comp rx=0 tx=0).stop 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 shutdown_connections 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3bfc0a1ba0 0x7f3bfc0a4060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c14075120 0x7f3c14073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c14106450 0x7f3c140739c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 --2- 172.21.15.42:0/2827790765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c14101380 0x7f3c14073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 >> 172.21.15.42:0/2827790765 conn(0x7f3c140fcc20 msgr2=0x7f3c140fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 shutdown_connections 2024-03-12T04:19:58.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:19:58.086+0000 7f3c18de9640 1 -- 172.21.15.42:0/2827790765 wait complete. 2024-03-12T04:19:58.099 INFO:tasks.cephfs.filesystem:scrub status for tag:4aaaffb5-b764-4c98-8c92-d24fcff005c5 - {'path': '/', 'tag': '4aaaffb5-b764-4c98-8c92-d24fcff005c5', 'options': 'recursive,force'} 2024-03-12T04:19:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:58 smithi119 ceph-mon[36119]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:58 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:19:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:58 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:58 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:19:58.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:58 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:58.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:58 smithi136 ceph-mon[35571]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:58.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:58 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:19:58.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:58 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:58.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:58 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:19:58.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:58 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:59.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:58 smithi042 ceph-mon[32260]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:19:59.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:58 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:19:59.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:58 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:59.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:58 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:19:59.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:58 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:19:59.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:19:59 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:19:59.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:19:59 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:20:00.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:19:59 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:20:00.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:00 smithi119 ceph-mon[36119]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:00.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:00 smithi119 ceph-mon[36119]: overall HEALTH_OK 2024-03-12T04:20:00.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:00 smithi136 ceph-mon[35571]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:00.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:00 smithi136 ceph-mon[35571]: overall HEALTH_OK 2024-03-12T04:20:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:00 smithi042 ceph-mon[32260]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:01.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:00 smithi042 ceph-mon[32260]: overall HEALTH_OK 2024-03-12T04:20:02.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:02 smithi119 ceph-mon[36119]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:02.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:02 smithi136 ceph-mon[35571]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:03.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:02 smithi042 ceph-mon[32260]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:04.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:04 smithi119 ceph-mon[36119]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:04.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:04 smithi136 ceph-mon[35571]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:05.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:04 smithi042 ceph-mon[32260]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:06.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:06 smithi119 ceph-mon[36119]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:06.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:06 smithi136 ceph-mon[35571]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:07.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:06 smithi042 ceph-mon[32260]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:08.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:08 smithi119 ceph-mon[36119]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:08.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:08 smithi136 ceph-mon[35571]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:09.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:08 smithi042 ceph-mon[32260]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:10 smithi119 ceph-mon[36119]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:10 smithi136 ceph-mon[35571]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:11.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:10 smithi042 ceph-mon[32260]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:12.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:12 smithi119 ceph-mon[36119]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:12.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:12 smithi136 ceph-mon[35571]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:13.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:12 smithi042 ceph-mon[32260]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:20:14.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:14 smithi119 ceph-mon[36119]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:14 smithi136 ceph-mon[35571]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:15.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:14 smithi042 ceph-mon[32260]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:20:16.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:16 smithi119 ceph-mon[36119]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:20:16.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:16 smithi136 ceph-mon[35571]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:20:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:16 smithi042 ceph-mon[32260]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:20:18.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:18 smithi119 ceph-mon[36119]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:20:18.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:18 smithi136 ceph-mon[35571]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:20:19.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:18 smithi042 ceph-mon[32260]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:20:20.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:20 smithi119 ceph-mon[36119]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:20:20.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:20 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:20:20.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:20 smithi136 ceph-mon[35571]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:20:20.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:20 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:20:21.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:20 smithi042 ceph-mon[32260]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:20:21.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:20 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:20:21.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:21 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:20:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:21 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:20:21.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:21 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:20:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:21 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:20:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:21 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:20:21.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:21 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:20:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:21 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:20:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:21 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:20:22.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:21 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:20:22.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:22 smithi119 ceph-mon[36119]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:20:22.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:22 smithi136 ceph-mon[35571]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:20:23.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:22 smithi042 ceph-mon[32260]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:20:24.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:24 smithi119 ceph-mon[36119]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:20:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:24 smithi136 ceph-mon[35571]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:20:25.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:24 smithi042 ceph-mon[32260]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:20:26.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:26 smithi119 ceph-mon[36119]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:26.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:26 smithi136 ceph-mon[35571]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:27.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:26 smithi042 ceph-mon[32260]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:28.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:20:28.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:20:28.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.216+0000 7fd2451e9640 1 -- 172.21.15.42:0/3870900365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100150 msgr2=0x7fd24010c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.216+0000 7fd2451e9640 1 --2- 172.21.15.42:0/3870900365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100150 0x7fd24010c660 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd238009ca0 tx=0x7fd23802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.216+0000 7fd2451e9640 1 -- 172.21.15.42:0/3870900365 shutdown_connections 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.216+0000 7fd2451e9640 1 --2- 172.21.15.42:0/3870900365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100150 0x7fd24010c660 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.216+0000 7fd2451e9640 1 --2- 172.21.15.42:0/3870900365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2400ff790 0x7fd2400ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.216+0000 7fd2451e9640 1 --2- 172.21.15.42:0/3870900365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2401057a0 0x7fd240105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.216+0000 7fd2451e9640 1 -- 172.21.15.42:0/3870900365 >> 172.21.15.42:0/3870900365 conn(0x7fd2400765b0 msgr2=0x7fd2400fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.217+0000 7fd2451e9640 1 -- 172.21.15.42:0/3870900365 shutdown_connections 2024-03-12T04:20:28.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.217+0000 7fd2451e9640 1 -- 172.21.15.42:0/3870900365 wait complete. 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.217+0000 7fd2451e9640 1 Processor -- start 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd2451e9640 1 -- start start 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd2451e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2400ff790 0x7fd2401975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd2451e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd240100150 0x7fd240197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd2451e9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2401057a0 0x7fd24019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd2451e9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019c470 con 0x7fd2400ff790 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd2451e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019c5e0 con 0x7fd240100150 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd2451e9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019c7a0 con 0x7fd2401057a0 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2400ff790 0x7fd2401975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2400ff790 0x7fd2401975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50734/0 (socket says 172.21.15.42:50734) 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 -- 172.21.15.42:0/839690956 learned_addr learned my addr 172.21.15.42:0/839690956 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23e59c640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd240100150 0x7fd240197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23f59e640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2401057a0 0x7fd24019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 -- 172.21.15.42:0/839690956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2401057a0 msgr2=0x7fd24019be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2401057a0 0x7fd24019be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 -- 172.21.15.42:0/839690956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd240100150 msgr2=0x7fd240197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd240100150 0x7fd240197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 -- 172.21.15.42:0/839690956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238009900 con 0x7fd2400ff790 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23e59c640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd240100150 0x7fd240197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.218+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2400ff790 0x7fd2401975b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c920 tx=0x7fd23000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.219+0000 7fd227fff640 1 -- 172.21.15.42:0/839690956 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230007c70 con 0x7fd2400ff790 2024-03-12T04:20:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.219+0000 7fd227fff640 1 -- 172.21.15.42:0/839690956 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd230007dd0 con 0x7fd2400ff790 2024-03-12T04:20:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.219+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24019ca80 con 0x7fd2400ff790 2024-03-12T04:20:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.219+0000 7fd227fff640 1 -- 172.21.15.42:0/839690956 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300187c0 con 0x7fd2400ff790 2024-03-12T04:20:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.219+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2401a4300 con 0x7fd2400ff790 2024-03-12T04:20:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.221+0000 7fd227fff640 1 -- 172.21.15.42:0/839690956 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd23000f450 con 0x7fd2400ff790 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.221+0000 7fd247dbe740 1 Processor -- start 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.221+0000 7fd247dbe740 1 -- start start 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a714690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd247dbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a683280 con 0x55bd6a714bd0 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd247dbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a6807b0 con 0x55bd6a71d7b0 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd247dbe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a67bf90 con 0x55bd6a7142b0 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd227fff640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2280a1ba0 0x7fd2280a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50748/0 (socket says 172.21.15.42:50748) 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23f59e640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2280a1ba0 0x7fd2280a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3974403888 learned_addr learned my addr 172.21.15.42:0/3974403888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23e59c640 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd227fff640 1 -- 172.21.15.42:0/839690956 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd2300cd960 con 0x7fd2400ff790 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23f59e640 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a714690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3974403888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 msgr2=0x55bd6a714690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a714690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3974403888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 msgr2=0x55bd6a71fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3974403888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd6a63a570 con 0x55bd6a714bd0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71d270 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd2300a2850 tx=0x7fd23006d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a93ea9f3fd6193 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.222+0000 7fd23f59e640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2280a1ba0 0x7fd2280a4060 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd23802e040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd2257fa640 1 -- 172.21.15.42:0/3974403888 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230099f40 con 0x55bd6a714bd0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd2257fa640 1 -- 172.21.15.42:0/3974403888 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd23000fa90 con 0x55bd6a714bd0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd2257fa640 1 -- 172.21.15.42:0/3974403888 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300ba0b0 con 0x55bd6a714bd0 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd247dbe740 1 -- 172.21.15.42:0/3974403888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 msgr2=0x55bd6a71d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71d270 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd2300a2850 tx=0x7fd23006d890 comp rx=0 tx=0).stop 2024-03-12T04:20:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd247dbe740 1 -- 172.21.15.42:0/3974403888 shutdown_connections 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71d270 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3974403888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a714690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.223+0000 7fd247dbe740 1 -- 172.21.15.42:0/3974403888 >> 172.21.15.42:0/3974403888 conn(0x55bd6a70e990 msgr2=0x55bd6a70ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 -- 172.21.15.42:0/3974403888 shutdown_connections 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 -- 172.21.15.42:0/3974403888 wait complete. 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 -- start start 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 Processor -- start 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a71d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a34f430 con 0x55bd6a714bd0 2024-03-12T04:20:28.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a30be00 con 0x55bd6a71d7b0 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a600fd0 con 0x55bd6a7142b0 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd247dbe740 10 client.0 _ll_register_callbacks cb 0x55bd6a656fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50762/0 (socket says 172.21.15.42:50762) 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd23e59c640 1 -- 172.21.15.42:0/2653747768 learned_addr learned my addr 172.21.15.42:0/2653747768 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a71d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.224+0000 7fd23f59e640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd23e59c640 1 -- 172.21.15.42:0/2653747768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 msgr2=0x55bd6a71d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd23e59c640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a71d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd23e59c640 1 -- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 msgr2=0x55bd6a71ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd23e59c640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd23e59c640 1 -- 172.21.15.42:0/2653747768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2401a4300 con 0x55bd6a714bd0 2024-03-12T04:20:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd23f59e640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd23e59c640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71a5f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd234010820 tx=0x7fd234010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ed0cfb6c2d2eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd222ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd222ff5640 1 -- 172.21.15.42:0/2653747768 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400eda0 con 0x55bd6a714bd0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd222ff5640 1 -- 172.21.15.42:0/2653747768 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd234018430 con 0x55bd6a714bd0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd6a37b7e0 con 0x55bd6a714bd0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd222ff5640 1 -- 172.21.15.42:0/2653747768 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2340185f0 con 0x55bd6a714bd0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd247dbe740 10 client.16935 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bd6a67a610 con 0x55bd6a714bd0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd222ff5640 1 -- 172.21.15.42:0/2653747768 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bd6a37b7e0 con 0x55bd6a714bd0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.225+0000 7fd222ff5640 1 -- 172.21.15.42:0/2653747768 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd23400d3c0 con 0x55bd6a714bd0 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd247dbe740 10 client.16935 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd247dbe740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd247dbe740 20 client.16935 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd247dbe740 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a72dac0 0x55bd6a72fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd247dbe740 4 client.16935 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd6a3b8720 con 0x55bd6a72dac0 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a72dac0 0x55bd6a72fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a72dac0 0x55bd6a72fee0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29a6bca85ce7c04 server_cookie=fe1213a9314d423f in_seq=0 out_seq=0 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.226+0000 7fd222ff5640 10 client.16935 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd222ff5640 1 -- 172.21.15.42:0/2653747768 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd6a3b8720 con 0x55bd6a72dac0 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd222ff5640 10 client.16935 handle_command_reply: tid=0 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 client.16935 shutdown 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 msgr2=0x55bd6a71a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71a5f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd234010820 tx=0x7fd234010ce0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 shutdown_connections 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a72dac0 0x55bd6a72fee0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a714bd0 0x55bd6a71a5f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd222ff5640 0 client.16935 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 --2- 172.21.15.42:0/2653747768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a71d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 >> 172.21.15.42:0/2653747768 conn(0x55bd6a70e990 msgr2=0x55bd6a710730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 shutdown_connections 2024-03-12T04:20:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 1 -- 172.21.15.42:0/2653747768 wait complete. 2024-03-12T04:20:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.227+0000 7fd247dbe740 20 client.16935 trim_cache size 0 max 16384 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 Processor -- start 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 -- start start 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a724660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a714bd0 0x55bd6a724ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a729570 con 0x55bd6a71d7b0 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a70f9e0 con 0x55bd6a714bd0 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.242+0000 7fd247dbe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a7350f0 con 0x55bd6a7142b0 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50776/0 (socket says 172.21.15.42:50776) 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 -- 172.21.15.42:0/4192600495 learned_addr learned my addr 172.21.15.42:0/4192600495 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23e59c640 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a714bd0 0x55bd6a724ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a724660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 -- 172.21.15.42:0/4192600495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 msgr2=0x55bd6a724660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a724660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 -- 172.21.15.42:0/4192600495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a714bd0 msgr2=0x55bd6a724ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a714bd0 0x55bd6a724ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 -- 172.21.15.42:0/4192600495 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd6a616ac0 con 0x55bd6a71d7b0 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23e59c640 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a714bd0 0x55bd6a724ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd23f59e640 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71cd20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd238036000 tx=0x7fd23803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972bc82c9a1908f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd224ff9640 1 -- 172.21.15.42:0/4192600495 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380327d0 con 0x55bd6a71d7b0 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd224ff9640 1 -- 172.21.15.42:0/4192600495 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd238034440 con 0x55bd6a71d7b0 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.243+0000 7fd224ff9640 1 -- 172.21.15.42:0/4192600495 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238004670 con 0x55bd6a71d7b0 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- 172.21.15.42:0/4192600495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a71d7b0 msgr2=0x55bd6a71cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71cd20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd238036000 tx=0x7fd23803f000 comp rx=0 tx=0).stop 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- 172.21.15.42:0/4192600495 shutdown_connections 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a71d7b0 0x55bd6a71cd20 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a714bd0 0x55bd6a724ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 --2- 172.21.15.42:0/4192600495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a7142b0 0x55bd6a724660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- 172.21.15.42:0/4192600495 >> 172.21.15.42:0/4192600495 conn(0x55bd6a70e990 msgr2=0x55bd6a715910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- 172.21.15.42:0/4192600495 shutdown_connections 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- 172.21.15.42:0/4192600495 wait complete. 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- start start 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 Processor -- start 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a7142b0 0x55bd6a724b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a714bd0 0x55bd6a722ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a723520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a723c20 con 0x55bd6a7142b0 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a783ff0 con 0x55bd6a71d7b0 2024-03-12T04:20:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a784130 con 0x55bd6a714bd0 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd247dbe740 10 client.0 _ll_register_callbacks cb 0x7fd230096950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.244+0000 7fd23ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a7142b0 0x55bd6a724b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a7142b0 0x55bd6a724b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50782/0 (socket says 172.21.15.42:50782) 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3420329054 learned_addr learned my addr 172.21.15.42:0/3420329054 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23e59c640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a714bd0 0x55bd6a722ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23f59e640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a723520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3420329054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a714bd0 msgr2=0x55bd6a722ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a714bd0 0x55bd6a722ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 msgr2=0x55bd6a723520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a723520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 -- 172.21.15.42:0/3420329054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd240069320 con 0x55bd6a7142b0 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23f59e640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a723520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd23ed9d640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a7142b0 0x55bd6a724b00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd23009a2e0 tx=0x7fd230085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e38f1303fe5f43 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd222ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd222ff5640 1 -- 172.21.15.42:0/3420329054 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23009a5e0 con 0x55bd6a7142b0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd222ff5640 1 -- 172.21.15.42:0/3420329054 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd23009ace0 con 0x55bd6a7142b0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd6a407890 con 0x55bd6a7142b0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd222ff5640 1 -- 172.21.15.42:0/3420329054 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2300ba0b0 con 0x55bd6a7142b0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd247dbe740 10 client.16941 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bd6a63c6a0 con 0x55bd6a7142b0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.245+0000 7fd222ff5640 1 -- 172.21.15.42:0/3420329054 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bd6a407890 con 0x55bd6a7142b0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd222ff5640 1 -- 172.21.15.42:0/3420329054 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd230016020 con 0x55bd6a7142b0 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd247dbe740 10 client.16941 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd247dbe740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd247dbe740 20 client.16941 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a7854a0 0x55bd6a787890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd247dbe740 4 client.16941 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bd6a36a4e0 con 0x55bd6a7854a0 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd23e59c640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a7854a0 0x55bd6a787890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd222ff5640 10 client.16941 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.246+0000 7fd23e59c640 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a7854a0 0x55bd6a787890 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9790b515dc81015f server_cookie=749f670747ce5e8a in_seq=0 out_seq=0 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd222ff5640 1 -- 172.21.15.42:0/3420329054 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bd6a36a4e0 con 0x55bd6a7854a0 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd222ff5640 10 client.16941 handle_command_reply: tid=0 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 client.16941 shutdown 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a7142b0 msgr2=0x55bd6a724b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a7142b0 0x55bd6a724b00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd23009a2e0 tx=0x7fd230085040 comp rx=0 tx=0).stop 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 shutdown_connections 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bd6a7854a0 0x55bd6a787890 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd6a71d7b0 0x55bd6a723520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd6a714bd0 0x55bd6a722ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd222ff5640 0 client.16941 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 --2- 172.21.15.42:0/3420329054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6a7142b0 0x55bd6a724b00 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 >> 172.21.15.42:0/3420329054 conn(0x55bd6a70e990 msgr2=0x55bd6a719e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 shutdown_connections 2024-03-12T04:20:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 1 -- 172.21.15.42:0/3420329054 wait complete. 2024-03-12T04:20:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd247dbe740 20 client.16941 trim_cache size 0 max 16384 2024-03-12T04:20:28.257 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:20:28.257 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:20:28.257 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:20:28.257 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:20:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2280a1ba0 msgr2=0x7fd2280a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2280a1ba0 0x7fd2280a4060 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd23802e040 comp rx=0 tx=0).stop 2024-03-12T04:20:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2400ff790 msgr2=0x7fd2401975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2400ff790 0x7fd2401975b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c920 tx=0x7fd23000cde0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 shutdown_connections 2024-03-12T04:20:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd2280a1ba0 0x7fd2280a4060 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2401057a0 0x7fd24019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd240100150 0x7fd240197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 --2- 172.21.15.42:0/839690956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2400ff790 0x7fd2401975b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.247+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 >> 172.21.15.42:0/839690956 conn(0x7fd2400765b0 msgr2=0x7fd240102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.248+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 shutdown_connections 2024-03-12T04:20:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.248+0000 7fd2451e9640 1 -- 172.21.15.42:0/839690956 wait complete. 2024-03-12T04:20:28.261 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:20:28.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:20:28.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.361+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3782742461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e14100ca0 msgr2=0x7f4e14104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.361+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3782742461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e14100ca0 0x7f4e14104ce0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009930 tx=0x7f4e0802bc30 comp rx=0 tx=0).stop 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.361+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3782742461 shutdown_connections 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.361+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3782742461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e141053c0 0x7f4e1410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.361+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3782742461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e14100ca0 0x7f4e14104ce0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.361+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3782742461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e141002f0 0x7f4e141006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.361+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3782742461 >> 172.21.15.42:0/3782742461 conn(0x7f4e14077180 msgr2=0x7f4e14077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3782742461 shutdown_connections 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3782742461 wait complete. 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 Processor -- start 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 -- start start 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e141002f0 0x7f4e141985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e141053c0 0x7f4e14198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e141990c0 0x7f4e141a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141a3d10 con 0x7f4e141002f0 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141a3e80 con 0x7f4e141990c0 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1beeb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141a4040 con 0x7f4e141053c0 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e19c87640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e141002f0 0x7f4e141985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e19c87640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e141002f0 0x7f4e141985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50802/0 (socket says 172.21.15.42:50802) 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e19c87640 1 -- 172.21.15.42:0/3202125719 learned_addr learned my addr 172.21.15.42:0/3202125719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.362+0000 7f4e1a488640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e141990c0 0x7f4e141a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e19c87640 1 -- 172.21.15.42:0/3202125719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e141053c0 msgr2=0x7f4e14198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e19c87640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e141053c0 0x7f4e14198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e19c87640 1 -- 172.21.15.42:0/3202125719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e141990c0 msgr2=0x7f4e141a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e19c87640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e141990c0 0x7f4e141a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e19c87640 1 -- 172.21.15.42:0/3202125719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e08009560 con 0x7f4e141002f0 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e1a488640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e141990c0 0x7f4e141a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e19c87640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e141002f0 0x7f4e141985f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00d950 tx=0x7f4e0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e12ffd640 1 -- 172.21.15.42:0/3202125719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c009a30 con 0x7f4e141002f0 2024-03-12T04:20:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e141a4320 con 0x7f4e141002f0 2024-03-12T04:20:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e12ffd640 1 -- 172.21.15.42:0/3202125719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e0c0044d0 con 0x7f4e141002f0 2024-03-12T04:20:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e12ffd640 1 -- 172.21.15.42:0/3202125719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0047a0 con 0x7f4e141002f0 2024-03-12T04:20:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.363+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e141a4880 con 0x7f4e141002f0 2024-03-12T04:20:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.364+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e14067f70 con 0x7f4e141002f0 2024-03-12T04:20:28.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.365+0000 7f4e12ffd640 1 -- 172.21.15.42:0/3202125719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f4e0c009b90 con 0x7f4e141002f0 2024-03-12T04:20:28.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.366+0000 7f4e12ffd640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4dfc0a1ba0 0x7f4dfc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.366+0000 7f4e19486640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4dfc0a1ba0 0x7f4dfc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.366+0000 7f4e12ffd640 1 -- 172.21.15.42:0/3202125719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4e0c0cd8e0 con 0x7f4e141002f0 2024-03-12T04:20:28.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.366+0000 7f4e19486640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4dfc0a1ba0 0x7f4dfc0a4060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e08036040 tx=0x7f4e08000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.369+0000 7f4e12ffd640 1 -- 172.21.15.42:0/3202125719 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e0c096800 con 0x7f4e141002f0 2024-03-12T04:20:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.591+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e140688d0 con 0x7f4e141002f0 2024-03-12T04:20:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.592+0000 7f4e12ffd640 1 -- 172.21.15.42:0/3202125719 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f4e0c017090 con 0x7f4e141002f0 2024-03-12T04:20:28.595 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:20:28.595 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:20:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:20:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4dfc0a1ba0 msgr2=0x7f4dfc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4dfc0a1ba0 0x7f4dfc0a4060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e08036040 tx=0x7f4e08000bc0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e141002f0 msgr2=0x7f4e141985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e141002f0 0x7f4e141985f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00d950 tx=0x7f4e0c00de10 comp rx=0 tx=0).stop 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 shutdown_connections 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4dfc0a1ba0 0x7f4dfc0a4060 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e141990c0 0x7f4e141a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e141053c0 0x7f4e14198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 --2- 172.21.15.42:0/3202125719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e141002f0 0x7f4e141985f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 >> 172.21.15.42:0/3202125719 conn(0x7f4e14077180 msgr2=0x7f4e140fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 shutdown_connections 2024-03-12T04:20:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.594+0000 7f4e1beeb640 1 -- 172.21.15.42:0/3202125719 wait complete. 2024-03-12T04:20:28.607 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:20:28.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 -- 172.21.15.42:0/3623545593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 msgr2=0x7f3e8c100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 --2- 172.21.15.42:0/3623545593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c100660 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009930 tx=0x7f3e8802bc30 comp rx=0 tx=0).stop 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 -- 172.21.15.42:0/3623545593 shutdown_connections 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 --2- 172.21.15.42:0/3623545593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c105350 0x7f3e8c10cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 --2- 172.21.15.42:0/3623545593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e8c100c30 0x7f3e8c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 --2- 172.21.15.42:0/3623545593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c100660 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 -- 172.21.15.42:0/3623545593 >> 172.21.15.42:0/3623545593 conn(0x7f3e8c077150 msgr2=0x7f3e8c077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 -- 172.21.15.42:0/3623545593 shutdown_connections 2024-03-12T04:20:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 -- 172.21.15.42:0/3623545593 wait complete. 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.710+0000 7f3e947e9640 1 Processor -- start 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e947e9640 1 -- start start 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e947e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e947e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c105350 0x7f3e8c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e947e9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e8c19b010 0x7f3e8c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e947e9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c19ba00 con 0x7f3e8c100280 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e947e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c19bb70 con 0x7f3e8c105350 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e947e9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c19bce0 con 0x7f3e8c19b010 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c1967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50822/0 (socket says 172.21.15.42:50822) 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 -- 172.21.15.42:0/4242352227 learned_addr learned my addr 172.21.15.42:0/4242352227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e91d84640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c105350 0x7f3e8c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92d86640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e8c19b010 0x7f3e8c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 -- 172.21.15.42:0/4242352227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e8c19b010 msgr2=0x7f3e8c19b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e8c19b010 0x7f3e8c19b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 -- 172.21.15.42:0/4242352227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c105350 msgr2=0x7f3e8c196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c105350 0x7f3e8c196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 -- 172.21.15.42:0/4242352227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009560 con 0x7f3e8c100280 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e92585640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c1967a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009230 tx=0x7f3e880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.711+0000 7f3e837fe640 1 -- 172.21.15.42:0/4242352227 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880044e0 con 0x7f3e8c100280 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.712+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c1a32c0 con 0x7f3e8c100280 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.712+0000 7f3e837fe640 1 -- 172.21.15.42:0/4242352227 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e8802de90 con 0x7f3e8c100280 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.712+0000 7f3e837fe640 1 -- 172.21.15.42:0/4242352227 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88005390 con 0x7f3e8c100280 2024-03-12T04:20:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.712+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c1a3770 con 0x7f3e8c100280 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e837fe640 1 -- 172.21.15.42:0/4242352227 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3e880054f0 con 0x7f3e8c100280 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 Processor -- start 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 -- start start 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217981630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981b70 0x56121798a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56121798a750 0x56121798cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121793c960 con 0x561217981250 2024-03-12T04:20:28.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612178f0310 con 0x56121798a750 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.714+0000 7f3e973c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612178ed840 con 0x561217981b70 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217981630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e837fe640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3e780a1ba0 0x7f3e780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217981630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50830/0 (socket says 172.21.15.42:50830) 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 -- 172.21.15.42:0/1660538736 learned_addr learned my addr 172.21.15.42:0/1660538736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e91d84640 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56121798a750 0x56121798cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e837fe640 1 -- 172.21.15.42:0/4242352227 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3e880ef9c0 con 0x7f3e8c100280 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92585640 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981b70 0x56121798a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3e780a1ba0 0x7f3e780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 -- 172.21.15.42:0/1660538736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981b70 msgr2=0x56121798a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981b70 0x56121798a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 -- 172.21.15.42:0/1660538736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56121798a750 msgr2=0x56121798cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56121798a750 0x56121798cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 -- 172.21.15.42:0/1660538736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612178ab060 con 0x561217981250 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217981630 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400cb20 tx=0x7f3e840072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15213012175b3f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e80ff9640 1 -- 172.21.15.42:0/1660538736 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8401a640 con 0x561217981250 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e80ff9640 1 -- 172.21.15.42:0/1660538736 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e8400c560 con 0x561217981250 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e80ff9640 1 -- 172.21.15.42:0/1660538736 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400a860 con 0x561217981250 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.715+0000 7f3e92d86640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3e780a1ba0 0x7f3e780a4060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e8401f000 tx=0x7f3e8401b000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 -- 172.21.15.42:0/1660538736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 msgr2=0x561217981630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217981630 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400cb20 tx=0x7f3e840072b0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 -- 172.21.15.42:0/1660538736 shutdown_connections 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56121798a750 0x56121798cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981b70 0x56121798a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1660538736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217981630 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 -- 172.21.15.42:0/1660538736 >> 172.21.15.42:0/1660538736 conn(0x56121797b980 msgr2=0x56121797bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 -- 172.21.15.42:0/1660538736 shutdown_connections 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.716+0000 7f3e973c2740 1 -- 172.21.15.42:0/1660538736 wait complete. 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 -- start start 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 Processor -- start 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217987780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217987cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217988200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612178e9020 con 0x561217981250 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612175bc430 con 0x561217981b70 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561217578e00 con 0x56121798a750 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e973c2740 10 client.0 _ll_register_callbacks cb 0x561217890cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217987780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217987780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50838/0 (socket says 172.21.15.42:50838) 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 -- 172.21.15.42:0/851077140 learned_addr learned my addr 172.21.15.42:0/851077140 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e91d84640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217987cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92d86640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217988200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 -- 172.21.15.42:0/851077140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 msgr2=0x561217988200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217988200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 -- 172.21.15.42:0/851077140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 msgr2=0x561217987cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217987cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 -- 172.21.15.42:0/851077140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8400c700 con 0x561217981250 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e91d84640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217987cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92d86640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217988200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e92585640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217987780 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb1560c8a4eb7fa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.717+0000 7f3e75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e75ffb640 1 -- 172.21.15.42:0/851077140 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880ed580 con 0x561217981250 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e75ffb640 1 -- 172.21.15.42:0/851077140 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3e880c0840 con 0x561217981250 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612175e87e0 con 0x561217981250 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e75ffb640 1 -- 172.21.15.42:0/851077140 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e880d5200 con 0x561217981250 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 10 client.16959 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5612178e76a0 con 0x561217981250 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e75ffb640 1 -- 172.21.15.42:0/851077140 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5612175e87e0 con 0x561217981250 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e75ffb640 1 -- 172.21.15.42:0/851077140 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e8803c070 con 0x561217981250 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 10 client.16959 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 10 client.16959 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 20 client.16959 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:28.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x56121799a750 0x56121799cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 4 client.16959 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:28.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561217625720 con 0x56121799a750 2024-03-12T04:20:28.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.718+0000 7f3e91d84640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x56121799a750 0x56121799cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e75ffb640 10 client.16959 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:20:28.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e91d84640 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x56121799a750 0x56121799cb40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e35663e6ec1b8f74 server_cookie=47bc48d415b2d976 in_seq=0 out_seq=0 2024-03-12T04:20:28.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e75ffb640 1 -- 172.21.15.42:0/851077140 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561217625720 con 0x56121799a750 2024-03-12T04:20:28.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e75ffb640 10 client.16959 handle_command_reply: tid=0 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 client.16959 shutdown 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 msgr2=0x561217987780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217987780 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e88009260 comp rx=0 tx=0).stop 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 shutdown_connections 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217988200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217987cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x56121799a750 0x56121799cb40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 --2- 172.21.15.42:0/851077140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217987780 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e75ffb640 0 client.16959 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 >> 172.21.15.42:0/851077140 conn(0x56121797b980 msgr2=0x56121797d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 shutdown_connections 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 1 -- 172.21.15.42:0/851077140 wait complete. 2024-03-12T04:20:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.719+0000 7f3e973c2740 20 client.16959 trim_cache size 0 max 16384 2024-03-12T04:20:28.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.731+0000 7f3e973c2740 1 Processor -- start 2024-03-12T04:20:28.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.731+0000 7f3e973c2740 1 -- start start 2024-03-12T04:20:28.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.731+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981250 0x561217989eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.731+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217988020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56121798a750 0x561217984a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e973c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121786e130 con 0x56121798a750 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e973c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612175e87e0 con 0x561217981b70 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e973c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121798cf10 con 0x561217981250 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56121798a750 0x561217984a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56121798a750 0x561217984a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50854/0 (socket says 172.21.15.42:50854) 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 -- 172.21.15.42:0/1775801648 learned_addr learned my addr 172.21.15.42:0/1775801648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92585640 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981250 0x561217989eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 -- 172.21.15.42:0/1775801648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981250 msgr2=0x561217989eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981250 0x561217989eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 -- 172.21.15.42:0/1775801648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 msgr2=0x561217988020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e91d84640 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217988020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217988020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 -- 172.21.15.42:0/1775801648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561217883c20 con 0x56121798a750 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92585640 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981250 0x561217989eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e91d84640 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217988020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e92d86640 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56121798a750 0x561217984a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009590 tx=0x7f3e8400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc1b5a199f28222 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e77fff640 1 -- 172.21.15.42:0/1775801648 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84028040 con 0x56121798a750 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e77fff640 1 -- 172.21.15.42:0/1775801648 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e84007eb0 con 0x56121798a750 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.732+0000 7f3e77fff640 1 -- 172.21.15.42:0/1775801648 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8402d6a0 con 0x56121798a750 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 -- 172.21.15.42:0/1775801648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56121798a750 msgr2=0x561217984a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56121798a750 0x561217984a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009590 tx=0x7f3e8400e040 comp rx=0 tx=0).stop 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 -- 172.21.15.42:0/1775801648 shutdown_connections 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56121798a750 0x561217984a40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217988020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 --2- 172.21.15.42:0/1775801648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561217981250 0x561217989eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 -- 172.21.15.42:0/1775801648 >> 172.21.15.42:0/1775801648 conn(0x56121797b980 msgr2=0x5612179a14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 -- 172.21.15.42:0/1775801648 shutdown_connections 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 -- 172.21.15.42:0/1775801648 wait complete. 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 -- start start 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.733+0000 7f3e973c2740 1 Processor -- start 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217990130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217990670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217990bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612179f0c70 con 0x561217981250 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612179f0db0 con 0x561217981b70 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612179f0f70 con 0x56121798a750 2024-03-12T04:20:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 10 client.0 _ll_register_callbacks cb 0x7f3e840097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217990130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217990130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50862/0 (socket says 172.21.15.42:50862) 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 -- 172.21.15.42:0/3989764887 learned_addr learned my addr 172.21.15.42:0/3989764887 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e91d84640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217990670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92d86640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217990bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 -- 172.21.15.42:0/3989764887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 msgr2=0x561217990bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217990bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 -- 172.21.15.42:0/3989764887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 msgr2=0x561217990670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217990670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 -- 172.21.15.42:0/3989764887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612178ab060 con 0x561217981250 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e91d84640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217990670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e92585640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217990130 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3e88006010 tx=0x7f3e880ed5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc91c100b7e86df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e75ffb640 1 -- 172.21.15.42:0/3989764887 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880bcdf0 con 0x561217981250 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e75ffb640 1 -- 172.21.15.42:0/3989764887 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3e880d5200 con 0x561217981250 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e8400c560 con 0x561217981250 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e75ffb640 1 -- 172.21.15.42:0/3989764887 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e880bc230 con 0x561217981250 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 10 client.16965 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.734+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5612178a98c0 con 0x561217981250 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e75ffb640 1 -- 172.21.15.42:0/3989764887 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e8400c560 con 0x561217981250 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e75ffb640 1 -- 172.21.15.42:0/3989764887 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e8803c040 con 0x561217981250 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 10 client.16965 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 10 client.16965 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 20 client.16965 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5612179f21d0 0x5612179f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 4 client.16965 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:20:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561217674890 con 0x5612179f21d0 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e91d84640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5612179f21d0 0x5612179f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e75ffb640 10 client.16965 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e91d84640 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5612179f21d0 0x5612179f45c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc96a3e7196da60e server_cookie=6ad2195f63b8e9ec in_seq=0 out_seq=0 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e75ffb640 1 -- 172.21.15.42:0/3989764887 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561217674890 con 0x5612179f21d0 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e75ffb640 10 client.16965 handle_command_reply: tid=0 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 1 client.16965 shutdown 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 msgr2=0x561217990130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.735+0000 7f3e973c2740 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217990130 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3e88006010 tx=0x7f3e880ed5a0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 shutdown_connections 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121798a750 0x561217990bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561217981b70 0x561217990670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5612179f21d0 0x5612179f45c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 --2- 172.21.15.42:0/3989764887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561217981250 0x561217990130 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e75ffb640 0 client.16965 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:20:28.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 >> 172.21.15.42:0/3989764887 conn(0x56121797b980 msgr2=0x56121797a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 shutdown_connections 2024-03-12T04:20:28.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 1 -- 172.21.15.42:0/3989764887 wait complete. 2024-03-12T04:20:28.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e973c2740 20 client.16965 trim_cache size 0 max 16384 2024-03-12T04:20:28.747 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:20:28.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3e780a1ba0 msgr2=0x7f3e780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3e780a1ba0 0x7f3e780a4060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e8401f000 tx=0x7f3e8401b000 comp rx=0 tx=0).stop 2024-03-12T04:20:28.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 msgr2=0x7f3e8c1967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c1967a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009230 tx=0x7f3e880042f0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 shutdown_connections 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3e780a1ba0 0x7f3e780a4060 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e8c19b010 0x7f3e8c19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c105350 0x7f3e8c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 --2- 172.21.15.42:0/4242352227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e8c100280 0x7f3e8c1967a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 >> 172.21.15.42:0/4242352227 conn(0x7f3e8c077150 msgr2=0x7f3e8c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 shutdown_connections 2024-03-12T04:20:28.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.736+0000 7f3e947e9640 1 -- 172.21.15.42:0/4242352227 wait complete. 2024-03-12T04:20:28.750 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:20:28.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:20:28.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 -- 172.21.15.42:0/42131898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474101d30 msgr2=0x7fa474105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/42131898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474101d30 0x7fa474105d70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa460009930 tx=0x7fa46002bc30 comp rx=0 tx=0).stop 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 -- 172.21.15.42:0/42131898 shutdown_connections 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/42131898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa474106450 0x7fa47410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/42131898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474101d30 0x7fa474105d70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/42131898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa474101380 0x7fa474101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 -- 172.21.15.42:0/42131898 >> 172.21.15.42:0/42131898 conn(0x7fa474078580 msgr2=0x7fa474078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 -- 172.21.15.42:0/42131898 shutdown_connections 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 -- 172.21.15.42:0/42131898 wait complete. 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 Processor -- start 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.853+0000 7fa47b1e9640 1 -- start start 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa47b1e9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa474101380 0x7fa474073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa47b1e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474106450 0x7fa4740739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa47b1e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa474073f00 0x7fa4741a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa47b1e9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a5ec0 con 0x7fa474106450 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa47b1e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a6030 con 0x7fa474073f00 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa47b1e9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a61f0 con 0x7fa474101380 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474106450 0x7fa4740739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474106450 0x7fa4740739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50884/0 (socket says 172.21.15.42:50884) 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 -- 172.21.15.42:0/3976640715 learned_addr learned my addr 172.21.15.42:0/3976640715 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa478f85640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa474101380 0x7fa474073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 -- 172.21.15.42:0/3976640715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa474101380 msgr2=0x7fa474073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa474101380 0x7fa474073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 -- 172.21.15.42:0/3976640715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa474073f00 msgr2=0x7fa4741a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa474073f00 0x7fa4741a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 -- 172.21.15.42:0/3976640715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460009560 con 0x7fa474106450 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa473fff640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474106450 0x7fa4740739c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa460004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa471ffb640 1 -- 172.21.15.42:0/3976640715 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46002cb70 con 0x7fa474106450 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa471ffb640 1 -- 172.21.15.42:0/3976640715 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa46002de90 con 0x7fa474106450 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4741a6470 con 0x7fa474106450 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.854+0000 7fa471ffb640 1 -- 172.21.15.42:0/3976640715 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600053c0 con 0x7fa474106450 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.855+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4741a6980 con 0x7fa474106450 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa471ffb640 1 -- 172.21.15.42:0/3976640715 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa4600035f0 con 0x7fa474106450 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 Processor -- start 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 -- start start 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db08f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db098210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 0x55f6db09ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6db04a960 con 0x55f6db08f250 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6daffe310 con 0x55f6db08fb70 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6daffb840 con 0x55f6db098750 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa471ffb640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa4640a1ba0 0x7fa4640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa479786640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db08f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa479786640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db08f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50900/0 (socket says 172.21.15.42:50900) 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa479786640 1 -- 172.21.15.42:0/2664716632 learned_addr learned my addr 172.21.15.42:0/2664716632 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa471ffb640 1 -- 172.21.15.42:0/3976640715 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa4600efb60 con 0x7fa474106450 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.857+0000 7fa479786640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa4640a1ba0 0x7fa4640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa473fff640 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 0x55f6db09ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa478f85640 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db098210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa479786640 1 -- 172.21.15.42:0/2664716632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 msgr2=0x55f6db09ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa479786640 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 0x55f6db09ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa479786640 1 -- 172.21.15.42:0/2664716632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 msgr2=0x55f6db098210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa479786640 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db098210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa479786640 1 -- 172.21.15.42:0/2664716632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6dafb9060 con 0x55f6db08f250 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa478f85640 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db098210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa479786640 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db08f630 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa46c012cb0 tx=0x7fa46c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ab9ac615d16768 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa479786640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa4640a1ba0 0x7fa4640a4060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005c40 tx=0x7fa46c01a000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa45effd640 1 -- 172.21.15.42:0/2664716632 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c012030 con 0x55f6db08f250 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa45effd640 1 -- 172.21.15.42:0/2664716632 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa46c018040 con 0x55f6db08f250 2024-03-12T04:20:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.858+0000 7fa45effd640 1 -- 172.21.15.42:0/2664716632 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005020 con 0x55f6db08f250 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2664716632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 msgr2=0x55f6db08f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db08f630 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa46c012cb0 tx=0x7fa46c00f730 comp rx=0 tx=0).stop 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2664716632 shutdown_connections 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 0x55f6db09ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db098210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2664716632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db08f630 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2664716632 >> 172.21.15.42:0/2664716632 conn(0x55f6db089980 msgr2=0x55f6db089d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2664716632 shutdown_connections 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2664716632 wait complete. 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 -- start start 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.859+0000 7fa47ddc0740 1 Processor -- start 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08f250 0x55f6db0957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08fb70 0x55f6db095d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db096250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6daff7020 con 0x55f6db098750 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dacca430 con 0x55f6db08f250 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dac86e00 con 0x55f6db08fb70 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa47ddc0740 10 client.0 _ll_register_callbacks cb 0x55f6daf9ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db096250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db096250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50910/0 (socket says 172.21.15.42:50910) 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 -- 172.21.15.42:0/2081266516 learned_addr learned my addr 172.21.15.42:0/2081266516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa473fff640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08fb70 0x55f6db095d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa478f85640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08f250 0x55f6db0957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 -- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08fb70 msgr2=0x55f6db095d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08fb70 0x55f6db095d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 -- 172.21.15.42:0/2081266516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08f250 msgr2=0x55f6db0957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08f250 0x55f6db0957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 -- 172.21.15.42:0/2081266516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c012860 con 0x55f6db098750 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa473fff640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08fb70 0x55f6db095d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa478f85640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08f250 0x55f6db0957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa479786640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db096250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa46c015090 tx=0x7fa46c019b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c735a9fc0b87116 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa453fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa453fff640 1 -- 172.21.15.42:0/2081266516 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c019df0 con 0x55f6db098750 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.860+0000 7fa453fff640 1 -- 172.21.15.42:0/2081266516 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa46c005020 con 0x55f6db098750 2024-03-12T04:20:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6dacf67e0 con 0x55f6db098750 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa453fff640 1 -- 172.21.15.42:0/2081266516 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa46c005200 con 0x55f6db098750 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 10 client.16977 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f6daff56a0 con 0x55f6db098750 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa453fff640 1 -- 172.21.15.42:0/2081266516 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f6dacf67e0 con 0x55f6db098750 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa453fff640 1 -- 172.21.15.42:0/2081266516 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa46c01c730 con 0x55f6db098750 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 10 client.16977 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 10 client.16977 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 20 client.16977 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db0a8ba0 0x55f6db0aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 4 client.16977 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6dad33720 con 0x55f6db0a8ba0 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.861+0000 7fa478f85640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db0a8ba0 0x55f6db0aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.862+0000 7fa478f85640 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db0a8ba0 0x55f6db0aaf90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=33c4a42796d86251 server_cookie=2591863e53c331c1 in_seq=0 out_seq=0 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.862+0000 7fa453fff640 10 client.16977 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.862+0000 7fa453fff640 1 -- 172.21.15.42:0/2081266516 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6dad33720 con 0x55f6db0a8ba0 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.862+0000 7fa453fff640 10 client.16977 handle_command_reply: tid=0 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.862+0000 7fa47ddc0740 1 client.16977 shutdown 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 msgr2=0x55f6db096250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db096250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa46c015090 tx=0x7fa46c019b80 comp rx=0 tx=0).stop 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 shutdown_connections 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db0a8ba0 0x55f6db0aaf90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db096250 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08fb70 0x55f6db095d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2081266516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08f250 0x55f6db0957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa453fff640 0 client.16977 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:20:28.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 >> 172.21.15.42:0/2081266516 conn(0x55f6db089980 msgr2=0x55f6db08b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 shutdown_connections 2024-03-12T04:20:28.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2081266516 wait complete. 2024-03-12T04:20:28.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.863+0000 7fa47ddc0740 20 client.16977 trim_cache size 0 max 16384 2024-03-12T04:20:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.874+0000 7fa47ddc0740 1 Processor -- start 2024-03-12T04:20:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.874+0000 7fa47ddc0740 1 -- start start 2024-03-12T04:20:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db097e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db096070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 0x55f6db090a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6daf7c130 con 0x55f6db08f250 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6db0a5680 con 0x55f6db08fb70 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6db0b01a0 con 0x55f6db098750 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db097e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db097e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50924/0 (socket says 172.21.15.42:50924) 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 -- 172.21.15.42:0/2424720169 learned_addr learned my addr 172.21.15.42:0/2424720169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 -- 172.21.15.42:0/2424720169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 msgr2=0x55f6db090a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 --2- 172.21.15.42:0/2424720169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 0x55f6db090a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 -- 172.21.15.42:0/2424720169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 msgr2=0x55f6db096070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 --2- 172.21.15.42:0/2424720169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db096070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 -- 172.21.15.42:0/2424720169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6daf91c20 con 0x55f6db08f250 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa478f85640 1 --2- 172.21.15.42:0/2424720169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db097e90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x55f6db0ab870 tx=0x7fa46800e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b3863af797c7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa45e7fc640 1 -- 172.21.15.42:0/2424720169 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa468017040 con 0x55f6db08f250 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa45e7fc640 1 -- 172.21.15.42:0/2424720169 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa468004020 con 0x55f6db08f250 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.875+0000 7fa45e7fc640 1 -- 172.21.15.42:0/2424720169 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4680197d0 con 0x55f6db08f250 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2424720169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 msgr2=0x55f6db097e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2424720169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db097e90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x55f6db0ab870 tx=0x7fa46800e700 comp rx=0 tx=0).stop 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2424720169 shutdown_connections 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2424720169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db098750 0x55f6db090a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2424720169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db096070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/2424720169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db08f250 0x55f6db097e90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2424720169 >> 172.21.15.42:0/2424720169 conn(0x55f6db089980 msgr2=0x55f6db084a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2424720169 shutdown_connections 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 -- 172.21.15.42:0/2424720169 wait complete. 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 -- start start 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.876+0000 7fa47ddc0740 1 Processor -- start 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08f250 0x55f6db09e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db09e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db09eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6db0ff080 con 0x55f6db098750 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6db0ff1c0 con 0x55f6db08fb70 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6db0ff300 con 0x55f6db08f250 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 10 client.0 _ll_register_callbacks cb 0x7fa46c009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db09eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db09eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50930/0 (socket says 172.21.15.42:50930) 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 -- 172.21.15.42:0/468971507 learned_addr learned my addr 172.21.15.42:0/468971507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 -- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08f250 msgr2=0x55f6db09e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa478f85640 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08f250 0x55f6db09e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa473fff640 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db09e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08f250 0x55f6db09e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 -- 172.21.15.42:0/468971507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 msgr2=0x55f6db09e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db09e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 -- 172.21.15.42:0/468971507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46800ee70 con 0x55f6db098750 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa479786640 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db09eab0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa46c01cbe0 tx=0x7fa46c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1886793ae6e896dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa453fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa453fff640 1 -- 172.21.15.42:0/468971507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c003b70 con 0x55f6db098750 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa453fff640 1 -- 172.21.15.42:0/468971507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa46c009b20 con 0x55f6db098750 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa46c018040 con 0x55f6db098750 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa453fff640 1 -- 172.21.15.42:0/468971507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa46c003cd0 con 0x55f6db098750 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 10 client.16983 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.877+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f6dafb78c0 con 0x55f6db098750 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa453fff640 1 -- 172.21.15.42:0/468971507 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa46c018040 con 0x55f6db098750 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa453fff640 1 -- 172.21.15.42:0/468971507 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa46c035430 con 0x55f6db098750 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa47ddc0740 10 client.16983 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa47ddc0740 10 client.16983 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa47ddc0740 20 client.16983 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db100810 0x55f6db102c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa47ddc0740 4 client.16983 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6dad82890 con 0x55f6db100810 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa478f85640 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db100810 0x55f6db102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa478f85640 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db100810 0x55f6db102c30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7365ecc483cd5bc server_cookie=11f92cdf33fd698c in_seq=0 out_seq=0 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.878+0000 7fa453fff640 10 client.16983 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa453fff640 1 -- 172.21.15.42:0/468971507 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6dad82890 con 0x55f6db100810 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa453fff640 10 client.16983 handle_command_reply: tid=0 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 client.16983 shutdown 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 msgr2=0x55f6db09eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db09eab0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa46c01cbe0 tx=0x7fa46c01c9e0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 shutdown_connections 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55f6db100810 0x55f6db102c30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6db098750 0x55f6db09eab0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa453fff640 0 client.16983 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f6db08fb70 0x55f6db09e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 --2- 172.21.15.42:0/468971507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6db08f250 0x55f6db09e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 >> 172.21.15.42:0/468971507 conn(0x55f6db089980 msgr2=0x55f6db093c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 shutdown_connections 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 1 -- 172.21.15.42:0/468971507 wait complete. 2024-03-12T04:20:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47ddc0740 20 client.16983 trim_cache size 0 max 16384 2024-03-12T04:20:28.892 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:20:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa4640a1ba0 msgr2=0x7fa4640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.879+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa4640a1ba0 0x7fa4640a4060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005c40 tx=0x7fa46c01a000 comp rx=0 tx=0).stop 2024-03-12T04:20:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474106450 msgr2=0x7fa4740739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474106450 0x7fa4740739c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa460004270 comp rx=0 tx=0).stop 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 shutdown_connections 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa4640a1ba0 0x7fa4640a4060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa474073f00 0x7fa4741a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa474106450 0x7fa4740739c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 --2- 172.21.15.42:0/3976640715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa474101380 0x7fa474073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 >> 172.21.15.42:0/3976640715 conn(0x7fa474078580 msgr2=0x7fa4740ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 shutdown_connections 2024-03-12T04:20:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.880+0000 7fa47b1e9640 1 -- 172.21.15.42:0/3976640715 wait complete. 2024-03-12T04:20:28.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:28 smithi042 ceph-mon[32260]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:28.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:28 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3202125719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:20:28.895 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:20:28.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:20:28.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:28 smithi119 ceph-mon[36119]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:28.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:28 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3202125719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:20:28.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:28 smithi136 ceph-mon[35571]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:28.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:28 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3202125719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:20:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 -- 172.21.15.42:0/791578375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c1023c0 msgr2=0x7f4c8c1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 --2- 172.21.15.42:0/791578375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c1023c0 0x7f4c8c1027a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009900 tx=0x7f4c8002bcd0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 -- 172.21.15.42:0/791578375 shutdown_connections 2024-03-12T04:20:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 --2- 172.21.15.42:0/791578375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c8c10b790 0x7f4c8c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 --2- 172.21.15.42:0/791578375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c8c102ce0 0x7f4c8c10b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 --2- 172.21.15.42:0/791578375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c1023c0 0x7f4c8c1027a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 -- 172.21.15.42:0/791578375 >> 172.21.15.42:0/791578375 conn(0x7f4c8c076400 msgr2=0x7f4c8c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 -- 172.21.15.42:0/791578375 shutdown_connections 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.997+0000 7f4c92d08640 1 -- 172.21.15.42:0/791578375 wait complete. 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 Processor -- start 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 -- start start 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c8c1023c0 0x7f4c8c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c8c102ce0 0x7f4c8c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c10b790 0x7f4c8c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c19c560 con 0x7f4c8c10b790 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c19c6d0 con 0x7f4c8c102ce0 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c92d08640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c19c890 con 0x7f4c8c1023c0 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c10b790 0x7f4c8c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c10b790 0x7f4c8c19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50944/0 (socket says 172.21.15.42:50944) 2024-03-12T04:20:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 -- 172.21.15.42:0/2570402537 learned_addr learned my addr 172.21.15.42:0/2570402537 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c8c102ce0 0x7f4c8c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c8c1023c0 0x7f4c8c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 -- 172.21.15.42:0/2570402537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c8c1023c0 msgr2=0x7f4c8c1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c8c1023c0 0x7f4c8c1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 -- 172.21.15.42:0/2570402537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c8c102ce0 msgr2=0x7f4c8c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c8c102ce0 0x7f4c8c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c912a5640 1 -- 172.21.15.42:0/2570402537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80009560 con 0x7f4c8c10b790 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.998+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c8c1023c0 0x7f4c8c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.999+0000 7f4c912a5640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c10b790 0x7f4c8c19bf60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400ccb0 tx=0x7f4c8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.999+0000 7f4c89ffb640 1 -- 172.21.15.42:0/2570402537 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400f450 con 0x7f4c8c10b790 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.999+0000 7f4c89ffb640 1 -- 172.21.15.42:0/2570402537 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c8400fa60 con 0x7f4c8c10b790 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.999+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c19cb70 con 0x7f4c8c10b790 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.999+0000 7f4c89ffb640 1 -- 172.21.15.42:0/2570402537 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84018a80 con 0x7f4c8c10b790 2024-03-12T04:20:29.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:28.999+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c1a43f0 con 0x7f4c8c10b790 2024-03-12T04:20:29.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.001+0000 7f4c89ffb640 1 -- 172.21.15.42:0/2570402537 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f4c8400fbd0 con 0x7f4c8c10b790 2024-03-12T04:20:29.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.001+0000 7f4c959b0740 1 Processor -- start 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.001+0000 7f4c959b0740 1 -- start start 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb13724630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 0x55cb1372fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c959b0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb136df960 con 0x55cb13724250 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c959b0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb13693310 con 0x55cb1372d750 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c959b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb13690840 con 0x55cb13724b70 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c89ffb640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4c7c0a1ba0 0x7f4c7c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb13724630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb13724630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50956/0 (socket says 172.21.15.42:50956) 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 -- 172.21.15.42:0/4169760791 learned_addr learned my addr 172.21.15.42:0/4169760791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c912a5640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4c7c0a1ba0 0x7f4c7c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 0x55cb1372fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c89ffb640 1 -- 172.21.15.42:0/2570402537 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4c840cdbb0 con 0x7f4c8c10b790 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 -- 172.21.15.42:0/4169760791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 msgr2=0x55cb1372d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 -- 172.21.15.42:0/4169760791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 msgr2=0x55cb1372fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 0x55cb1372fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 -- 172.21.15.42:0/4169760791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb1364e060 con 0x55cb13724250 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb13724630 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4c78013620 tx=0x7f4c78009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9a88ca42ab9a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c912a5640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4c7c0a1ba0 0x7f4c7c0a4060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4c8409a760 tx=0x7f4c8400af30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c76ffd640 1 -- 172.21.15.42:0/4169760791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78004050 con 0x55cb13724250 2024-03-12T04:20:29.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c76ffd640 1 -- 172.21.15.42:0/4169760791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c780035f0 con 0x55cb13724250 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.002+0000 7f4c76ffd640 1 -- 172.21.15.42:0/4169760791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7800f7c0 con 0x55cb13724250 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 -- 172.21.15.42:0/4169760791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 msgr2=0x55cb13724630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb13724630 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4c78013620 tx=0x7f4c78009830 comp rx=0 tx=0).stop 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 -- 172.21.15.42:0/4169760791 shutdown_connections 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 0x55cb1372fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 --2- 172.21.15.42:0/4169760791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb13724630 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 -- 172.21.15.42:0/4169760791 >> 172.21.15.42:0/4169760791 conn(0x55cb1371e980 msgr2=0x55cb1371ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 -- 172.21.15.42:0/4169760791 shutdown_connections 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 -- 172.21.15.42:0/4169760791 wait complete. 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.003+0000 7f4c959b0740 1 -- start start 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 1 Processor -- start 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724250 0x55cb1372a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1372b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1368c020 con 0x55cb1372d750 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1335f430 con 0x55cb13724250 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1331be00 con 0x55cb13724b70 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c959b0740 10 client.0 _ll_register_callbacks cb 0x55cb13633cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1372b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1372b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50972/0 (socket says 172.21.15.42:50972) 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 -- 172.21.15.42:0/923873049 learned_addr learned my addr 172.21.15.42:0/923873049 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724250 0x55cb1372a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 -- 172.21.15.42:0/923873049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 msgr2=0x55cb1372ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 -- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724250 msgr2=0x55cb1372a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724250 0x55cb1372a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.004+0000 7f4c8bfff640 1 -- 172.21.15.42:0/923873049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c78009d70 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1372b1b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4c78003c80 tx=0x7f4c78009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf5fbb214e88e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c6bfff640 1 -- 172.21.15.42:0/923873049 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78004490 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c6bfff640 1 -- 172.21.15.42:0/923873049 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4c7800f580 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb1338b7e0 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c6bfff640 1 -- 172.21.15.42:0/923873049 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c7800f740 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 10 client.16995 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb1368a6a0 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c6bfff640 1 -- 172.21.15.42:0/923873049 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55cb1338b7e0 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c6bfff640 1 -- 172.21.15.42:0/923873049 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4c78010840 con 0x55cb1372d750 2024-03-12T04:20:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 10 client.16995 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 10 client.16995 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 20 client.16995 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb1373da30 0x55cb1373fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 4 client.16995 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.005+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb133c8720 con 0x55cb1373da30 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.006+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb1373da30 0x55cb1373fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.006+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb1373da30 0x55cb1373fe20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46c2c5381f39f95e server_cookie=58f94e66cb30c0a3 in_seq=0 out_seq=0 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.006+0000 7f4c6bfff640 10 client.16995 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c6bfff640 1 -- 172.21.15.42:0/923873049 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb133c8720 con 0x55cb1373da30 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c6bfff640 10 client.16995 handle_command_reply: tid=0 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 client.16995 shutdown 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 msgr2=0x55cb1372b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1372b1b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4c78003c80 tx=0x7f4c78009640 comp rx=0 tx=0).stop 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 shutdown_connections 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb1373da30 0x55cb1373fe20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1372b1b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb1372ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c6bfff640 0 client.16995 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 --2- 172.21.15.42:0/923873049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724250 0x55cb1372a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 >> 172.21.15.42:0/923873049 conn(0x55cb1371e980 msgr2=0x55cb137206d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 shutdown_connections 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 1 -- 172.21.15.42:0/923873049 wait complete. 2024-03-12T04:20:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.007+0000 7f4c959b0740 20 client.16995 trim_cache size 0 max 16384 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 Processor -- start 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 -- start start 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb1372ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb13734a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 0x55cb13729000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb13611130 con 0x55cb13724250 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1373a640 con 0x55cb1372d750 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c959b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1372a520 con 0x55cb13724b70 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.019+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb1372ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb1372ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50982/0 (socket says 172.21.15.42:50982) 2024-03-12T04:20:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1801060668 learned_addr learned my addr 172.21.15.42:0/1801060668 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb13734a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1801060668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 msgr2=0x55cb13734a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb13734a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1801060668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 msgr2=0x55cb13729000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 0x55cb13729000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1801060668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb13626c20 con 0x55cb13724250 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb13734a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb1372ce60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800a010 tx=0x7f4c78009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62f122b5fea9d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c767fc640 1 -- 172.21.15.42:0/1801060668 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78003e00 con 0x55cb13724250 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c767fc640 1 -- 172.21.15.42:0/1801060668 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c78028070 con 0x55cb13724250 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.020+0000 7f4c767fc640 1 -- 172.21.15.42:0/1801060668 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78010690 con 0x55cb13724250 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- 172.21.15.42:0/1801060668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 msgr2=0x55cb1372ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb1372ce60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800a010 tx=0x7f4c78009920 comp rx=0 tx=0).stop 2024-03-12T04:20:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- 172.21.15.42:0/1801060668 shutdown_connections 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb1372d750 0x55cb13729000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724b70 0x55cb13734a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1801060668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb13724250 0x55cb1372ce60 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- 172.21.15.42:0/1801060668 >> 172.21.15.42:0/1801060668 conn(0x55cb1371e980 msgr2=0x55cb13744710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- 172.21.15.42:0/1801060668 shutdown_connections 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- 172.21.15.42:0/1801060668 wait complete. 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- start start 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 Processor -- start 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724250 0x55cb13790480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724b70 0x55cb137909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1378be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1378c540 con 0x55cb1372d750 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1378c6b0 con 0x55cb13724b70 2024-03-12T04:20:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1378c870 con 0x55cb13724250 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.021+0000 7f4c959b0740 10 client.0 _ll_register_callbacks cb 0x7f4c78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1378be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1378be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50990/0 (socket says 172.21.15.42:50990) 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1628796051 learned_addr learned my addr 172.21.15.42:0/1628796051 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724250 0x55cb13790480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724b70 0x55cb137909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1628796051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724250 msgr2=0x55cb13790480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724250 0x55cb13790480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724b70 msgr2=0x55cb137909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724b70 0x55cb137909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 -- 172.21.15.42:0/1628796051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb1368a6a0 con 0x55cb1372d750 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c8bfff640 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1378be40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4c780099f0 tx=0x7f4c78013390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aae04daa2ca89dc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c6bfff640 1 -- 172.21.15.42:0/1628796051 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78010430 con 0x55cb1372d750 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c6bfff640 1 -- 172.21.15.42:0/1628796051 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4c78010590 con 0x55cb1372d750 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c780035f0 con 0x55cb1372d750 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c6bfff640 1 -- 172.21.15.42:0/1628796051 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c7800f580 con 0x55cb1372d750 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c959b0740 10 client.17001 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb1364c8c0 con 0x55cb1372d750 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.022+0000 7f4c6bfff640 1 -- 172.21.15.42:0/1628796051 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c780035f0 con 0x55cb1372d750 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c6bfff640 1 -- 172.21.15.42:0/1628796051 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4c7802f470 con 0x55cb1372d750 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c959b0740 10 client.17001 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c959b0740 10 client.17001 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c959b0740 20 client.17001 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb13799050 0x55cb1379b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c959b0740 4 client.17001 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb13417890 con 0x55cb13799050 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb13799050 0x55cb1379b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c90aa4640 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb13799050 0x55cb1379b440 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4f5f9b48de548ab server_cookie=9c4baa5f661089f8 in_seq=0 out_seq=0 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c6bfff640 10 client.17001 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:20:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c6bfff640 1 -- 172.21.15.42:0/1628796051 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb13417890 con 0x55cb13799050 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c6bfff640 10 client.17001 handle_command_reply: tid=0 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.023+0000 7f4c959b0740 1 client.17001 shutdown 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 msgr2=0x55cb1378be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1378be40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4c780099f0 tx=0x7f4c78013390 comp rx=0 tx=0).stop 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 shutdown_connections 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55cb13799050 0x55cb1379b440 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1372d750 0x55cb1378be40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c6bfff640 0 client.17001 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cb13724b70 0x55cb137909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 --2- 172.21.15.42:0/1628796051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb13724250 0x55cb13790480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 >> 172.21.15.42:0/1628796051 conn(0x55cb1371e980 msgr2=0x55cb13728fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 shutdown_connections 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 1 -- 172.21.15.42:0/1628796051 wait complete. 2024-03-12T04:20:29.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c959b0740 20 client.17001 trim_cache size 0 max 16384 2024-03-12T04:20:29.034 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:20:29.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4c7c0a1ba0 msgr2=0x7f4c7c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4c7c0a1ba0 0x7f4c7c0a4060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4c8409a760 tx=0x7f4c8400af30 comp rx=0 tx=0).stop 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c10b790 msgr2=0x7f4c8c19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c10b790 0x7f4c8c19bf60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400ccb0 tx=0x7f4c8400a620 comp rx=0 tx=0).stop 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 shutdown_connections 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4c7c0a1ba0 0x7f4c7c0a4060 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c8c10b790 0x7f4c8c19bf60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c8c102ce0 0x7f4c8c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 --2- 172.21.15.42:0/2570402537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c8c1023c0 0x7f4c8c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 >> 172.21.15.42:0/2570402537 conn(0x7f4c8c076400 msgr2=0x7f4c8c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.024+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 shutdown_connections 2024-03-12T04:20:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.025+0000 7f4c92d08640 1 -- 172.21.15.42:0/2570402537 wait complete. 2024-03-12T04:20:29.038 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:20:29.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:20:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 -- 172.21.15.42:0/1982550534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74101d20 msgr2=0x7fcb74105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 --2- 172.21.15.42:0/1982550534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74101d20 0x7fcb74105d60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009930 tx=0x7fcb5c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:20:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 -- 172.21.15.42:0/1982550534 shutdown_connections 2024-03-12T04:20:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 --2- 172.21.15.42:0/1982550534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb74106440 0x7fcb7410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 --2- 172.21.15.42:0/1982550534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74101d20 0x7fcb74105d60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 --2- 172.21.15.42:0/1982550534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb74101370 0x7fcb74101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 -- 172.21.15.42:0/1982550534 >> 172.21.15.42:0/1982550534 conn(0x7fcb740fcc10 msgr2=0x7fcb740ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 -- 172.21.15.42:0/1982550534 shutdown_connections 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.140+0000 7fcb7962e640 1 -- 172.21.15.42:0/1982550534 wait complete. 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 Processor -- start 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 -- start start 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb74101370 0x7fcb74073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74106440 0x7fcb740739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb74073f00 0x7fcb741a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741a5ed0 con 0x7fcb74106440 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741a6040 con 0x7fcb74101370 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb7962e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741a6200 con 0x7fcb74073f00 2024-03-12T04:20:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74106440 0x7fcb740739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74106440 0x7fcb740739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51020/0 (socket says 172.21.15.42:51020) 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 -- 172.21.15.42:0/925348902 learned_addr learned my addr 172.21.15.42:0/925348902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb74101370 0x7fcb74073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb737fe640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb74073f00 0x7fcb741a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 -- 172.21.15.42:0/925348902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb74073f00 msgr2=0x7fcb741a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb74073f00 0x7fcb741a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 -- 172.21.15.42:0/925348902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb74101370 msgr2=0x7fcb74073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb74101370 0x7fcb74073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.141+0000 7fcb6bfff640 1 -- 172.21.15.42:0/925348902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009560 con 0x7fcb74106440 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.142+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74106440 0x7fcb740739c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c000c00 tx=0x7fcb5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.142+0000 7fcb70ff9640 1 -- 172.21.15.42:0/925348902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c02cb70 con 0x7fcb74106440 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.142+0000 7fcb70ff9640 1 -- 172.21.15.42:0/925348902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb5c02de90 con 0x7fcb74106440 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.142+0000 7fcb70ff9640 1 -- 172.21.15.42:0/925348902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0053c0 con 0x7fcb74106440 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.142+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb741a6480 con 0x7fcb74106440 2024-03-12T04:20:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.142+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb741a6990 con 0x7fcb74106440 2024-03-12T04:20:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.144+0000 7fcb70ff9640 1 -- 172.21.15.42:0/925348902 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fcb5c005520 con 0x7fcb74106440 2024-03-12T04:20:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.144+0000 7fcb7c201740 1 Processor -- start 2024-03-12T04:20:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.144+0000 7fcb7c201740 1 -- start start 2024-03-12T04:20:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb70ff9640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb600a1ba0 0x7fcb600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb7c201740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a651ad960 con 0x561a651f2b70 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb7c201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a65161310 con 0x561a651fb750 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb7c201740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6515e840 con 0x561a651f2250 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb737fe640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb600a1ba0 0x7fcb600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651fb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51032/0 (socket says 172.21.15.42:51032) 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 -- 172.21.15.42:0/3425550648 learned_addr learned my addr 172.21.15.42:0/3425550648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb70ff9640 1 -- 172.21.15.42:0/925348902 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb5c0ef860 con 0x7fcb74106440 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb737fe640 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 -- 172.21.15.42:0/3425550648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 msgr2=0x561a651f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 -- 172.21.15.42:0/3425550648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 msgr2=0x561a651fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.145+0000 7fcb6bfff640 1 -- 172.21.15.42:0/3425550648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a6511c060 con 0x561a651f2b70 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.146+0000 7fcb737fe640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb600a1ba0 0x7fcb600a4060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c005fd0 tx=0x7fcb6c014b70 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.146+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651fb210 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b8f60 tx=0x7fcb5c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2344344a9879e482 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.146+0000 7fcb697fa640 1 -- 172.21.15.42:0/3425550648 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0bc0d0 con 0x561a651f2b70 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.146+0000 7fcb697fa640 1 -- 172.21.15.42:0/3425550648 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb5c0bbef0 con 0x561a651f2b70 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.146+0000 7fcb697fa640 1 -- 172.21.15.42:0/3425550648 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0c4d30 con 0x561a651f2b70 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.146+0000 7fcb7c201740 1 -- 172.21.15.42:0/3425550648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 msgr2=0x561a651fb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.146+0000 7fcb7c201740 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651fb210 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b8f60 tx=0x7fcb5c009260 comp rx=0 tx=0).stop 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- 172.21.15.42:0/3425550648 shutdown_connections 2024-03-12T04:20:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651fb210 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 --2- 172.21.15.42:0/3425550648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- 172.21.15.42:0/3425550648 >> 172.21.15.42:0/3425550648 conn(0x561a651ec980 msgr2=0x561a651ecd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- 172.21.15.42:0/3425550648 shutdown_connections 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- 172.21.15.42:0/3425550648 wait complete. 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- start start 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 Processor -- start 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651f9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6515a020 con 0x561a651f2b70 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a64e2d430 con 0x561a651fb750 2024-03-12T04:20:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a64de9e00 con 0x561a651f2250 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb7c201740 10 client.0 _ll_register_callbacks cb 0x561a65101cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51048/0 (socket says 172.21.15.42:51048) 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1644513591 learned_addr learned my addr 172.21.15.42:0/1644513591 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb737fe640 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.147+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 msgr2=0x561a651f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1644513591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 msgr2=0x561a651f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1644513591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb741a6990 con 0x561a651f2b70 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651f8c30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b89f0 tx=0x7fcb5c0dd640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654ef4e33024a2c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1644513591 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0d52c0 con 0x561a651f2b70 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1644513591 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb5c0bc590 con 0x561a651f2b70 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a64e597e0 con 0x561a651f2b70 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1644513591 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5c0bcc90 con 0x561a651f2b70 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 10 client.17013 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a651586a0 con 0x561a651f2b70 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1644513591 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561a64e597e0 con 0x561a651f2b70 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1644513591 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb5c03c070 con 0x561a651f2b70 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 10 client.17013 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 10 client.17013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 20 client.17013 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a6520b420 0x561a6520d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 4 client.17013 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.148+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a64e96720 con 0x561a6520b420 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.149+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a6520b420 0x561a6520d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.149+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a6520b420 0x561a6520d810 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77a426eb95608c3 server_cookie=5104b51cd40b8b16 in_seq=0 out_seq=0 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.149+0000 7fcb5a7fc640 10 client.17013 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1644513591 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a64e96720 con 0x561a6520b420 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb5a7fc640 10 client.17013 handle_command_reply: tid=0 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 client.17013 shutdown 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 msgr2=0x561a651f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651f8c30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b89f0 tx=0x7fcb5c0dd640 comp rx=0 tx=0).stop 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 shutdown_connections 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a6520b420 0x561a6520d810 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651fb750 0x561a651f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb5a7fc640 0 client.17013 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a651f8c30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1644513591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2250 0x561a651f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 >> 172.21.15.42:0/1644513591 conn(0x561a651ec980 msgr2=0x561a651ee6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 shutdown_connections 2024-03-12T04:20:29.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 1 -- 172.21.15.42:0/1644513591 wait complete. 2024-03-12T04:20:29.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.150+0000 7fcb7c201740 20 client.17013 trim_cache size 0 max 16384 2024-03-12T04:20:29.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 Processor -- start 2024-03-12T04:20:29.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 -- start start 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a651fac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2b70 0x561a651f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651fb750 0x561a651f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a650df130 con 0x561a651fb750 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a64e597e0 con 0x561a651f2250 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb7c201740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a651dea40 con 0x561a651f2b70 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651fb750 0x561a651f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651fb750 0x561a651f5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51050/0 (socket says 172.21.15.42:51050) 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 -- 172.21.15.42:0/2583834254 learned_addr learned my addr 172.21.15.42:0/2583834254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2b70 0x561a651f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a651fac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 -- 172.21.15.42:0/2583834254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2b70 msgr2=0x561a651f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2b70 0x561a651f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 -- 172.21.15.42:0/2583834254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 msgr2=0x561a651fac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a651fac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.162+0000 7fcb737fe640 1 -- 172.21.15.42:0/2583834254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a650f4c20 con 0x561a651fb750 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb737fe640 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651fb750 0x561a651f5970 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c012d10 tx=0x7fcb6c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a43eb3a5925b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb68ff9640 1 -- 172.21.15.42:0/2583834254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0045c0 con 0x561a651fb750 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb68ff9640 1 -- 172.21.15.42:0/2583834254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb6c01a070 con 0x561a651fb750 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb68ff9640 1 -- 172.21.15.42:0/2583834254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c022400 con 0x561a651fb750 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 -- 172.21.15.42:0/2583834254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651fb750 msgr2=0x561a651f5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651fb750 0x561a651f5970 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c012d10 tx=0x7fcb6c01d000 comp rx=0 tx=0).stop 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 -- 172.21.15.42:0/2583834254 shutdown_connections 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651fb750 0x561a651f5970 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651f2b70 0x561a651f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 --2- 172.21.15.42:0/2583834254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a651fac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 -- 172.21.15.42:0/2583834254 >> 172.21.15.42:0/2583834254 conn(0x561a651ec980 msgr2=0x561a651ebb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 -- 172.21.15.42:0/2583834254 shutdown_connections 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.163+0000 7fcb7c201740 1 -- 172.21.15.42:0/2583834254 wait complete. 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 -- start start 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 Processor -- start 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a65201730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a65201c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651fb750 0x561a6525b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6525bf20 con 0x561a651f2b70 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6525c090 con 0x561a651f2250 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6525c200 con 0x561a651fb750 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb7c201740 10 client.0 _ll_register_callbacks cb 0x7fcb6400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a65201c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a65201c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51054/0 (socket says 172.21.15.42:51054) 2024-03-12T04:20:29.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1557821913 learned_addr learned my addr 172.21.15.42:0/1557821913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb737fe640 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651fb750 0x561a6525b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a65201730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651fb750 msgr2=0x561a6525b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651fb750 0x561a6525b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1557821913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 msgr2=0x561a65201730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a65201730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.164+0000 7fcb6bfff640 1 -- 172.21.15.42:0/1557821913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c012a50 con 0x561a651f2b70 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb6bfff640 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a65201c70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b8f60 tx=0x7fcb5c0c4c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85cc85725592809 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1557821913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0dd6b0 con 0x561a651f2b70 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1557821913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb5c0bcc90 con 0x561a651f2b70 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a651e80f0 con 0x561a651f2b70 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1557821913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5c0b8830 con 0x561a651f2b70 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 10 client.17019 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:29.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a6511a8c0 con 0x561a651f2b70 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1557821913 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561a651e80f0 con 0x561a651f2b70 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1557821913 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb5c03c070 con 0x561a651f2b70 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 10 client.17019 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 10 client.17019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 20 client.17019 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a65263510 0x561a65265900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 4 client.17019 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fcb5c0bbef0 con 0x561a65263510 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.165+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a65263510 0x561a65265900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb5a7fc640 10 client.17019 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb72ffd640 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a65263510 0x561a65265900 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=340257133b3cdec9 server_cookie=3fdac13ce61ed76e in_seq=0 out_seq=0 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb5a7fc640 1 -- 172.21.15.42:0/1557821913 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fcb5c0bbef0 con 0x561a65263510 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb5a7fc640 10 client.17019 handle_command_reply: tid=0 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 client.17019 shutdown 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 msgr2=0x561a65201c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a65201c70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b8f60 tx=0x7fcb5c0c4c40 comp rx=0 tx=0).stop 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 shutdown_connections 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x561a65263510 0x561a65265900 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561a651fb750 0x561a6525b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a651f2b70 0x561a65201c70 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 --2- 172.21.15.42:0/1557821913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a651f2250 0x561a65201730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 >> 172.21.15.42:0/1557821913 conn(0x561a651ec980 msgr2=0x561a651f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb5a7fc640 0 client.17019 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 shutdown_connections 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 1 -- 172.21.15.42:0/1557821913 wait complete. 2024-03-12T04:20:29.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.166+0000 7fcb7c201740 20 client.17019 trim_cache size 0 max 16384 2024-03-12T04:20:29.177 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:20:29.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb600a1ba0 msgr2=0x7fcb600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb600a1ba0 0x7fcb600a4060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c005fd0 tx=0x7fcb6c014b70 comp rx=0 tx=0).stop 2024-03-12T04:20:29.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74106440 msgr2=0x7fcb740739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74106440 0x7fcb740739c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c000c00 tx=0x7fcb5c004270 comp rx=0 tx=0).stop 2024-03-12T04:20:29.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 shutdown_connections 2024-03-12T04:20:29.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb600a1ba0 0x7fcb600a4060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb74073f00 0x7fcb741a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74106440 0x7fcb740739c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 --2- 172.21.15.42:0/925348902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb74101370 0x7fcb74073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 >> 172.21.15.42:0/925348902 conn(0x7fcb740fcc10 msgr2=0x7fcb740fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 shutdown_connections 2024-03-12T04:20:29.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.167+0000 7fcb7962e640 1 -- 172.21.15.42:0/925348902 wait complete. 2024-03-12T04:20:29.180 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:20:29.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:20:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 -- 172.21.15.42:0/3227557710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c100260 msgr2=0x7f298c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 --2- 172.21.15.42:0/3227557710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c100260 0x7f298c100640 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2988009930 tx=0x7f298802bc30 comp rx=0 tx=0).stop 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 -- 172.21.15.42:0/3227557710 shutdown_connections 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 --2- 172.21.15.42:0/3227557710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f298c105330 0x7f298c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 --2- 172.21.15.42:0/3227557710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298c100c10 0x7f298c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 --2- 172.21.15.42:0/3227557710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c100260 0x7f298c100640 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 -- 172.21.15.42:0/3227557710 >> 172.21.15.42:0/3227557710 conn(0x7f298c077180 msgr2=0x7f298c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 -- 172.21.15.42:0/3227557710 shutdown_connections 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 -- 172.21.15.42:0/3227557710 wait complete. 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 Processor -- start 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.283+0000 7f29940f4640 1 -- start start 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298c100260 0x7f298c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f298c105330 0x7f298c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c19bfd0 0x7f298c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c19c9c0 con 0x7f298c19bfd0 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c19cb30 con 0x7f298c100260 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c19cca0 con 0x7f298c105330 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c19bfd0 0x7f298c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c19bfd0 0x7f298c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51078/0 (socket says 172.21.15.42:51078) 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 -- 172.21.15.42:0/1790962029 learned_addr learned my addr 172.21.15.42:0/1790962029 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 -- 172.21.15.42:0/1790962029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f298c105330 msgr2=0x7f298c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f298c105330 0x7f298c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 -- 172.21.15.42:0/1790962029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298c100260 msgr2=0x7f298c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298c100260 0x7f298c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 -- 172.21.15.42:0/1790962029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988009560 con 0x7f298c19bfd0 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2992691640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c19bfd0 0x7f298c19c480 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2984009fd0 tx=0x7f298400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2982ffd640 1 -- 172.21.15.42:0/1790962029 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298400bea0 con 0x7f298c19bfd0 2024-03-12T04:20:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2982ffd640 1 -- 172.21.15.42:0/1790962029 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2984009940 con 0x7f298c19bfd0 2024-03-12T04:20:29.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f2982ffd640 1 -- 172.21.15.42:0/1790962029 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29840106c0 con 0x7f298c19bfd0 2024-03-12T04:20:29.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c1a4280 con 0x7f298c19bfd0 2024-03-12T04:20:29.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.284+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c1a4730 con 0x7f298c19bfd0 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.287+0000 7f2982ffd640 1 -- 172.21.15.42:0/1790962029 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f2984003970 con 0x7f298c19bfd0 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.287+0000 7f2996da6740 1 Processor -- start 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.287+0000 7f2996da6740 1 -- start start 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2996da6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1d9250 0x556c9d1d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2996da6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2996da6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1e2750 0x556c9d1e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2996da6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d194960 con 0x556c9d1d9b70 2024-03-12T04:20:29.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2996da6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d148310 con 0x556c9d1d9250 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2996da6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d145840 con 0x556c9d1e2750 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51088/0 (socket says 172.21.15.42:51088) 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 -- 172.21.15.42:0/3259294363 learned_addr learned my addr 172.21.15.42:0/3259294363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2982ffd640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29780a1c20 0x7f29780a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f299168f640 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1d9250 0x556c9d1d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f299168f640 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1e2750 0x556c9d1e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2992691640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29780a1c20 0x7f29780a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 -- 172.21.15.42:0/3259294363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1e2750 msgr2=0x556c9d1e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1e2750 0x556c9d1e4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 -- 172.21.15.42:0/3259294363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1d9250 msgr2=0x556c9d1d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1d9250 0x556c9d1d9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2991e90640 1 -- 172.21.15.42:0/3259294363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c9d103060 con 0x556c9d1d9b70 2024-03-12T04:20:29.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f299168f640 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1e2750 0x556c9d1e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.288+0000 7f2982ffd640 1 -- 172.21.15.42:0/1790962029 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f29840cd4d0 con 0x7f298c19bfd0 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2991e90640 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e2210 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2988009430 tx=0x7f2988005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac59d42cc98cd857 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2992691640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29780a1c20 0x7f29780a40e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f298409a0e0 tx=0x7f29840c4810 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2977fff640 1 -- 172.21.15.42:0/3259294363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988004840 con 0x556c9d1d9b70 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2977fff640 1 -- 172.21.15.42:0/3259294363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f298802de90 con 0x556c9d1d9b70 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2977fff640 1 -- 172.21.15.42:0/3259294363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988003850 con 0x556c9d1d9b70 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2996da6740 1 -- 172.21.15.42:0/3259294363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 msgr2=0x556c9d1e2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2996da6740 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e2210 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2988009430 tx=0x7f2988005e80 comp rx=0 tx=0).stop 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2996da6740 1 -- 172.21.15.42:0/3259294363 shutdown_connections 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2996da6740 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1e2750 0x556c9d1e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2996da6740 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e2210 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2996da6740 1 --2- 172.21.15.42:0/3259294363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1d9250 0x556c9d1d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.289+0000 7f2996da6740 1 -- 172.21.15.42:0/3259294363 >> 172.21.15.42:0/3259294363 conn(0x556c9d1d3980 msgr2=0x556c9d1d3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 -- 172.21.15.42:0/3259294363 shutdown_connections 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 -- 172.21.15.42:0/3259294363 wait complete. 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 -- start start 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 Processor -- start 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d141020 con 0x556c9d1d9b70 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9ce14430 con 0x556c9d1e2750 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9cdd0e00 con 0x556c9d1d9250 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2996da6740 10 client.0 _ll_register_callbacks cb 0x556c9d0e8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1dfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51098/0 (socket says 172.21.15.42:51098) 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f2991e90640 1 -- 172.21.15.42:0/3805040260 learned_addr learned my addr 172.21.15.42:0/3805040260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.290+0000 7f299168f640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2991e90640 1 -- 172.21.15.42:0/3805040260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 msgr2=0x556c9d1df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f299168f640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2991e90640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1df7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2991e90640 1 -- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 msgr2=0x556c9d1e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2991e90640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2991e90640 1 -- 172.21.15.42:0/3805040260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c0682e0 con 0x556c9d1d9b70 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f299168f640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2991e90640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1dfd00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f298802ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bef418581c7308 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f29757fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f29757fa640 1 -- 172.21.15.42:0/3805040260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988004fc0 con 0x556c9d1d9b70 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f29757fa640 1 -- 172.21.15.42:0/3805040260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2988005120 con 0x556c9d1d9b70 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c9ce407e0 con 0x556c9d1d9b70 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f29757fa640 1 -- 172.21.15.42:0/3805040260 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2988004510 con 0x556c9d1d9b70 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2996da6740 10 client.17031 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556c9d13f6a0 con 0x556c9d1d9b70 2024-03-12T04:20:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f29757fa640 1 -- 172.21.15.42:0/3805040260 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x556c9ce407e0 con 0x556c9d1d9b70 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.291+0000 7f29757fa640 1 -- 172.21.15.42:0/3805040260 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f298803c070 con 0x556c9d1d9b70 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f2996da6740 10 client.17031 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f2996da6740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f2996da6740 20 client.17031 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f2996da6740 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d1f29e0 0x556c9d1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f2996da6740 4 client.17031 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c9ce7d720 con 0x556c9d1f29e0 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f299168f640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d1f29e0 0x556c9d1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f299168f640 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d1f29e0 0x556c9d1f4e00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6a4af01f12c5b4c server_cookie=caa67403417e7239 in_seq=0 out_seq=0 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.292+0000 7f29757fa640 10 client.17031 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f29757fa640 1 -- 172.21.15.42:0/3805040260 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c9ce7d720 con 0x556c9d1f29e0 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f29757fa640 10 client.17031 handle_command_reply: tid=0 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 client.17031 shutdown 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 msgr2=0x556c9d1dfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1dfd00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f298802ca20 comp rx=0 tx=0).stop 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 shutdown_connections 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d1f29e0 0x556c9d1f4e00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1dfd00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 --2- 172.21.15.42:0/3805040260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f29757fa640 0 client.17031 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 >> 172.21.15.42:0/3805040260 conn(0x556c9d1d3980 msgr2=0x556c9d1d56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 shutdown_connections 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 1 -- 172.21.15.42:0/3805040260 wait complete. 2024-03-12T04:20:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.293+0000 7f2996da6740 20 client.17031 trim_cache size 0 max 16384 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 Processor -- start 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 -- start start 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d0c6130 con 0x556c9d1d9b70 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d1fa010 con 0x556c9d1e2750 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2996da6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d1f2790 con 0x556c9d1d9250 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51114/0 (socket says 172.21.15.42:51114) 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.305+0000 7f2991e90640 1 -- 172.21.15.42:0/619104867 learned_addr learned my addr 172.21.15.42:0/619104867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f2991e90640 1 -- 172.21.15.42:0/619104867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 msgr2=0x556c9d1e1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f2991e90640 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f2991e90640 1 -- 172.21.15.42:0/619104867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 msgr2=0x556c9d1daaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f299168f640 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f2991e90640 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1daaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f2991e90640 1 -- 172.21.15.42:0/619104867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c9d0dbc20 con 0x556c9d1d9b70 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f2991e90640 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e0060 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f298802c170 tx=0x7f298802c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e32ceddfaeb0c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f29777fe640 1 -- 172.21.15.42:0/619104867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298802ca90 con 0x556c9d1d9b70 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f29777fe640 1 -- 172.21.15.42:0/619104867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f298804c070 con 0x556c9d1d9b70 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.306+0000 7f29777fe640 1 -- 172.21.15.42:0/619104867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988051ac0 con 0x556c9d1d9b70 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- 172.21.15.42:0/619104867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 msgr2=0x556c9d1e0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e0060 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f298802c170 tx=0x7f298802c980 comp rx=0 tx=0).stop 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- 172.21.15.42:0/619104867 shutdown_connections 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1daaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e0060 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 --2- 172.21.15.42:0/619104867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1e1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- 172.21.15.42:0/619104867 >> 172.21.15.42:0/619104867 conn(0x556c9d1d3980 msgr2=0x556c9d1cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- 172.21.15.42:0/619104867 shutdown_connections 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- 172.21.15.42:0/619104867 wait complete. 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- start start 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 Processor -- start 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d1e8c80 con 0x556c9d1d9b70 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d248ef0 con 0x556c9d1e2750 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9d249030 con 0x556c9d1d9250 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.307+0000 7f2996da6740 10 client.0 _ll_register_callbacks cb 0x7f2988009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51122/0 (socket says 172.21.15.42:51122) 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 -- 172.21.15.42:0/3802004360 learned_addr learned my addr 172.21.15.42:0/3802004360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f299168f640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f299168f640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 -- 172.21.15.42:0/3802004360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 msgr2=0x556c9d1e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 -- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 msgr2=0x556c9d1e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 -- 172.21.15.42:0/3802004360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c9d13f6a0 con 0x556c9d1d9b70 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f299168f640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2991e90640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e7e40 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2988041000 tx=0x7f298802c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d67a80faaf734da server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f29757fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f29757fa640 1 -- 172.21.15.42:0/3802004360 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988051c50 con 0x556c9d1d9b70 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f29757fa640 1 -- 172.21.15.42:0/3802004360 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2988051db0 con 0x556c9d1d9b70 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f298802de90 con 0x556c9d1d9b70 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f29757fa640 1 -- 172.21.15.42:0/3802004360 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2988034440 con 0x556c9d1d9b70 2024-03-12T04:20:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2996da6740 10 client.17037 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.308+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556c9d1018c0 con 0x556c9d1d9b70 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f29757fa640 1 -- 172.21.15.42:0/3802004360 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2988071020 con 0x556c9d1d9b70 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f29757fa640 1 -- 172.21.15.42:0/3802004360 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f298803c070 con 0x556c9d1d9b70 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f2996da6740 10 client.17037 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f2996da6740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f2996da6740 20 client.17037 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f2996da6740 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d24a340 0x556c9d24c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f2996da6740 4 client.17037 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c9cecc890 con 0x556c9d24a340 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f299168f640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d24a340 0x556c9d24c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f299168f640 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d24a340 0x556c9d24c730 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c6281147c882260 server_cookie=67e55d6ce71d7b53 in_seq=0 out_seq=0 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.309+0000 7f29757fa640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29757fa640 1 -- 172.21.15.42:0/3802004360 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c9cecc890 con 0x556c9d24a340 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29757fa640 10 client.17037 handle_command_reply: tid=0 2024-03-12T04:20:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 client.17037 shutdown 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 msgr2=0x556c9d1e7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e7e40 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2988041000 tx=0x7f298802c6a0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 shutdown_connections 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556c9d24a340 0x556c9d24c730 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c9d1e2750 0x556c9d1e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556c9d1d9b70 0x556c9d1e7e40 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 --2- 172.21.15.42:0/3802004360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c9d1d9250 0x556c9d1e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29757fa640 0 client.17037 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 >> 172.21.15.42:0/3802004360 conn(0x556c9d1d3980 msgr2=0x556c9d1daaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 shutdown_connections 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 1 -- 172.21.15.42:0/3802004360 wait complete. 2024-03-12T04:20:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f2996da6740 20 client.17037 trim_cache size 0 max 16384 2024-03-12T04:20:29.320 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:20:29.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29780a1c20 msgr2=0x7f29780a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29780a1c20 0x7f29780a40e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f298409a0e0 tx=0x7f29840c4810 comp rx=0 tx=0).stop 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c19bfd0 msgr2=0x7f298c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c19bfd0 0x7f298c19c480 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2984009fd0 tx=0x7f298400b260 comp rx=0 tx=0).stop 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 shutdown_connections 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29780a1c20 0x7f29780a40e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f298c19bfd0 0x7f298c19c480 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f298c105330 0x7f298c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 --2- 172.21.15.42:0/1790962029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298c100260 0x7f298c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 >> 172.21.15.42:0/1790962029 conn(0x7f298c077180 msgr2=0x7f298c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 shutdown_connections 2024-03-12T04:20:29.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:29.310+0000 7f29940f4640 1 -- 172.21.15.42:0/1790962029 wait complete. 2024-03-12T04:20:30.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:20:30.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:20:30.324 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:20:30.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 -- 172.21.15.42:0/4124675477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c0ffcb0 msgr2=0x7fe67c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 --2- 172.21.15.42:0/4124675477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c0ffcb0 0x7fe67c100090 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe678009930 tx=0x7fe67802bc30 comp rx=0 tx=0).stop 2024-03-12T04:20:30.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 -- 172.21.15.42:0/4124675477 shutdown_connections 2024-03-12T04:20:30.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 --2- 172.21.15.42:0/4124675477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe67c109930 0x7fe67c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 --2- 172.21.15.42:0/4124675477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe67c100660 0x7fe67c109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 --2- 172.21.15.42:0/4124675477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c0ffcb0 0x7fe67c100090 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 -- 172.21.15.42:0/4124675477 >> 172.21.15.42:0/4124675477 conn(0x7fe67c0753c0 msgr2=0x7fe67c0757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 -- 172.21.15.42:0/4124675477 shutdown_connections 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.441+0000 7fe683963640 1 -- 172.21.15.42:0/4124675477 wait complete. 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 Processor -- start 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 -- start start 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe67c0ffcb0 0x7fe67c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe67c109930 0x7fe67c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c1997c0 0x7fe67c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c19a270 con 0x7fe67c1997c0 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c19a3e0 con 0x7fe67c0ffcb0 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe683963640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c19a550 con 0x7fe67c109930 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe681f00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c1997c0 0x7fe67c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe681f00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c1997c0 0x7fe67c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51152/0 (socket says 172.21.15.42:51152) 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe681f00640 1 -- 172.21.15.42:0/1463023709 learned_addr learned my addr 172.21.15.42:0/1463023709 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe67c0ffcb0 0x7fe67c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.442+0000 7fe680efe640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe67c109930 0x7fe67c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe681f00640 1 -- 172.21.15.42:0/1463023709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe67c109930 msgr2=0x7fe67c19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe681f00640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe67c109930 0x7fe67c19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe681f00640 1 -- 172.21.15.42:0/1463023709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe67c0ffcb0 msgr2=0x7fe67c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe681f00640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe67c0ffcb0 0x7fe67c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe681f00640 1 -- 172.21.15.42:0/1463023709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe678009560 con 0x7fe67c1997c0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe681f00640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c1997c0 0x7fe67c199c70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe67400eaa0 tx=0x7fe67400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe6727fc640 1 -- 172.21.15.42:0/1463023709 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400cdb0 con 0x7fe67c1997c0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe6727fc640 1 -- 172.21.15.42:0/1463023709 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe674005020 con 0x7fe67c1997c0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe6727fc640 1 -- 172.21.15.42:0/1463023709 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6740116b0 con 0x7fe67c1997c0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c1a6500 con 0x7fe67c1997c0 2024-03-12T04:20:30.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.443+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c1a69e0 con 0x7fe67c1997c0 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.445+0000 7fe6727fc640 1 -- 172.21.15.42:0/1463023709 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe674005190 con 0x7fe67c1997c0 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 Processor -- start 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 -- start start 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1141760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf114a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf114a8b0 0x55faf114cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe6727fc640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6680a1ba0 0x7fe6680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf10aff90 con 0x55faf1141d30 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf10ad4c0 con 0x55faf1141380 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe68660b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf10a8ca0 con 0x55faf114a8b0 2024-03-12T04:20:30.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe6816ff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf114a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe680efe640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6680a1ba0 0x7fe6680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.446+0000 7fe6727fc640 1 -- 172.21.15.42:0/1463023709 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe6740cd930 con 0x7fe67c1997c0 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf114a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51168/0 (socket says 172.21.15.42:51168) 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 -- 172.21.15.42:0/1066956954 learned_addr learned my addr 172.21.15.42:0/1066956954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe680efe640 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe681f00640 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf114a8b0 0x55faf114cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 -- 172.21.15.42:0/1066956954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf114a8b0 msgr2=0x55faf114cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf114a8b0 0x55faf114cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 -- 172.21.15.42:0/1066956954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 msgr2=0x55faf1141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 -- 172.21.15.42:0/1066956954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faf106b0b0 con 0x55faf1141d30 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe681f00640 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf114a8b0 0x55faf114cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe680efe640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6680a1ba0 0x7fe6680a4060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe67c109300 tx=0x7fe66c00aa50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf114a2e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe678005ab0 tx=0x7fe678004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae18f70119d26f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6677fe640 1 -- 172.21.15.42:0/1066956954 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678004a20 con 0x55faf1141d30 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6677fe640 1 -- 172.21.15.42:0/1066956954 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe67802de90 con 0x55faf1141d30 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.447+0000 7fe6677fe640 1 -- 172.21.15.42:0/1066956954 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678003850 con 0x55faf1141d30 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 -- 172.21.15.42:0/1066956954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 msgr2=0x55faf114a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf114a2e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe678005ab0 tx=0x7fe678004770 comp rx=0 tx=0).stop 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 -- 172.21.15.42:0/1066956954 shutdown_connections 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf114a8b0 0x55faf114cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf114a2e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 --2- 172.21.15.42:0/1066956954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 -- 172.21.15.42:0/1066956954 >> 172.21.15.42:0/1066956954 conn(0x55faf113b840 msgr2=0x55faf113bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 -- 172.21.15.42:0/1066956954 shutdown_connections 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 -- 172.21.15.42:0/1066956954 wait complete. 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 -- start start 2024-03-12T04:20:30.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.448+0000 7fe68660b740 1 Processor -- start 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe68660b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe68660b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf1147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe68660b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf114a8b0 0x55faf1148200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe68660b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf104df40 con 0x55faf114a8b0 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe68660b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf0d7c4b0 con 0x55faf1141380 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe68660b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf102e670 con 0x55faf1141d30 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe68660b740 10 client.0 _ll_register_callbacks cb 0x55faf109bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf114a8b0 0x55faf1148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf114a8b0 0x55faf1148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51172/0 (socket says 172.21.15.42:51172) 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 -- 172.21.15.42:0/1620809945 learned_addr learned my addr 172.21.15.42:0/1620809945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe680efe640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf1147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 -- 172.21.15.42:0/1620809945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 msgr2=0x55faf1147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf1147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 -- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 msgr2=0x55faf1147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe681f00640 1 -- 172.21.15.42:0/1620809945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c110b20 con 0x55faf114a8b0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.449+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe681f00640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf114a8b0 0x55faf1148200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe67409a300 tx=0x7fe674097ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcb5397dc8eacdb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe664ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe664ff9640 1 -- 172.21.15.42:0/1620809945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67409e6a0 con 0x55faf114a8b0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe664ff9640 1 -- 172.21.15.42:0/1620809945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe6740a1550 con 0x55faf114a8b0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faf1082f40 con 0x55faf114a8b0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe664ff9640 1 -- 172.21.15.42:0/1620809945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6740a2aa0 con 0x55faf114a8b0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 10 client.17049 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55faf10a7320 con 0x55faf114a8b0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe664ff9640 1 -- 172.21.15.42:0/1620809945 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55faf1082f40 con 0x55faf114a8b0 2024-03-12T04:20:30.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe664ff9640 1 -- 172.21.15.42:0/1620809945 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe674018030 con 0x55faf114a8b0 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 10 client.17049 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 20 client.17049 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf115ae30 0x55faf115d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 4 client.17049 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.450+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faf109f310 con 0x55faf115ae30 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.451+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf115ae30 0x55faf115d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.451+0000 7fe664ff9640 10 client.17049 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.451+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf115ae30 0x55faf115d270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb848231a2f0580e server_cookie=d568b9c8c75afa7f in_seq=0 out_seq=0 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe664ff9640 1 -- 172.21.15.42:0/1620809945 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55faf109f310 con 0x55faf115ae30 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe664ff9640 10 client.17049 handle_command_reply: tid=0 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 client.17049 shutdown 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf114a8b0 msgr2=0x55faf1148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf114a8b0 0x55faf1148200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe67409a300 tx=0x7fe674097ef0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 shutdown_connections 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf115ae30 0x55faf115d270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf114a8b0 0x55faf1148200 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf1147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe664ff9640 0 client.17049 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 --2- 172.21.15.42:0/1620809945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf1141380 0x55faf1147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 >> 172.21.15.42:0/1620809945 conn(0x55faf113b840 msgr2=0x55faf113d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 shutdown_connections 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 1 -- 172.21.15.42:0/1620809945 wait complete. 2024-03-12T04:20:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.452+0000 7fe68660b740 20 client.17049 trim_cache size 0 max 16384 2024-03-12T04:20:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.467+0000 7fe68660b740 1 Processor -- start 2024-03-12T04:20:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe68660b740 1 -- start start 2024-03-12T04:20:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe68660b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141380 0x55faf1148020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe68660b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf11518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe68660b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe68660b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf113c640 con 0x55faf1141d30 2024-03-12T04:20:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe68660b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf1137180 con 0x55faf114a8b0 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe68660b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf113b370 con 0x55faf1141380 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf11518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf11518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51186/0 (socket says 172.21.15.42:51186) 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 -- 172.21.15.42:0/279526298 learned_addr learned my addr 172.21.15.42:0/279526298 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe6816ff640 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141380 0x55faf1148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe681f00640 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 -- 172.21.15.42:0/279526298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141380 msgr2=0x55faf1148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141380 0x55faf1148020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 -- 172.21.15.42:0/279526298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 msgr2=0x55faf1151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 -- 172.21.15.42:0/279526298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faf1043c00 con 0x55faf1141d30 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe681f00640 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.468+0000 7fe680efe640 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf11518e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0058c0 tx=0x7fe66c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311f2dd12775e719 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe666ffd640 1 -- 172.21.15.42:0/279526298 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00fe20 con 0x55faf1141d30 2024-03-12T04:20:30.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe666ffd640 1 -- 172.21.15.42:0/279526298 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe66c01fe10 con 0x55faf1141d30 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe666ffd640 1 -- 172.21.15.42:0/279526298 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c029b00 con 0x55faf1141d30 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 -- 172.21.15.42:0/279526298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 msgr2=0x55faf11518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf11518e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0058c0 tx=0x7fe66c004730 comp rx=0 tx=0).stop 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 -- 172.21.15.42:0/279526298 shutdown_connections 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141d30 0x55faf11518e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 --2- 172.21.15.42:0/279526298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141380 0x55faf1148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 -- 172.21.15.42:0/279526298 >> 172.21.15.42:0/279526298 conn(0x55faf113b840 msgr2=0x55faf113a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 -- 172.21.15.42:0/279526298 shutdown_connections 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 -- 172.21.15.42:0/279526298 wait complete. 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.469+0000 7fe68660b740 1 -- start start 2024-03-12T04:20:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 1 Processor -- start 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141380 0x55faf11aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf11ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf1151710 con 0x55faf1141380 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf1151880 con 0x55faf114a8b0 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf1151a40 con 0x55faf1141d30 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe68660b740 10 client.0 _ll_register_callbacks cb 0x7fe678005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141380 0x55faf11aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141380 0x55faf11aaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51190/0 (socket says 172.21.15.42:51190) 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 -- 172.21.15.42:0/1469411958 learned_addr learned my addr 172.21.15.42:0/1469411958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe680efe640 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf11ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe681f00640 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 -- 172.21.15.42:0/1469411958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 msgr2=0x55faf11ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf11ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 -- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 msgr2=0x55faf1151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.470+0000 7fe6816ff640 1 -- 172.21.15.42:0/1469411958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c019040 con 0x55faf1141380 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe6816ff640 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141380 0x55faf11aaba0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe67802c170 tx=0x7fe67802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f6cf016d474fdf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe664ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe664ff9640 1 -- 172.21.15.42:0/1469411958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678050b70 con 0x55faf1141380 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe664ff9640 1 -- 172.21.15.42:0/1469411958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe67804e070 con 0x55faf1141380 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faf10ce900 con 0x55faf1141380 2024-03-12T04:20:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe664ff9640 1 -- 172.21.15.42:0/1469411958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe678050cd0 con 0x55faf1141380 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 10 client.17055 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55faf1069910 con 0x55faf1141380 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe664ff9640 1 -- 172.21.15.42:0/1469411958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55faf10ce900 con 0x55faf1141380 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe664ff9640 1 -- 172.21.15.42:0/1469411958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe67803c050 con 0x55faf1141380 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 10 client.17055 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 20 client.17055 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf11b3810 0x55faf11b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 4 client.17055 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.471+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55faf0da8860 con 0x55faf11b3810 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.472+0000 7fe680efe640 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf11b3810 0x55faf11b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.472+0000 7fe664ff9640 10 client.17055 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.472+0000 7fe680efe640 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf11b3810 0x55faf11b5c30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830e137d960d2cc0 server_cookie=f61eaeda585c2941 in_seq=0 out_seq=0 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe664ff9640 1 -- 172.21.15.42:0/1469411958 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55faf0da8860 con 0x55faf11b3810 2024-03-12T04:20:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe664ff9640 10 client.17055 handle_command_reply: tid=0 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 client.17055 shutdown 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141380 msgr2=0x55faf11aaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141380 0x55faf11aaba0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe67802c170 tx=0x7fe67802bc30 comp rx=0 tx=0).stop 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 shutdown_connections 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55faf11b3810 0x55faf11b5c30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faf114a8b0 0x55faf1151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55faf1141d30 0x55faf11ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 --2- 172.21.15.42:0/1469411958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faf1141380 0x55faf11aaba0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe664ff9640 0 client.17055 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 >> 172.21.15.42:0/1469411958 conn(0x55faf113b840 msgr2=0x55faf113a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 shutdown_connections 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 1 -- 172.21.15.42:0/1469411958 wait complete. 2024-03-12T04:20:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.473+0000 7fe68660b740 20 client.17055 trim_cache size 0 max 16384 2024-03-12T04:20:30.482 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:20:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:20:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "98186684-a6b2-4693-85a8-e34840b582a9", 2024-03-12T04:20:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:20:30.482 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6680a1ba0 msgr2=0x7fe6680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6680a1ba0 0x7fe6680a4060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe67c109300 tx=0x7fe66c00aa50 comp rx=0 tx=0).stop 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c1997c0 msgr2=0x7fe67c199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c1997c0 0x7fe67c199c70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe67400eaa0 tx=0x7fe67400ef60 comp rx=0 tx=0).stop 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 shutdown_connections 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6680a1ba0 0x7fe6680a4060 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe67c1997c0 0x7fe67c199c70 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe67c109930 0x7fe67c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 --2- 172.21.15.42:0/1463023709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe67c0ffcb0 0x7fe67c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 >> 172.21.15.42:0/1463023709 conn(0x7fe67c0753c0 msgr2=0x7fe67c1074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 shutdown_connections 2024-03-12T04:20:30.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.474+0000 7fe683963640 1 -- 172.21.15.42:0/1463023709 wait complete. 2024-03-12T04:20:30.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:20:30.488 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:20:30.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.589+0000 7ff10e423640 1 -- 172.21.15.42:0/2247642498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff10806a170 msgr2=0x7ff10810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.589+0000 7ff10e423640 1 --2- 172.21.15.42:0/2247642498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff10806a170 0x7ff10810e000 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff100009cd0 tx=0x7ff10002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 -- 172.21.15.42:0/2247642498 shutdown_connections 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 --2- 172.21.15.42:0/2247642498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff10806a170 0x7ff10810e000 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 --2- 172.21.15.42:0/2247642498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1080697b0 0x7ff108069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 --2- 172.21.15.42:0/2247642498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff108106130 0x7ff108106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 -- 172.21.15.42:0/2247642498 >> 172.21.15.42:0/2247642498 conn(0x7ff108076e20 msgr2=0x7ff108077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 -- 172.21.15.42:0/2247642498 shutdown_connections 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 -- 172.21.15.42:0/2247642498 wait complete. 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 Processor -- start 2024-03-12T04:20:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 -- start start 2024-03-12T04:20:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1080697b0 0x7ff10819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff10806a170 0x7ff10819f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.590+0000 7ff10e423640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108106130 0x7ff1081995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10e423640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108199c30 con 0x7ff108106130 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10e423640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108199da0 con 0x7ff1080697b0 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10e423640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108199f60 con 0x7ff10806a170 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108106130 0x7ff1081995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108106130 0x7ff1081995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51202/0 (socket says 172.21.15.42:51202) 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 -- 172.21.15.42:0/2476142410 learned_addr learned my addr 172.21.15.42:0/2476142410 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff107fff640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1080697b0 0x7ff10819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff10806a170 0x7ff10819f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 -- 172.21.15.42:0/2476142410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff10806a170 msgr2=0x7ff10819f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff10806a170 0x7ff10819f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 -- 172.21.15.42:0/2476142410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1080697b0 msgr2=0x7ff10819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1080697b0 0x7ff10819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 -- 172.21.15.42:0/2476142410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff100009900 con 0x7ff108106130 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff10806a170 0x7ff10819f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff107fff640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1080697b0 0x7ff10819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:20:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108106130 0x7ff1081995e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff100009ca0 tx=0x7ff100035a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff1057fa640 1 -- 172.21.15.42:0/2476142410 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000327d0 con 0x7ff108106130 2024-03-12T04:20:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff1057fa640 1 -- 172.21.15.42:0/2476142410 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff100035e70 con 0x7ff108106130 2024-03-12T04:20:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10819a1e0 con 0x7ff108106130 2024-03-12T04:20:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff1057fa640 1 -- 172.21.15.42:0/2476142410 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100004680 con 0x7ff108106130 2024-03-12T04:20:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.591+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1081a64d0 con 0x7ff108106130 2024-03-12T04:20:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.593+0000 7ff1057fa640 1 -- 172.21.15.42:0/2476142410 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7ff10003b040 con 0x7ff108106130 2024-03-12T04:20:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 Processor -- start 2024-03-12T04:20:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff1057fa640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff0f00a1ba0 0x7ff0f00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 -- start start 2024-03-12T04:20:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff0f00a1ba0 0x7ff0f00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff1057fa640 1 -- 172.21.15.42:0/2476142410 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff1000f0210 con 0x7ff108106130 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6768690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6771270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67717b0 0x562eb6773ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb66d7280 con 0x562eb67717b0 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb66d47b0 con 0x562eb6768bd0 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff110ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb66cff90 con 0x562eb67682b0 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff10c9c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67717b0 0x562eb6773ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff10c9c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67717b0 0x562eb6773ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51208/0 (socket says 172.21.15.42:51208) 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff10c9c0640 1 -- 172.21.15.42:0/3567664748 learned_addr learned my addr 172.21.15.42:0/3567664748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff107fff640 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6768690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff1077fe640 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6771270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff10c9c0640 1 -- 172.21.15.42:0/3567664748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 msgr2=0x562eb6768690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6768690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff10c9c0640 1 -- 172.21.15.42:0/3567664748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 msgr2=0x562eb6771270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.594+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6771270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff10c9c0640 1 -- 172.21.15.42:0/3567664748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eb668e570 con 0x562eb67717b0 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff1077fe640 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6771270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff0f00a1ba0 0x7ff0f00a4060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff10810e3a0 tx=0x7ff0f8009f30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67717b0 0x562eb6773ba0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b9400 tx=0x7ff1000bc9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47a6cfdd12d4855 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff0ee7fc640 1 -- 172.21.15.42:0/3567664748 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000c10f0 con 0x562eb67717b0 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff0ee7fc640 1 -- 172.21.15.42:0/3567664748 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff1000c5740 con 0x562eb67717b0 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff0ee7fc640 1 -- 172.21.15.42:0/3567664748 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b9610 con 0x562eb67717b0 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff110ff9740 1 -- 172.21.15.42:0/3567664748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67717b0 msgr2=0x562eb6773ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.595+0000 7ff110ff9740 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67717b0 0x562eb6773ba0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b9400 tx=0x7ff1000bc9c0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- 172.21.15.42:0/3567664748 shutdown_connections 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67717b0 0x562eb6773ba0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6771270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 --2- 172.21.15.42:0/3567664748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6768690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- 172.21.15.42:0/3567664748 >> 172.21.15.42:0/3567664748 conn(0x562eb6762990 msgr2=0x562eb6762d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- 172.21.15.42:0/3567664748 shutdown_connections 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- 172.21.15.42:0/3567664748 wait complete. 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- start start 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 Processor -- start 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6771120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb676e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67717b0 0x562eb676ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb63a3430 con 0x562eb6768bd0 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb635fe00 con 0x562eb67717b0 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6654fd0 con 0x562eb67682b0 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff110ff9740 10 client.0 _ll_register_callbacks cb 0x562eb66aafd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.596+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb676e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb676e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51220/0 (socket says 172.21.15.42:51220) 2024-03-12T04:20:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 -- 172.21.15.42:0/2329468894 learned_addr learned my addr 172.21.15.42:0/2329468894 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff107fff640 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6771120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67717b0 0x562eb676ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 -- 172.21.15.42:0/2329468894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 msgr2=0x562eb6771120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6771120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 -- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67717b0 msgr2=0x562eb676ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67717b0 0x562eb676ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 -- 172.21.15.42:0/2329468894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1081a64d0 con 0x562eb6768bd0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb676e4b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80136e0 tx=0x7ff0f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0c4b0039efb297 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff0e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2329468894 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004220 con 0x562eb6768bd0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2329468894 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff0f8019040 con 0x562eb6768bd0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eb63cf7e0 con 0x562eb6768bd0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2329468894 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f8004380 con 0x562eb6768bd0 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff110ff9740 10 client.17067 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562eb66ce610 con 0x562eb6768bd0 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.597+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2329468894 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0f8034020 con 0x562eb6768bd0 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2329468894 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff0f8003910 con 0x562eb6768bd0 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff110ff9740 10 client.17067 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff110ff9740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff110ff9740 20 client.17067 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb6781760 0x562eb6783b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff110ff9740 4 client.17067 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eb640c720 con 0x562eb6781760 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff107fff640 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb6781760 0x562eb6783b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff107fff640 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb6781760 0x562eb6783b80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547910eedeaa8f55 server_cookie=e696d339d5b339c8 in_seq=0 out_seq=0 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.598+0000 7ff0e77fe640 10 client.17067 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2329468894 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562eb640c720 con 0x562eb6781760 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff0e77fe640 10 client.17067 handle_command_reply: tid=0 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 client.17067 shutdown 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 msgr2=0x562eb676e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb676e4b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80136e0 tx=0x7ff0f8004730 comp rx=0 tx=0).stop 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 shutdown_connections 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb6781760 0x562eb6783b80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67717b0 0x562eb676ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb676e4b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff0e77fe640 0 client.17067 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2329468894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67682b0 0x562eb6771120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 >> 172.21.15.42:0/2329468894 conn(0x562eb6762990 msgr2=0x562eb6764730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 shutdown_connections 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 1 -- 172.21.15.42:0/2329468894 wait complete. 2024-03-12T04:20:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.599+0000 7ff110ff9740 20 client.17067 trim_cache size 0 max 16384 2024-03-12T04:20:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 Processor -- start 2024-03-12T04:20:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 -- start start 2024-03-12T04:20:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67682b0 0x562eb67618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6770be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb6771120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6767f10 con 0x562eb67682b0 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb677da90 con 0x562eb6768bd0 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff110ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6767c80 con 0x562eb67717b0 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.614+0000 7ff107fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67682b0 0x562eb67618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67682b0 0x562eb67618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51236/0 (socket says 172.21.15.42:51236) 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 -- 172.21.15.42:0/1943344821 learned_addr learned my addr 172.21.15.42:0/1943344821 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff1077fe640 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6770be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb6771120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 -- 172.21.15.42:0/1943344821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 msgr2=0x562eb6771120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb6771120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 -- 172.21.15.42:0/1943344821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 msgr2=0x562eb6770be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6770be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 -- 172.21.15.42:0/1943344821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eb666aac0 con 0x562eb67682b0 2024-03-12T04:20:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff1077fe640 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6770be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff107fff640 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67682b0 0x562eb67618d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x562eb6784460 tx=0x7ff0fc0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a4c6a9914dea44 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff0edffb640 1 -- 172.21.15.42:0/1943344821 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc018040 con 0x562eb67682b0 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff0edffb640 1 -- 172.21.15.42:0/1943344821 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0fc0054d0 con 0x562eb67682b0 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.615+0000 7ff0edffb640 1 -- 172.21.15.42:0/1943344821 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc01a710 con 0x562eb67682b0 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 -- 172.21.15.42:0/1943344821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67682b0 msgr2=0x562eb67618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67682b0 0x562eb67618d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x562eb6784460 tx=0x7ff0fc0094a0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 -- 172.21.15.42:0/1943344821 shutdown_connections 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb6771120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb6768bd0 0x562eb6770be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 --2- 172.21.15.42:0/1943344821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb67682b0 0x562eb67618d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 -- 172.21.15.42:0/1943344821 >> 172.21.15.42:0/1943344821 conn(0x562eb6762990 msgr2=0x562eb67611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 -- 172.21.15.42:0/1943344821 shutdown_connections 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 -- 172.21.15.42:0/1943344821 wait complete. 2024-03-12T04:20:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 -- start start 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 Processor -- start 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67682b0 0x562eb67756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb6775bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.616+0000 7ff110ff9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb67d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff110ff9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb67d21e0 con 0x562eb6768bd0 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff110ff9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb67d2350 con 0x562eb67682b0 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff110ff9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb67d2510 con 0x562eb67717b0 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff110ff9740 10 client.0 _ll_register_callbacks cb 0x7ff1000b8b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb6775bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb6775bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51246/0 (socket says 172.21.15.42:51246) 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 -- 172.21.15.42:0/2306659950 learned_addr learned my addr 172.21.15.42:0/2306659950 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff107fff640 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67682b0 0x562eb67756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff10c9c0640 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb67d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 -- 172.21.15.42:0/2306659950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 msgr2=0x562eb67d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb67d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 -- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67682b0 msgr2=0x562eb67756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67682b0 0x562eb67756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 -- 172.21.15.42:0/2306659950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc0099e0 con 0x562eb6768bd0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff1077fe640 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb6775bf0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80059d0 tx=0x7ff0f8003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb18e32eb1763a3a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff0e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2306659950 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004410 con 0x562eb6768bd0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2306659950 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff0f8019040 con 0x562eb6768bd0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1000c5740 con 0x562eb6768bd0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2306659950 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f8010ae0 con 0x562eb6768bd0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff110ff9740 10 client.17073 fetch_fsmap learned FSMap version 27 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.617+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562eb66906a0 con 0x562eb6768bd0 2024-03-12T04:20:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2306659950 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff1000c5740 con 0x562eb6768bd0 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2306659950 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff0f8004aa0 con 0x562eb6768bd0 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff110ff9740 10 client.17073 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff110ff9740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff110ff9740 20 client.17073 populate_metadata read hostname 'smithi042' 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb67d9de0 0x562eb67dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff110ff9740 4 client.17073 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562eb645b890 con 0x562eb67d9de0 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff107fff640 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb67d9de0 0x562eb67dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff107fff640 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb67d9de0 0x562eb67dc200 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8739ca46a938e221 server_cookie=9c0c65d673b49f1d in_seq=0 out_seq=0 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.618+0000 7ff0e77fe640 10 client.17073 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff0e77fe640 1 -- 172.21.15.42:0/2306659950 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562eb645b890 con 0x562eb67d9de0 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff0e77fe640 10 client.17073 handle_command_reply: tid=0 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 client.17073 shutdown 2024-03-12T04:20:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 msgr2=0x562eb6775bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb6775bf0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80059d0 tx=0x7ff0f8003b30 comp rx=0 tx=0).stop 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 shutdown_connections 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562eb67d9de0 0x562eb67dc200 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562eb67717b0 0x562eb67d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562eb6768bd0 0x562eb6775bf0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 --2- 172.21.15.42:0/2306659950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eb67682b0 0x562eb67756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 >> 172.21.15.42:0/2306659950 conn(0x562eb6762990 msgr2=0x562eb6761a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff0e77fe640 0 client.17073 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 shutdown_connections 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 1 -- 172.21.15.42:0/2306659950 wait complete. 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff110ff9740 20 client.17073 trim_cache size 0 max 16384 2024-03-12T04:20:30.629 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "98186684-a6b2-4693-85a8-e34840b582a9": { 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "98186684-a6b2-4693-85a8-e34840b582a9", 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:20:30.630 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff0f00a1ba0 msgr2=0x7ff0f00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff10e423640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff0f00a1ba0 0x7ff0f00a4060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff10810e3a0 tx=0x7ff0f8009f30 comp rx=0 tx=0).stop 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108106130 msgr2=0x7ff1081995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.619+0000 7ff10e423640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108106130 0x7ff1081995e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff100009ca0 tx=0x7ff100035a70 comp rx=0 tx=0).stop 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 shutdown_connections 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff0f00a1ba0 0x7ff0f00a4060 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108106130 0x7ff1081995e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff10806a170 0x7ff10819f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 --2- 172.21.15.42:0/2476142410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1080697b0 0x7ff10819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:20:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 >> 172.21.15.42:0/2476142410 conn(0x7ff108076e20 msgr2=0x7ff10810c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:20:30.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 shutdown_connections 2024-03-12T04:20:30.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:20:30.620+0000 7ff10e423640 1 -- 172.21.15.42:0/2476142410 wait complete. 2024-03-12T04:20:30.632 INFO:tasks.cephfs.filesystem:scrub status for tag:98186684-a6b2-4693-85a8-e34840b582a9 - {'path': '/', 'tag': '98186684-a6b2-4693-85a8-e34840b582a9', 'options': 'recursive,force'} 2024-03-12T04:20:30.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:30 smithi136 ceph-mon[35571]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:31.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:30 smithi042 ceph-mon[32260]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:31.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:30 smithi119 ceph-mon[36119]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:31 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:20:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:31 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:20:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:31 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:20:32.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:31 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:20:32.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:31 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:20:32.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:31 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:20:32.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:31 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:20:32.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:31 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:20:32.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:31 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:20:32.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:31 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:20:32.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:31 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:20:32.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:31 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:20:32.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:32 smithi136 ceph-mon[35571]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:33.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:32 smithi042 ceph-mon[32260]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:33.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:32 smithi119 ceph-mon[36119]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:34 smithi042 ceph-mon[32260]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:35.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:34 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:20:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:34 smithi119 ceph-mon[36119]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:35.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:34 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:20:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:34 smithi136 ceph-mon[35571]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:35.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:34 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:20:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:36 smithi042 ceph-mon[32260]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:37.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:36 smithi119 ceph-mon[36119]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:37.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:36 smithi136 ceph-mon[35571]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:39.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:38 smithi042 ceph-mon[32260]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:39.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:38 smithi119 ceph-mon[36119]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:39.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:38 smithi136 ceph-mon[35571]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:41.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:40 smithi042 ceph-mon[32260]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:41.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:40 smithi119 ceph-mon[36119]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:41.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:40 smithi136 ceph-mon[35571]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:43.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:42 smithi042 ceph-mon[32260]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:43.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:42 smithi119 ceph-mon[36119]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:43.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:42 smithi136 ceph-mon[35571]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:20:45.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:44 smithi042 ceph-mon[32260]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:45.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:44 smithi119 ceph-mon[36119]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:45.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:44 smithi136 ceph-mon[35571]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:20:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:47 smithi119 ceph-mon[36119]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:47 smithi136 ceph-mon[35571]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:47.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:47 smithi042 ceph-mon[32260]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:49.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:49 smithi119 ceph-mon[36119]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:49 smithi136 ceph-mon[35571]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:49 smithi042 ceph-mon[32260]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:51 smithi119 ceph-mon[36119]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:51 smithi136 ceph-mon[35571]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:51.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:51 smithi042 ceph-mon[32260]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:53.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:53 smithi119 ceph-mon[36119]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:53 smithi136 ceph-mon[35571]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:53 smithi042 ceph-mon[32260]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:55.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:55 smithi119 ceph-mon[36119]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:55.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:55 smithi136 ceph-mon[35571]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:55.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:55 smithi042 ceph-mon[32260]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:57.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:57 smithi119 ceph-mon[36119]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:57.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:57 smithi136 ceph-mon[35571]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:57.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:57 smithi042 ceph-mon[32260]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:20:59.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:20:59 smithi119 ceph-mon[36119]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:59.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:20:59 smithi136 ceph-mon[35571]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:20:59.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:20:59 smithi042 ceph-mon[32260]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:00.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:21:00.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:21:00.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.748+0000 7f7f2b19f640 1 -- 172.21.15.42:0/2672495071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 msgr2=0x7f7f24100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.748+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/2672495071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f24100440 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009930 tx=0x7f7f2002bc30 comp rx=0 tx=0).stop 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.748+0000 7f7f2b19f640 1 -- 172.21.15.42:0/2672495071 shutdown_connections 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.748+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/2672495071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f24105130 0x7f7f2410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.748+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/2672495071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f24100a10 0x7f7f24104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.748+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/2672495071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f24100440 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.748+0000 7f7f2b19f640 1 -- 172.21.15.42:0/2672495071 >> 172.21.15.42:0/2672495071 conn(0x7f7f240fb8e0 msgr2=0x7f7f240fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 -- 172.21.15.42:0/2672495071 shutdown_connections 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 -- 172.21.15.42:0/2672495071 wait complete. 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 Processor -- start 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 -- start start 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f241974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f24105130 0x7f7f24197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2419bd80 0x7f7f2419c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2419c770 con 0x7f7f24100060 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2419c8e0 con 0x7f7f2419bd80 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.749+0000 7f7f2b19f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2419ca50 con 0x7f7f24105130 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f241974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f241974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46246/0 (socket says 172.21.15.42:46246) 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 -- 172.21.15.42:0/3503705431 learned_addr learned my addr 172.21.15.42:0/3503705431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2419bd80 0x7f7f2419c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2999c640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f24105130 0x7f7f24197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 -- 172.21.15.42:0/3503705431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f24105130 msgr2=0x7f7f24197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f24105130 0x7f7f24197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 -- 172.21.15.42:0/3503705431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2419bd80 msgr2=0x7f7f2419c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2419bd80 0x7f7f2419c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 -- 172.21.15.42:0/3503705431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f20009560 con 0x7f7f24100060 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2999c640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f24105130 0x7f7f24197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f241974c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009230 tx=0x7f7f20005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f1b7fe640 1 -- 172.21.15.42:0/3503705431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004350 con 0x7f7f24100060 2024-03-12T04:21:00.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f241a4030 con 0x7f7f24100060 2024-03-12T04:21:00.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f1b7fe640 1 -- 172.21.15.42:0/3503705431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f2002de90 con 0x7f7f24100060 2024-03-12T04:21:00.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f1b7fe640 1 -- 172.21.15.42:0/3503705431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20005220 con 0x7f7f24100060 2024-03-12T04:21:00.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.750+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f241a44b0 con 0x7f7f24100060 2024-03-12T04:21:00.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.753+0000 7f7f1b7fe640 1 -- 172.21.15.42:0/3503705431 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7f20003c30 con 0x7f7f24100060 2024-03-12T04:21:00.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.753+0000 7f7f2efcc740 1 Processor -- start 2024-03-12T04:21:00.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2efcc740 1 -- start start 2024-03-12T04:21:00.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f1b7fe640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7f100a1ba0 0x7f7f100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2200 0x5622772e25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2b20 0x5622772ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277251280 con 0x5622772e2200 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227724e7b0 con 0x5622772e2b20 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7f100a1ba0 0x7f7f100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277249f90 con 0x5622772efde0 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f1b7fe640 1 -- 172.21.15.42:0/3503705431 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7f200ef6c0 con 0x7f7f24100060 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2200 0x5622772e25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2200 0x5622772e25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46256/0 (socket says 172.21.15.42:46256) 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 -- 172.21.15.42:0/1637042904 learned_addr learned my addr 172.21.15.42:0/1637042904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2999c640 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2b20 0x5622772ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 -- 172.21.15.42:0/1637042904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 msgr2=0x5622772f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 -- 172.21.15.42:0/1637042904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2b20 msgr2=0x5622772ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2b20 0x5622772ef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a19d640 1 -- 172.21.15.42:0/1637042904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562277208570 con 0x5622772e2200 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.754+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.755+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7f100a1ba0 0x7f7f100a4060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c005fd0 tx=0x7f7f1c009a90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.755+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2200 0x5622772e25e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7f200bc3e0 tx=0x7f7f20009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc01534005fe9e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.755+0000 7f7f18bfe640 1 -- 172.21.15.42:0/1637042904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f200dc0b0 con 0x5622772e2200 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.755+0000 7f7f18bfe640 1 -- 172.21.15.42:0/1637042904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f200bc450 con 0x5622772e2200 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.755+0000 7f7f18bfe640 1 -- 172.21.15.42:0/1637042904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f200a5bd0 con 0x5622772e2200 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.755+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1637042904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2200 msgr2=0x5622772e25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.755+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2200 0x5622772e25e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7f200bc3e0 tx=0x7f7f20009260 comp rx=0 tx=0).stop 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1637042904 shutdown_connections 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2b20 0x5622772ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1637042904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2200 0x5622772e25e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1637042904 >> 172.21.15.42:0/1637042904 conn(0x5622772dcc00 msgr2=0x5622772dcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1637042904 shutdown_connections 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1637042904 wait complete. 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- start start 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 Processor -- start 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772e2b20 0x5622772ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772efde0 0x5622772ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562276f1d430 con 0x5622772efde0 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562276ed9e00 con 0x5622772e2200 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622771cefd0 con 0x5622772e2b20 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2efcc740 10 client.0 _ll_register_callbacks cb 0x562277224fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:00.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2a99e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772efde0 0x5622772ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2a19d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.756+0000 7f7f2a99e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772efde0 0x5622772ed1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46258/0 (socket says 172.21.15.42:46258) 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a99e640 1 -- 172.21.15.42:0/1363170217 learned_addr learned my addr 172.21.15.42:0/1363170217 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2999c640 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772e2b20 0x5622772ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a99e640 1 -- 172.21.15.42:0/1363170217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772e2b20 msgr2=0x5622772ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772e2b20 0x5622772ecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a99e640 1 -- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 msgr2=0x5622772ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a99e640 1 -- 172.21.15.42:0/1363170217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f241a44b0 con 0x5622772efde0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772efde0 0x5622772ed1a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c014fd0 tx=0x7f7f1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f994d37f509612 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f0dffb640 1 -- 172.21.15.42:0/1363170217 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c004b20 con 0x5622772efde0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562276f497e0 con 0x5622772efde0 2024-03-12T04:21:00.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f0dffb640 1 -- 172.21.15.42:0/1363170217 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7f1c004c80 con 0x5622772efde0 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f0dffb640 1 -- 172.21.15.42:0/1363170217 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f1c004e40 con 0x5622772efde0 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2efcc740 10 client.17085 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562277248610 con 0x5622772efde0 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.757+0000 7f7f0dffb640 1 -- 172.21.15.42:0/1363170217 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x562276f497e0 con 0x5622772efde0 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f0dffb640 1 -- 172.21.15.42:0/1363170217 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7f1c005670 con 0x5622772efde0 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2efcc740 10 client.17085 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2efcc740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2efcc740 20 client.17085 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622772ffd20 0x562277302140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2efcc740 4 client.17085 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562276f86720 con 0x5622772ffd20 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622772ffd20 0x562277302140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622772ffd20 0x562277302140 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b34e7184fb7b8b server_cookie=5fb0069e3c466e11 in_seq=0 out_seq=0 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.758+0000 7f7f0dffb640 10 client.17085 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f0dffb640 1 -- 172.21.15.42:0/1363170217 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562276f86720 con 0x5622772ffd20 2024-03-12T04:21:00.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f0dffb640 10 client.17085 handle_command_reply: tid=0 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 client.17085 shutdown 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772efde0 msgr2=0x5622772ed1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772efde0 0x5622772ed1a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c014fd0 tx=0x7f7f1c004ab0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 shutdown_connections 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622772ffd20 0x562277302140 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772efde0 0x5622772ed1a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772e2b20 0x5622772ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f0dffb640 0 client.17085 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1363170217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 >> 172.21.15.42:0/1363170217 conn(0x5622772dcc00 msgr2=0x5622772de950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 shutdown_connections 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1363170217 wait complete. 2024-03-12T04:21:00.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.759+0000 7f7f2efcc740 20 client.17085 trim_cache size 0 max 16384 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 Processor -- start 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 -- start start 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622772d82d0 con 0x5622772e2b20 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622772fc850 con 0x5622772e2200 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277307350 con 0x5622772efde0 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46266/0 (socket says 172.21.15.42:46266) 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 -- 172.21.15.42:0/1527504730 learned_addr learned my addr 172.21.15.42:0/1527504730 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 -- 172.21.15.42:0/1527504730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 msgr2=0x5622772f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 -- 172.21.15.42:0/1527504730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 msgr2=0x5622772ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 -- 172.21.15.42:0/1527504730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622771e4ac0 con 0x5622772e2b20 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f2999c640 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772ef800 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7f24069220 tx=0x7f7f140123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794014b3d71fdca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.775+0000 7f7f0ffff640 1 -- 172.21.15.42:0/1527504730 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14017070 con 0x5622772e2b20 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f0ffff640 1 -- 172.21.15.42:0/1527504730 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f14004050 con 0x5622772e2b20 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f0ffff640 1 -- 172.21.15.42:0/1527504730 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14003d40 con 0x5622772e2b20 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1527504730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 msgr2=0x5622772ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772ef800 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7f24069220 tx=0x7f7f140123f0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1527504730 shutdown_connections 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x5622772f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772ef800 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/1527504730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1527504730 >> 172.21.15.42:0/1527504730 conn(0x5622772dcc00 msgr2=0x5622772e7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1527504730 shutdown_connections 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 -- 172.21.15.42:0/1527504730 wait complete. 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 -- start start 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.776+0000 7f7f2efcc740 1 Processor -- start 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x562277356220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277356760 con 0x5622772e2b20 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622773568a0 con 0x5622772e2200 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277356a10 con 0x5622772efde0 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 10 client.0 _ll_register_callbacks cb 0x7f7f20005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46270/0 (socket says 172.21.15.42:46270) 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 -- 172.21.15.42:0/2017599593 learned_addr learned my addr 172.21.15.42:0/2017599593 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x562277356220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 -- 172.21.15.42:0/2017599593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 msgr2=0x562277356220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x562277356220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 -- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 msgr2=0x5622772f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 -- 172.21.15.42:0/2017599593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14009850 con 0x5622772e2b20 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2a99e640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x562277356220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2999c640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772f4140 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7f1401c000 tx=0x7f7f14004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58871a58b98a8a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f0dffb640 1 -- 172.21.15.42:0/2017599593 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14017040 con 0x5622772e2b20 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f0dffb640 1 -- 172.21.15.42:0/2017599593 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7f14003d10 con 0x5622772e2b20 2024-03-12T04:21:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.777+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f200bc450 con 0x5622772e2b20 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f0dffb640 1 -- 172.21.15.42:0/2017599593 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f140035f0 con 0x5622772e2b20 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 10 client.17091 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56227720a6a0 con 0x5622772e2b20 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f0dffb640 1 -- 172.21.15.42:0/2017599593 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f200bc450 con 0x5622772e2b20 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f0dffb640 1 -- 172.21.15.42:0/2017599593 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7f14004bd0 con 0x5622772e2b20 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 10 client.17091 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 20 client.17091 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622773581e0 0x56227735a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 4 client.17091 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562276fd5890 con 0x5622773581e0 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.778+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622773581e0 0x56227735a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f0dffb640 10 client.17091 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2a19d640 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622773581e0 0x56227735a600 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad76c8ab6d088332 server_cookie=55b247e300534f1e in_seq=0 out_seq=0 2024-03-12T04:21:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f0dffb640 1 -- 172.21.15.42:0/2017599593 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562276fd5890 con 0x5622773581e0 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f0dffb640 10 client.17091 handle_command_reply: tid=0 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 client.17091 shutdown 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 msgr2=0x5622772f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772f4140 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7f1401c000 tx=0x7f7f14004450 comp rx=0 tx=0).stop 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 shutdown_connections 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5622773581e0 0x56227735a600 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622772efde0 0x562277356220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f0dffb640 0 client.17091 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622772e2b20 0x5622772f4140 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 --2- 172.21.15.42:0/2017599593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622772e2200 0x5622772f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 >> 172.21.15.42:0/2017599593 conn(0x5622772dcc00 msgr2=0x5622772db4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 shutdown_connections 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 1 -- 172.21.15.42:0/2017599593 wait complete. 2024-03-12T04:21:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.779+0000 7f7f2efcc740 20 client.17091 trim_cache size 0 max 16384 2024-03-12T04:21:00.789 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:21:00.789 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:21:00.789 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:21:00.789 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7f100a1ba0 msgr2=0x7f7f100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7f100a1ba0 0x7f7f100a4060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c005fd0 tx=0x7f7f1c009a90 comp rx=0 tx=0).stop 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 msgr2=0x7f7f241974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f241974c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009230 tx=0x7f7f20005dd0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 shutdown_connections 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7f100a1ba0 0x7f7f100a4060 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2419bd80 0x7f7f2419c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f24105130 0x7f7f24197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 --2- 172.21.15.42:0/3503705431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24100060 0x7f7f241974c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 >> 172.21.15.42:0/3503705431 conn(0x7f7f240fb8e0 msgr2=0x7f7f24069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 shutdown_connections 2024-03-12T04:21:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.780+0000 7f7f2b19f640 1 -- 172.21.15.42:0/3503705431 wait complete. 2024-03-12T04:21:00.793 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:21:00.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:21:00.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 -- 172.21.15.42:0/774314960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1002f0 msgr2=0x7f2e8c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 --2- 172.21.15.42:0/774314960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1002f0 0x7f2e8c1006d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009cd0 tx=0x7f2e8402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 -- 172.21.15.42:0/774314960 shutdown_connections 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 --2- 172.21.15.42:0/774314960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e8c1053c0 0x7f2e8c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 --2- 172.21.15.42:0/774314960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e8c100ca0 0x7f2e8c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 --2- 172.21.15.42:0/774314960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1002f0 0x7f2e8c1006d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 -- 172.21.15.42:0/774314960 >> 172.21.15.42:0/774314960 conn(0x7f2e8c077180 msgr2=0x7f2e8c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 -- 172.21.15.42:0/774314960 shutdown_connections 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.893+0000 7f2e92de2640 1 -- 172.21.15.42:0/774314960 wait complete. 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 Processor -- start 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 -- start start 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e8c1002f0 0x7f2e8c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1053c0 0x7f2e8c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e8c1975e0 0x7f2e8c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c197fd0 con 0x7f2e8c1053c0 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c198140 con 0x7f2e8c1975e0 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e92de2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c198300 con 0x7f2e8c1002f0 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1053c0 0x7f2e8c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1053c0 0x7f2e8c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46284/0 (socket says 172.21.15.42:46284) 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e8bfff640 1 -- 172.21.15.42:0/2738430450 learned_addr learned my addr 172.21.15.42:0/2738430450 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e90b7e640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e8c1002f0 0x7f2e8c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.894+0000 7f2e9137f640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e8c1975e0 0x7f2e8c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e8bfff640 1 -- 172.21.15.42:0/2738430450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e8c1002f0 msgr2=0x7f2e8c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e8bfff640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e8c1002f0 0x7f2e8c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e8bfff640 1 -- 172.21.15.42:0/2738430450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e8c1975e0 msgr2=0x7f2e8c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e8bfff640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e8c1975e0 0x7f2e8c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e8bfff640 1 -- 172.21.15.42:0/2738430450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e84009900 con 0x7f2e8c1053c0 2024-03-12T04:21:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e9137f640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e8c1975e0 0x7f2e8c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e8bfff640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1053c0 0x7f2e8c19d970 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c970 tx=0x7f2e7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e89ffb640 1 -- 172.21.15.42:0/2738430450 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c007bb0 con 0x7f2e8c1053c0 2024-03-12T04:21:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e89ffb640 1 -- 172.21.15.42:0/2738430450 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c007d10 con 0x7f2e8c1053c0 2024-03-12T04:21:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c1a42f0 con 0x7f2e8c1053c0 2024-03-12T04:21:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e89ffb640 1 -- 172.21.15.42:0/2738430450 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c018660 con 0x7f2e8c1053c0 2024-03-12T04:21:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.895+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c1a4800 con 0x7f2e8c1053c0 2024-03-12T04:21:00.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.896+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c067f70 con 0x7f2e8c1053c0 2024-03-12T04:21:00.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.897+0000 7f2e89ffb640 1 -- 172.21.15.42:0/2738430450 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f2e7c00f450 con 0x7f2e8c1053c0 2024-03-12T04:21:00.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.898+0000 7f2e89ffb640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2e780a1c20 0x7f2e780a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:00.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.898+0000 7f2e90b7e640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2e780a1c20 0x7f2e780a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:00.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.898+0000 7f2e89ffb640 1 -- 172.21.15.42:0/2738430450 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2e7c0cdc30 con 0x7f2e8c1053c0 2024-03-12T04:21:00.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.899+0000 7f2e90b7e640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2e780a1c20 0x7f2e780a40e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009ca0 tx=0x7f2e84005c50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:00.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:00.902+0000 7f2e89ffb640 1 -- 172.21.15.42:0/2738430450 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e7c096b70 con 0x7f2e8c1053c0 2024-03-12T04:21:01.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.124+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e8c198b10 con 0x7f2e8c1053c0 2024-03-12T04:21:01.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.125+0000 7f2e89ffb640 1 -- 172.21.15.42:0/2738430450 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f2e7c01d020 con 0x7f2e8c1053c0 2024-03-12T04:21:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:21:01.128 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:21:01.128 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2e780a1c20 msgr2=0x7f2e780a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2e780a1c20 0x7f2e780a40e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009ca0 tx=0x7f2e84005c50 comp rx=0 tx=0).stop 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1053c0 msgr2=0x7f2e8c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1053c0 0x7f2e8c19d970 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c970 tx=0x7f2e7c00ce30 comp rx=0 tx=0).stop 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 shutdown_connections 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2e780a1c20 0x7f2e780a40e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e8c1975e0 0x7f2e8c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8c1053c0 0x7f2e8c19d970 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 --2- 172.21.15.42:0/2738430450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e8c1002f0 0x7f2e8c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 >> 172.21.15.42:0/2738430450 conn(0x7f2e8c077180 msgr2=0x7f2e8c0687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 shutdown_connections 2024-03-12T04:21:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.127+0000 7f2e92de2640 1 -- 172.21.15.42:0/2738430450 wait complete. 2024-03-12T04:21:01.141 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:01.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:21:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 -- 172.21.15.42:0/801265791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c105360 msgr2=0x7f520c10cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 --2- 172.21.15.42:0/801265791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c105360 0x7f520c10cf30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009cd0 tx=0x7f51fc02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 -- 172.21.15.42:0/801265791 shutdown_connections 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 --2- 172.21.15.42:0/801265791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c105360 0x7f520c10cf30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 --2- 172.21.15.42:0/801265791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f520c100c90 0x7f520c104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 --2- 172.21.15.42:0/801265791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520c1002e0 0x7f520c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 -- 172.21.15.42:0/801265791 >> 172.21.15.42:0/801265791 conn(0x7f520c0771e0 msgr2=0x7f520c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.243+0000 7f52132eb640 1 -- 172.21.15.42:0/801265791 shutdown_connections 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 -- 172.21.15.42:0/801265791 wait complete. 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 Processor -- start 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 -- start start 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f520c1002e0 0x7f520c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520c105360 0x7f520c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c19bf70 0x7f520c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c19c960 con 0x7f520c19bf70 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c19cad0 con 0x7f520c105360 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f52132eb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c19cc40 con 0x7f520c1002e0 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f5211888640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c19bf70 0x7f520c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f5211888640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c19bf70 0x7f520c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46292/0 (socket says 172.21.15.42:46292) 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f5211888640 1 -- 172.21.15.42:0/3422957180 learned_addr learned my addr 172.21.15.42:0/3422957180 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f5210886640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520c105360 0x7f520c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.244+0000 7f5211087640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f520c1002e0 0x7f520c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f5211888640 1 -- 172.21.15.42:0/3422957180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f520c1002e0 msgr2=0x7f520c1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f5211888640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f520c1002e0 0x7f520c1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f5211888640 1 -- 172.21.15.42:0/3422957180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520c105360 msgr2=0x7f520c197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f5211888640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520c105360 0x7f520c197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f5211888640 1 -- 172.21.15.42:0/3422957180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009900 con 0x7f520c19bf70 2024-03-12T04:21:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f5211087640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f520c1002e0 0x7f520c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f5211888640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c19bf70 0x7f520c19c420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f51fc037040 tx=0x7f51fc035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f520a7fc640 1 -- 172.21.15.42:0/3422957180 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0329e0 con 0x7f520c19bf70 2024-03-12T04:21:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f520a7fc640 1 -- 172.21.15.42:0/3422957180 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51fc003e20 con 0x7f520c19bf70 2024-03-12T04:21:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520c1a4220 con 0x7f520c19bf70 2024-03-12T04:21:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f520a7fc640 1 -- 172.21.15.42:0/3422957180 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc03f440 con 0x7f520c19bf70 2024-03-12T04:21:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.245+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c1a4640 con 0x7f520c19bf70 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.247+0000 7f520a7fc640 1 -- 172.21.15.42:0/3422957180 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f51fc03b040 con 0x7f520c19bf70 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 Processor -- start 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 -- start start 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f520a7fc640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51f40a1ba0 0x7f51f40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab48630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48b70 0x559f9ab51210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab51750 0x559f9ab53b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9ab03960 con 0x559f9ab48250 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9aab7310 con 0x559f9ab48b70 2024-03-12T04:21:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5215f98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9aab4840 con 0x559f9ab51750 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f520a7fc640 1 -- 172.21.15.42:0/3422957180 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f51fc0f0410 con 0x7f520c19bf70 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5211087640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab48630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5210886640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51f40a1ba0 0x7f51f40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5211087640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab48630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46294/0 (socket says 172.21.15.42:46294) 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.248+0000 7f5211087640 1 -- 172.21.15.42:0/4057909396 learned_addr learned my addr 172.21.15.42:0/4057909396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5211888640 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab51750 0x559f9ab53b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5210886640 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48b70 0x559f9ab51210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5211087640 1 -- 172.21.15.42:0/4057909396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab51750 msgr2=0x559f9ab53b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5211087640 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab51750 0x559f9ab53b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5211087640 1 -- 172.21.15.42:0/4057909396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48b70 msgr2=0x559f9ab51210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5211087640 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48b70 0x559f9ab51210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5211087640 1 -- 172.21.15.42:0/4057909396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f9aa72060 con 0x559f9ab48250 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5210886640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51f40a1ba0 0x7f51f40a4060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5200006e10 tx=0x7f52000156d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f5211087640 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab48630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5204008c60 tx=0x7f520400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf8f44ab0fc6fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f51f37fe640 1 -- 172.21.15.42:0/4057909396 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204005220 con 0x559f9ab48250 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f51f37fe640 1 -- 172.21.15.42:0/4057909396 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5204004c50 con 0x559f9ab48250 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.249+0000 7f51f37fe640 1 -- 172.21.15.42:0/4057909396 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040107b0 con 0x559f9ab48250 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 -- 172.21.15.42:0/4057909396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 msgr2=0x559f9ab48630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab48630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5204008c60 tx=0x7f520400e720 comp rx=0 tx=0).stop 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 -- 172.21.15.42:0/4057909396 shutdown_connections 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab51750 0x559f9ab53b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48b70 0x559f9ab51210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 --2- 172.21.15.42:0/4057909396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab48630 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 -- 172.21.15.42:0/4057909396 >> 172.21.15.42:0/4057909396 conn(0x559f9ab42980 msgr2=0x559f9ab42d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 -- 172.21.15.42:0/4057909396 shutdown_connections 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 -- 172.21.15.42:0/4057909396 wait complete. 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 -- start start 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.250+0000 7f5215f98740 1 Processor -- start 2024-03-12T04:21:01.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab4e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9aab0020 con 0x559f9ab48250 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9a783430 con 0x559f9ab51750 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9a73fe00 con 0x559f9ab48b70 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab4e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 10 client.0 _ll_register_callbacks cb 0x559f9aa57cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab4e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46306/0 (socket says 172.21.15.42:46306) 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 -- 172.21.15.42:0/2772775338 learned_addr learned my addr 172.21.15.42:0/2772775338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5210886640 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 -- 172.21.15.42:0/2772775338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 msgr2=0x559f9ab4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 -- 172.21.15.42:0/2772775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 msgr2=0x559f9ab4f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211888640 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 -- 172.21.15.42:0/2772775338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5204008890 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5211087640 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab4e790 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5204010490 tx=0x7f5204004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5d2ddda16aaa94 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f51f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f51f0ff9640 1 -- 172.21.15.42:0/2772775338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204010730 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f51f0ff9640 1 -- 172.21.15.42:0/2772775338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5204004dc0 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.251+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f9a7af7e0 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f51f0ff9640 1 -- 172.21.15.42:0/2772775338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52040055a0 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 10 client.17109 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f9aaae6a0 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f51f0ff9640 1 -- 172.21.15.42:0/2772775338 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5204031020 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f51f0ff9640 1 -- 172.21.15.42:0/2772775338 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5204023a50 con 0x559f9ab48250 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 10 client.17109 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 10 client.17109 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 20 client.17109 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9ab61b60 0x559f9ab63f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 4 client.17109 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f9a7ec720 con 0x559f9ab61b60 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f5210886640 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9ab61b60 0x559f9ab63f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.252+0000 7f51f0ff9640 10 client.17109 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5210886640 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9ab61b60 0x559f9ab63f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86946011540ecd80 server_cookie=6aea37b8c2393abe in_seq=0 out_seq=0 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f51f0ff9640 1 -- 172.21.15.42:0/2772775338 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f9a7ec720 con 0x559f9ab61b60 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f51f0ff9640 10 client.17109 handle_command_reply: tid=0 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 client.17109 shutdown 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 msgr2=0x559f9ab4e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab4e790 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5204010490 tx=0x7f5204004830 comp rx=0 tx=0).stop 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 shutdown_connections 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9ab61b60 0x559f9ab63f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 --2- 172.21.15.42:0/2772775338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48250 0x559f9ab4e790 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f51f0ff9640 0 client.17109 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:01.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 >> 172.21.15.42:0/2772775338 conn(0x559f9ab42980 msgr2=0x559f9ab446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 shutdown_connections 2024-03-12T04:21:01.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 1 -- 172.21.15.42:0/2772775338 wait complete. 2024-03-12T04:21:01.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.253+0000 7f5215f98740 20 client.17109 trim_cache size 0 max 16384 2024-03-12T04:21:01.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 Processor -- start 2024-03-12T04:21:01.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 -- start start 2024-03-12T04:21:01.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48250 0x559f9ab50ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48b70 0x559f9ab4f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9aa35130 con 0x559f9ab48b70 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9ab5d610 con 0x559f9ab51750 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5215f98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9ab5e680 con 0x559f9ab48250 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5210886640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48b70 0x559f9ab4f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5210886640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48b70 0x559f9ab4f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46308/0 (socket says 172.21.15.42:46308) 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.265+0000 7f5210886640 1 -- 172.21.15.42:0/2432500939 learned_addr learned my addr 172.21.15.42:0/2432500939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5211087640 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48250 0x559f9ab50ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5211888640 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5210886640 1 -- 172.21.15.42:0/2432500939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48250 msgr2=0x559f9ab50ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5210886640 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48250 0x559f9ab50ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5210886640 1 -- 172.21.15.42:0/2432500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 msgr2=0x559f9ab4b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5210886640 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5210886640 1 -- 172.21.15.42:0/2432500939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f9aa4ac20 con 0x559f9ab48b70 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5211888640 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f5210886640 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48b70 0x559f9ab4f030 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5200015f20 tx=0x7f520001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8d9c184bd31d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f51f2ffd640 1 -- 172.21.15.42:0/2432500939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000292d0 con 0x559f9ab48b70 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f51f2ffd640 1 -- 172.21.15.42:0/2432500939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f52000298e0 con 0x559f9ab48b70 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.266+0000 7f51f2ffd640 1 -- 172.21.15.42:0/2432500939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200025410 con 0x559f9ab48b70 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- 172.21.15.42:0/2432500939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48b70 msgr2=0x559f9ab4f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48b70 0x559f9ab4f030 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5200015f20 tx=0x7f520001b000 comp rx=0 tx=0).stop 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- 172.21.15.42:0/2432500939 shutdown_connections 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab51750 0x559f9ab4b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab48b70 0x559f9ab4f030 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 --2- 172.21.15.42:0/2432500939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48250 0x559f9ab50ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- 172.21.15.42:0/2432500939 >> 172.21.15.42:0/2432500939 conn(0x559f9ab42980 msgr2=0x559f9ab49ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- 172.21.15.42:0/2432500939 shutdown_connections 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- 172.21.15.42:0/2432500939 wait complete. 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- start start 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 Processor -- start 2024-03-12T04:21:01.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48250 0x559f9ab576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab57bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab51750 0x559f9abb8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9ab582f0 con 0x559f9ab51750 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9abb85b0 con 0x559f9ab48250 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9abb8720 con 0x559f9ab48b70 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.267+0000 7f5215f98740 10 client.0 _ll_register_callbacks cb 0x7f52040083f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab51750 0x559f9abb8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab51750 0x559f9abb8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46324/0 (socket says 172.21.15.42:46324) 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 -- 172.21.15.42:0/3094861798 learned_addr learned my addr 172.21.15.42:0/3094861798 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211087640 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48250 0x559f9ab576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5210886640 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab57bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 -- 172.21.15.42:0/3094861798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 msgr2=0x559f9ab57bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab57bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 -- 172.21.15.42:0/3094861798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48250 msgr2=0x559f9ab576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48250 0x559f9ab576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 -- 172.21.15.42:0/3094861798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520001f040 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5211888640 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab51750 0x559f9abb8070 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0b9b10 tx=0x7f51fc0bcd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dfeaa8f1d01208 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f51f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f51f0ff9640 1 -- 172.21.15.42:0/3094861798 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0bd140 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f51f0ff9640 1 -- 172.21.15.42:0/3094861798 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f51fc0c1290 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5204004c50 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f51f0ff9640 1 -- 172.21.15.42:0/3094861798 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc09f2f0 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5215f98740 10 client.17115 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f9aa708c0 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.268+0000 7f51f0ff9640 1 -- 172.21.15.42:0/3094861798 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5204004c50 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f51f0ff9640 1 -- 172.21.15.42:0/3094861798 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f51fc0eddd0 con 0x559f9ab51750 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 10 client.17115 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 10 client.17115 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 20 client.17115 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9abb9cd0 0x559f9abbc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 4 client.17115 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f9a83b890 con 0x559f9abb9cd0 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5211087640 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9abb9cd0 0x559f9abbc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5211087640 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9abb9cd0 0x559f9abbc0f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=621a20763a2b70fb server_cookie=570f68b60d0ac69 in_seq=0 out_seq=0 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f51f0ff9640 10 client.17115 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f51f0ff9640 1 -- 172.21.15.42:0/3094861798 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559f9a83b890 con 0x559f9abb9cd0 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f51f0ff9640 10 client.17115 handle_command_reply: tid=0 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 client.17115 shutdown 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab51750 msgr2=0x559f9abb8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab51750 0x559f9abb8070 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0b9b10 tx=0x7f51fc0bcd40 comp rx=0 tx=0).stop 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 shutdown_connections 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x559f9abb9cd0 0x559f9abbc0f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f9ab51750 0x559f9abb8070 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f51f0ff9640 0 client.17115 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:01.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.269+0000 7f5215f98740 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f9ab48b70 0x559f9ab57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f5215f98740 1 --2- 172.21.15.42:0/3094861798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9ab48250 0x559f9ab576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 >> 172.21.15.42:0/3094861798 conn(0x559f9ab42980 msgr2=0x559f9ab412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 shutdown_connections 2024-03-12T04:21:01.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f5215f98740 1 -- 172.21.15.42:0/3094861798 wait complete. 2024-03-12T04:21:01.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f5215f98740 20 client.17115 trim_cache size 0 max 16384 2024-03-12T04:21:01.279 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51f40a1ba0 msgr2=0x7f51f40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51f40a1ba0 0x7f51f40a4060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5200006e10 tx=0x7f52000156d0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c19bf70 msgr2=0x7f520c19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c19bf70 0x7f520c19c420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f51fc037040 tx=0x7f51fc035d10 comp rx=0 tx=0).stop 2024-03-12T04:21:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 shutdown_connections 2024-03-12T04:21:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51f40a1ba0 0x7f51f40a4060 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f520c19bf70 0x7f520c19c420 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520c105360 0x7f520c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 --2- 172.21.15.42:0/3422957180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f520c1002e0 0x7f520c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 >> 172.21.15.42:0/3422957180 conn(0x7f520c0771e0 msgr2=0x7f520c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 shutdown_connections 2024-03-12T04:21:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.270+0000 7f52132eb640 1 -- 172.21.15.42:0/3422957180 wait complete. 2024-03-12T04:21:01.284 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:01.284 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 -- 172.21.15.42:0/3875311328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb341053e0 msgr2=0x7fcb3410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 --2- 172.21.15.42:0/3875311328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb341053e0 0x7fcb3410cfb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009cd0 tx=0x7fcb2c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 -- 172.21.15.42:0/3875311328 shutdown_connections 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 --2- 172.21.15.42:0/3875311328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb341053e0 0x7fcb3410cfb0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 --2- 172.21.15.42:0/3875311328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb34100cc0 0x7fcb34104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 --2- 172.21.15.42:0/3875311328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb34100310 0x7fcb341006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 -- 172.21.15.42:0/3875311328 >> 172.21.15.42:0/3875311328 conn(0x7fcb340771e0 msgr2=0x7fcb340775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 -- 172.21.15.42:0/3875311328 shutdown_connections 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.386+0000 7fcb3c478640 1 -- 172.21.15.42:0/3875311328 wait complete. 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 Processor -- start 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 -- start start 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb34100310 0x7fcb34196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341053e0 0x7fcb34196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3419ae90 0x7fcb3419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3419b880 con 0x7fcb3419ae90 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3419b9f0 con 0x7fcb34100310 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3c478640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3419bb60 con 0x7fcb341053e0 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3419ae90 0x7fcb3419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3419ae90 0x7fcb3419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46348/0 (socket says 172.21.15.42:46348) 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 -- 172.21.15.42:0/189604446 learned_addr learned my addr 172.21.15.42:0/189604446 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3a214640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb34100310 0x7fcb34196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 -- 172.21.15.42:0/189604446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341053e0 msgr2=0x7fcb34196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341053e0 0x7fcb34196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 -- 172.21.15.42:0/189604446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb34100310 msgr2=0x7fcb34196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb34100310 0x7fcb34196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.387+0000 7fcb3aa15640 1 -- 172.21.15.42:0/189604446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009900 con 0x7fcb3419ae90 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.388+0000 7fcb3a214640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb34100310 0x7fcb34196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.388+0000 7fcb3aa15640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3419ae90 0x7fcb3419b340 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c004af0 tx=0x7fcb2c035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.388+0000 7fcb2b7fe640 1 -- 172.21.15.42:0/189604446 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0329e0 con 0x7fcb3419ae90 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.388+0000 7fcb2b7fe640 1 -- 172.21.15.42:0/189604446 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb2c035e90 con 0x7fcb3419ae90 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.388+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb341a3140 con 0x7fcb3419ae90 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.388+0000 7fcb2b7fe640 1 -- 172.21.15.42:0/189604446 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c004720 con 0x7fcb3419ae90 2024-03-12T04:21:01.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.388+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb341a3560 con 0x7fcb3419ae90 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.390+0000 7fcb2b7fe640 1 -- 172.21.15.42:0/189604446 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fcb2c03b040 con 0x7fcb3419ae90 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.390+0000 7fcb3f045740 1 Processor -- start 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3f045740 1 -- start start 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c841f10 0x559f4c8422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c84d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c84d960 0x559f4c84fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3f045740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c7fd960 con 0x559f4c842830 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3f045740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c7b1310 con 0x559f4c841f10 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3f045740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c7ae840 con 0x559f4c84d960 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb2b7fe640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb200a1ba0 0x7fcb200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c84d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c84d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60244/0 (socket says 172.21.15.42:60244) 2024-03-12T04:21:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 -- 172.21.15.42:0/3320177586 learned_addr learned my addr 172.21.15.42:0/3320177586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3aa15640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb200a1ba0 0x7fcb200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c84d960 0x559f4c84fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c841f10 0x559f4c8422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 -- 172.21.15.42:0/3320177586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c84d960 msgr2=0x559f4c84fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c84d960 0x559f4c84fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 -- 172.21.15.42:0/3320177586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c841f10 msgr2=0x559f4c8422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c841f10 0x559f4c8422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb2b7fe640 1 -- 172.21.15.42:0/189604446 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb2c0f02c0 con 0x7fcb3419ae90 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb3a214640 1 -- 172.21.15.42:0/3320177586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f4c76c060 con 0x559f4c842830 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.391+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c841f10 0x559f4c8422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.392+0000 7fcb3a214640 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c84d420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009c50 tx=0x7fcb3000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6358dbf7b040ea31 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.392+0000 7fcb3aa15640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb200a1ba0 0x7fcb200a4060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b99c0 tx=0x7fcb2c0e76b0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.392+0000 7fcb28bfe640 1 -- 172.21.15.42:0/3320177586 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30005020 con 0x559f4c842830 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.392+0000 7fcb28bfe640 1 -- 172.21.15.42:0/3320177586 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb30005630 con 0x559f4c842830 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.392+0000 7fcb28bfe640 1 -- 172.21.15.42:0/3320177586 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300116d0 con 0x559f4c842830 2024-03-12T04:21:01.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.392+0000 7fcb3f045740 1 -- 172.21.15.42:0/3320177586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 msgr2=0x559f4c84d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.392+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c84d420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009c50 tx=0x7fcb3000e6c0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- 172.21.15.42:0/3320177586 shutdown_connections 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c84d960 0x559f4c84fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c84d420 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3320177586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c841f10 0x559f4c8422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- 172.21.15.42:0/3320177586 >> 172.21.15.42:0/3320177586 conn(0x559f4c83c980 msgr2=0x559f4c83cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- 172.21.15.42:0/3320177586 shutdown_connections 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- 172.21.15.42:0/3320177586 wait complete. 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- start start 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 Processor -- start 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c84aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c7aa020 con 0x559f4c84d960 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c47d430 con 0x559f4c842830 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c439e00 con 0x559f4c841f10 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3f045740 10 client.0 _ll_register_callbacks cb 0x559f4c751cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb39a13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb39a13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60258/0 (socket says 172.21.15.42:60258) 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb39a13640 1 -- 172.21.15.42:0/173142391 learned_addr learned my addr 172.21.15.42:0/173142391 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.393+0000 7fcb3a214640 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c84aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb39a13640 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb39a13640 1 -- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 msgr2=0x559f4c84a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb39a13640 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb39a13640 1 -- 172.21.15.42:0/173142391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 msgr2=0x559f4c84aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb39a13640 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c84aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb39a13640 1 -- 172.21.15.42:0/173142391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb30009880 con 0x559f4c84d960 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb39a13640 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b430 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fcb34069620 tx=0x7fcb2400d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25f1bae1aaabe0a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb1dffb640 1 -- 172.21.15.42:0/173142391 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24005220 con 0x559f4c84d960 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb1dffb640 1 -- 172.21.15.42:0/173142391 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb24005380 con 0x559f4c84d960 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f4c4a97e0 con 0x559f4c84d960 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb1dffb640 1 -- 172.21.15.42:0/173142391 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb24005540 con 0x559f4c84d960 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb3f045740 10 client.17127 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f4c7a86a0 con 0x559f4c84d960 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb1dffb640 1 -- 172.21.15.42:0/173142391 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x559f4c4a97e0 con 0x559f4c84d960 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb1dffb640 1 -- 172.21.15.42:0/173142391 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb2400fa00 con 0x559f4c84d960 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb3f045740 10 client.17127 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb3f045740 10 client.17127 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.394+0000 7fcb3f045740 20 client.17127 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.395+0000 7fcb3f045740 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c85dc50 0x559f4c860040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.395+0000 7fcb3f045740 4 client.17127 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.395+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f4c4e6720 con 0x559f4c85dc50 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.395+0000 7fcb3a214640 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c85dc50 0x559f4c860040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.395+0000 7fcb3a214640 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c85dc50 0x559f4c860040 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37f98491dcf7c4eb server_cookie=97b65f99ada5d7ae in_seq=0 out_seq=0 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.395+0000 7fcb1dffb640 10 client.17127 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb1dffb640 1 -- 172.21.15.42:0/173142391 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f4c4e6720 con 0x559f4c85dc50 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb1dffb640 10 client.17127 handle_command_reply: tid=0 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 client.17127 shutdown 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 msgr2=0x559f4c84b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b430 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fcb34069620 tx=0x7fcb2400d660 comp rx=0 tx=0).stop 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 shutdown_connections 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c85dc50 0x559f4c860040 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b430 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c84aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb1dffb640 0 client.17127 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 --2- 172.21.15.42:0/173142391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 >> 172.21.15.42:0/173142391 conn(0x559f4c83c980 msgr2=0x559f4c83e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 shutdown_connections 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 1 -- 172.21.15.42:0/173142391 wait complete. 2024-03-12T04:21:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.396+0000 7fcb3f045740 20 client.17127 trim_cache size 0 max 16384 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 Processor -- start 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 -- start start 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c854c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c72f130 con 0x559f4c84d960 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c85a8d0 con 0x559f4c842830 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3f045740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c865250 con 0x559f4c841f10 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60260/0 (socket says 172.21.15.42:60260) 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 -- 172.21.15.42:0/819411254 learned_addr learned my addr 172.21.15.42:0/819411254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb3a214640 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c854c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 -- 172.21.15.42:0/819411254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 msgr2=0x559f4c84d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 -- 172.21.15.42:0/819411254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 msgr2=0x559f4c854c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.408+0000 7fcb39a13640 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c854c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb39a13640 1 -- 172.21.15.42:0/819411254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f4c744c20 con 0x559f4c84d960 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb39a13640 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b250 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400dbc0 tx=0x7fcb2400f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999cb71799005c67 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb1ffff640 1 -- 172.21.15.42:0/819411254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24012230 con 0x559f4c84d960 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb1ffff640 1 -- 172.21.15.42:0/819411254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb2400d6d0 con 0x559f4c84d960 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb1ffff640 1 -- 172.21.15.42:0/819411254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240047a0 con 0x559f4c84d960 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 -- 172.21.15.42:0/819411254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 msgr2=0x559f4c84b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b250 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400dbc0 tx=0x7fcb2400f5d0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 -- 172.21.15.42:0/819411254 shutdown_connections 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c84d960 0x559f4c84b250 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c842830 0x559f4c854c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 --2- 172.21.15.42:0/819411254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c84d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 -- 172.21.15.42:0/819411254 >> 172.21.15.42:0/819411254 conn(0x559f4c83c980 msgr2=0x559f4c845ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 -- 172.21.15.42:0/819411254 shutdown_connections 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.409+0000 7fcb3f045740 1 -- 172.21.15.42:0/819411254 wait complete. 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 -- start start 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 Processor -- start 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c8b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c8b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c84d960 0x559f4c8aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c8ab190 con 0x559f4c842830 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c8ab300 con 0x559f4c84d960 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4c8ab4c0 con 0x559f4c841f10 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3f045740 10 client.0 _ll_register_callbacks cb 0x7fcb30009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c8b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c8b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60266/0 (socket says 172.21.15.42:60266) 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 -- 172.21.15.42:0/3936983682 learned_addr learned my addr 172.21.15.42:0/3936983682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c8b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c84d960 0x559f4c8aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 -- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 msgr2=0x559f4c8b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c8b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 -- 172.21.15.42:0/3936983682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c84d960 msgr2=0x559f4c8aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c84d960 0x559f4c8aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb3a214640 1 -- 172.21.15.42:0/3936983682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f4c7a86a0 con 0x559f4c842830 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.410+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c84d960 0x559f4c8aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3a214640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c8b0d00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x559f4c83d9a0 tx=0x7fcb30009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718855339503b816 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb1dffb640 1 -- 172.21.15.42:0/3936983682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30021040 con 0x559f4c842830 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb1dffb640 1 -- 172.21.15.42:0/3936983682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb30004890 con 0x559f4c842830 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb30005630 con 0x559f4c842830 2024-03-12T04:21:01.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb1dffb640 1 -- 172.21.15.42:0/3936983682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb30004c80 con 0x559f4c842830 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 10 client.17133 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f4c76a8c0 con 0x559f4c842830 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb1dffb640 1 -- 172.21.15.42:0/3936983682 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb30005630 con 0x559f4c842830 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb1dffb640 1 -- 172.21.15.42:0/3936983682 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb3000c070 con 0x559f4c842830 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 10 client.17133 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 10 client.17133 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 20 client.17133 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c8b87b0 0x559f4c8babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 4 client.17133 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f4c535890 con 0x559f4c8b87b0 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.411+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c8b87b0 0x559f4c8babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb1dffb640 10 client.17133 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb39a13640 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c8b87b0 0x559f4c8babd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f93c50ad68bbec71 server_cookie=c120acc70bdf1d1d in_seq=0 out_seq=0 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb1dffb640 1 -- 172.21.15.42:0/3936983682 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559f4c535890 con 0x559f4c8b87b0 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb1dffb640 10 client.17133 handle_command_reply: tid=0 2024-03-12T04:21:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 client.17133 shutdown 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 msgr2=0x559f4c8b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c8b0d00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x559f4c83d9a0 tx=0x7fcb30009ea0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 shutdown_connections 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x559f4c8b87b0 0x559f4c8babd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f4c84d960 0x559f4c8aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559f4c842830 0x559f4c8b0d00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 --2- 172.21.15.42:0/3936983682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4c841f10 0x559f4c8b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb1dffb640 0 client.17133 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 >> 172.21.15.42:0/3936983682 conn(0x559f4c83c980 msgr2=0x559f4c83ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 shutdown_connections 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 1 -- 172.21.15.42:0/3936983682 wait complete. 2024-03-12T04:21:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.412+0000 7fcb3f045740 20 client.17133 trim_cache size 0 max 16384 2024-03-12T04:21:01.422 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb200a1ba0 msgr2=0x7fcb200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb200a1ba0 0x7fcb200a4060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b99c0 tx=0x7fcb2c0e76b0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3419ae90 msgr2=0x7fcb3419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3419ae90 0x7fcb3419b340 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c004af0 tx=0x7fcb2c035ca0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 shutdown_connections 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb200a1ba0 0x7fcb200a4060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3419ae90 0x7fcb3419b340 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341053e0 0x7fcb34196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 --2- 172.21.15.42:0/189604446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb34100310 0x7fcb34196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 >> 172.21.15.42:0/189604446 conn(0x7fcb340771e0 msgr2=0x7fcb340fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 shutdown_connections 2024-03-12T04:21:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.413+0000 7fcb3c478640 1 -- 172.21.15.42:0/189604446 wait complete. 2024-03-12T04:21:01.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:01 smithi042 ceph-mon[32260]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:01.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:01 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2738430450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:21:01.426 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:01.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:21:01.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:01 smithi119 ceph-mon[36119]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:01.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:01 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2738430450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:21:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:01 smithi136 ceph-mon[35571]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:01.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:01 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2738430450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 -- 172.21.15.42:0/4235157370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7010b7c0 msgr2=0x7fcb7010dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 --2- 172.21.15.42:0/4235157370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7010b7c0 0x7fcb7010dc30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009ca0 tx=0x7fcb6802c180 comp rx=0 tx=0).stop 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 -- 172.21.15.42:0/4235157370 shutdown_connections 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 --2- 172.21.15.42:0/4235157370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7010b7c0 0x7fcb7010dc30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 --2- 172.21.15.42:0/4235157370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb70102d10 0x7fcb7010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 --2- 172.21.15.42:0/4235157370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb701023f0 0x7fcb701027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 -- 172.21.15.42:0/4235157370 >> 172.21.15.42:0/4235157370 conn(0x7fcb700771e0 msgr2=0x7fcb700775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 -- 172.21.15.42:0/4235157370 shutdown_connections 2024-03-12T04:21:01.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 -- 172.21.15.42:0/4235157370 wait complete. 2024-03-12T04:21:01.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.529+0000 7fcb75e98640 1 Processor -- start 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb75e98640 1 -- start start 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb75e98640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb701023f0 0x7fcb701976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb75e98640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb70102d10 0x7fcb70197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb75e98640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb7010b7c0 0x7fcb7019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb75e98640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7019c5a0 con 0x7fcb70102d10 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb75e98640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7019c710 con 0x7fcb7010b7c0 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb75e98640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7019c8d0 con 0x7fcb701023f0 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb70102d10 0x7fcb70197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb70102d10 0x7fcb70197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60278/0 (socket says 172.21.15.42:60278) 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 -- 172.21.15.42:0/1206771734 learned_addr learned my addr 172.21.15.42:0/1206771734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb7010b7c0 0x7fcb7019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb701023f0 0x7fcb701976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 -- 172.21.15.42:0/1206771734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb701023f0 msgr2=0x7fcb701976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb701023f0 0x7fcb701976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 -- 172.21.15.42:0/1206771734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb7010b7c0 msgr2=0x7fcb7019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb7010b7c0 0x7fcb7019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6effd640 1 -- 172.21.15.42:0/1206771734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68009900 con 0x7fcb70102d10 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.530+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb701023f0 0x7fcb701976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.531+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb70102d10 0x7fcb70197c20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d950 tx=0x7fcb6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.531+0000 7fcb6cff9640 1 -- 172.21.15.42:0/1206771734 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64009c30 con 0x7fcb70102d10 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.531+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7019cbb0 con 0x7fcb70102d10 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.531+0000 7fcb6cff9640 1 -- 172.21.15.42:0/1206771734 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb64009d90 con 0x7fcb70102d10 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.531+0000 7fcb6cff9640 1 -- 172.21.15.42:0/1206771734 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400f6b0 con 0x7fcb70102d10 2024-03-12T04:21:01.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.531+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb701a4430 con 0x7fcb70102d10 2024-03-12T04:21:01.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.533+0000 7fcb6cff9640 1 -- 172.21.15.42:0/1206771734 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fcb64010aa0 con 0x7fcb70102d10 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 Processor -- start 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 -- start start 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac190630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac199210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac199750 0x5585ac19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac14b960 con 0x5585ac190250 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac0ff310 con 0x5585ac190b70 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac0fc840 con 0x5585ac199750 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac190630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac190630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60292/0 (socket says 172.21.15.42:60292) 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1121114637 learned_addr learned my addr 172.21.15.42:0/1121114637 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6cff9640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb580a1ba0 0x7fcb580a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac199750 0x5585ac19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1121114637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac199750 msgr2=0x5585ac19bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac199750 0x5585ac19bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1121114637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 msgr2=0x5585ac199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1121114637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ac0ba060 con 0x5585ac190250 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6cff9640 1 -- 172.21.15.42:0/1206771734 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb640cd3f0 con 0x7fcb70102d10 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.534+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb580a1ba0 0x7fcb580a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac190630 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcb6802c790 tx=0x7fcb68032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991911ac954cc9f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb55ffb640 1 -- 172.21.15.42:0/1121114637 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68003c30 con 0x5585ac190250 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb55ffb640 1 -- 172.21.15.42:0/1121114637 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb68003d90 con 0x5585ac190250 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb55ffb640 1 -- 172.21.15.42:0/1121114637 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68004320 con 0x5585ac190250 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1121114637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 msgr2=0x5585ac190630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb580a1ba0 0x7fcb580a4060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb6804d000 tx=0x7fcb6804a000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac190630 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcb6802c790 tx=0x7fcb68032460 comp rx=0 tx=0).stop 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1121114637 shutdown_connections 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac199750 0x5585ac19bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1121114637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac190630 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.535+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1121114637 >> 172.21.15.42:0/1121114637 conn(0x5585ac18a980 msgr2=0x5585ac18ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1121114637 shutdown_connections 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1121114637 wait complete. 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 -- start start 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 Processor -- start 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac196740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190b70 0x5585ac196c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac0f8020 con 0x5585ac190250 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585abdcb430 con 0x5585ac199750 2024-03-12T04:21:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585abd87e00 con 0x5585ac190b70 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb78a5f740 10 client.0 _ll_register_callbacks cb 0x5585ac09fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb6f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60308/0 (socket says 172.21.15.42:60308) 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/2044188615 learned_addr learned my addr 172.21.15.42:0/2044188615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.536+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6effd640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190b70 0x5585ac196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/2044188615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190b70 msgr2=0x5585ac196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190b70 0x5585ac196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 msgr2=0x5585ac1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/2044188615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7010e8b0 con 0x5585ac190250 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6effd640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190b70 0x5585ac196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac196740 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcb60010670 tx=0x7fcb60010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5046e0dd79c650a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb4effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb4effd640 1 -- 172.21.15.42:0/2044188615 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000ead0 con 0x5585ac190250 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb4effd640 1 -- 172.21.15.42:0/2044188615 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb60016070 con 0x5585ac190250 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585abdf77e0 con 0x5585ac190250 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb4effd640 1 -- 172.21.15.42:0/2044188615 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb6000ec30 con 0x5585ac190250 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb78a5f740 10 client.17145 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5585ac0f66a0 con 0x5585ac190250 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.537+0000 7fcb4effd640 1 -- 172.21.15.42:0/2044188615 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5585abdf77e0 con 0x5585ac190250 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb4effd640 1 -- 172.21.15.42:0/2044188615 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb6001a420 con 0x5585ac190250 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb78a5f740 10 client.17145 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb78a5f740 10 client.17145 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb78a5f740 20 client.17145 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac1a9bc0 0x5585ac1ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb78a5f740 4 client.17145 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585abe34720 con 0x5585ac1a9bc0 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb6effd640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac1a9bc0 0x5585ac1ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb6effd640 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac1a9bc0 0x5585ac1ac000 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acd13ca8868880aa server_cookie=b17132d037bbe353 in_seq=0 out_seq=0 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.538+0000 7fcb4effd640 10 client.17145 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb4effd640 1 -- 172.21.15.42:0/2044188615 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585abe34720 con 0x5585ac1a9bc0 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb4effd640 10 client.17145 handle_command_reply: tid=0 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 client.17145 shutdown 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 msgr2=0x5585ac196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac196740 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcb60010670 tx=0x7fcb60010b30 comp rx=0 tx=0).stop 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 shutdown_connections 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac1a9bc0 0x5585ac1ac000 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190b70 0x5585ac196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb4effd640 0 client.17145 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/2044188615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190250 0x5585ac196740 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 >> 172.21.15.42:0/2044188615 conn(0x5585ac18a980 msgr2=0x5585ac18c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 shutdown_connections 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 1 -- 172.21.15.42:0/2044188615 wait complete. 2024-03-12T04:21:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.539+0000 7fcb78a5f740 20 client.17145 trim_cache size 0 max 16384 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 Processor -- start 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 -- start start 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac198e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190b70 0x5585ac1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac07d130 con 0x5585ac190b70 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac1862b0 con 0x5585ac199750 2024-03-12T04:21:01.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac194c90 con 0x5585ac190250 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb6effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190b70 0x5585ac1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb6effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190b70 0x5585ac1a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60324/0 (socket says 172.21.15.42:60324) 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.551+0000 7fcb6effd640 1 -- 172.21.15.42:0/1091867687 learned_addr learned my addr 172.21.15.42:0/1091867687 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6effd640 1 -- 172.21.15.42:0/1091867687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 msgr2=0x5585ac198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6effd640 1 -- 172.21.15.42:0/1091867687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 msgr2=0x5585ac196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6effd640 1 -- 172.21.15.42:0/1091867687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ac092c20 con 0x5585ac190b70 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190b70 0x5585ac1a09e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fcb64096a40 tx=0x7fcb6409a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1045236415417f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb557fa640 1 -- 172.21.15.42:0/1091867687 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640a2810 con 0x5585ac190b70 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb557fa640 1 -- 172.21.15.42:0/1091867687 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb64098370 con 0x5585ac190b70 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb557fa640 1 -- 172.21.15.42:0/1091867687 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6409a2d0 con 0x5585ac190b70 2024-03-12T04:21:01.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1091867687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190b70 msgr2=0x5585ac1a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190b70 0x5585ac1a09e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fcb64096a40 tx=0x7fcb6409a830 comp rx=0 tx=0).stop 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1091867687 shutdown_connections 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac199750 0x5585ac196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac190b70 0x5585ac1a09e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1091867687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.552+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1091867687 >> 172.21.15.42:0/1091867687 conn(0x5585ac18a980 msgr2=0x5585ac185a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1091867687 shutdown_connections 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1091867687 wait complete. 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 -- start start 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 Processor -- start 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac1fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac1fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac199750 0x5585ac1f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac1f8dc0 con 0x5585ac199750 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac1f8f30 con 0x5585ac190b70 2024-03-12T04:21:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ac1f90f0 con 0x5585ac190250 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb78a5f740 10 client.0 _ll_register_callbacks cb 0x7fcb68009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac199750 0x5585ac1f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac199750 0x5585ac1f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60334/0 (socket says 172.21.15.42:60334) 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1400060742 learned_addr learned my addr 172.21.15.42:0/1400060742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.553+0000 7fcb6effd640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac1fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac1fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1400060742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 msgr2=0x5585ac1fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac1fe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 msgr2=0x5585ac1fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac1fe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6ffff640 1 -- 172.21.15.42:0/1400060742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb701a4430 con 0x5585ac199750 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac1fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb6ffff640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac199750 0x5585ac1f8800 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009ca0 tx=0x7fcb680323f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddea70c51da7f10 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb4effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb4effd640 1 -- 172.21.15.42:0/1400060742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6805a040 con 0x5585ac199750 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb4effd640 1 -- 172.21.15.42:0/1400060742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb68004240 con 0x5585ac199750 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb68003d90 con 0x5585ac199750 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb4effd640 1 -- 172.21.15.42:0/1400060742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb68032680 con 0x5585ac199750 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb78a5f740 10 client.17151 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5585ac0b88c0 con 0x5585ac199750 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb4effd640 1 -- 172.21.15.42:0/1400060742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb68003d90 con 0x5585ac199750 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.554+0000 7fcb4effd640 1 -- 172.21.15.42:0/1400060742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb6804a6a0 con 0x5585ac199750 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 10 client.17151 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 10 client.17151 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 20 client.17151 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac205fe0 0x5585ac208400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 4 client.17151 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585abe83890 con 0x5585ac205fe0 2024-03-12T04:21:01.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac205fe0 0x5585ac208400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb4effd640 10 client.17151 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac205fe0 0x5585ac208400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb4010a1690a4072 server_cookie=12ba0b471d8cec62 in_seq=0 out_seq=0 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb4effd640 1 -- 172.21.15.42:0/1400060742 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585abe83890 con 0x5585ac205fe0 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb4effd640 10 client.17151 handle_command_reply: tid=0 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 1 client.17151 shutdown 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac199750 msgr2=0x5585ac1f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.555+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac199750 0x5585ac1f8800 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009ca0 tx=0x7fcb680323f0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 shutdown_connections 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5585ac205fe0 0x5585ac208400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585ac199750 0x5585ac1f8800 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ac190b70 0x5585ac1fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 --2- 172.21.15.42:0/1400060742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ac190250 0x5585ac1fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb4effd640 0 client.17151 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 >> 172.21.15.42:0/1400060742 conn(0x5585ac18a980 msgr2=0x5585ac1b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 shutdown_connections 2024-03-12T04:21:01.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 1 -- 172.21.15.42:0/1400060742 wait complete. 2024-03-12T04:21:01.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb78a5f740 20 client.17151 trim_cache size 0 max 16384 2024-03-12T04:21:01.565 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb580a1ba0 msgr2=0x7fcb580a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb580a1ba0 0x7fcb580a4060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb6804d000 tx=0x7fcb6804a000 comp rx=0 tx=0).stop 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb70102d10 msgr2=0x7fcb70197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb70102d10 0x7fcb70197c20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d950 tx=0x7fcb6400de10 comp rx=0 tx=0).stop 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 shutdown_connections 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fcb580a1ba0 0x7fcb580a4060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb7010b7c0 0x7fcb7019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb70102d10 0x7fcb70197c20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 --2- 172.21.15.42:0/1206771734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb701023f0 0x7fcb701976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 >> 172.21.15.42:0/1206771734 conn(0x7fcb700771e0 msgr2=0x7fcb700fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 shutdown_connections 2024-03-12T04:21:01.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.556+0000 7fcb75e98640 1 -- 172.21.15.42:0/1206771734 wait complete. 2024-03-12T04:21:01.570 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:01.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.671+0000 7f1795135640 1 -- 172.21.15.42:0/124985575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 msgr2=0x7f179010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.671+0000 7f1795135640 1 --2- 172.21.15.42:0/124985575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f179010cf00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1788009cd0 tx=0x7f178802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.671+0000 7f1795135640 1 -- 172.21.15.42:0/124985575 shutdown_connections 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.671+0000 7f1795135640 1 --2- 172.21.15.42:0/124985575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f179010cf00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.671+0000 7f1795135640 1 --2- 172.21.15.42:0/124985575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1790100c10 0x7f1790104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.671+0000 7f1795135640 1 --2- 172.21.15.42:0/124985575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1790100260 0x7f1790100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.671+0000 7f1795135640 1 -- 172.21.15.42:0/124985575 >> 172.21.15.42:0/124985575 conn(0x7f1790077180 msgr2=0x7f1790077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.672+0000 7f1795135640 1 -- 172.21.15.42:0/124985575 shutdown_connections 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.672+0000 7f1795135640 1 -- 172.21.15.42:0/124985575 wait complete. 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.672+0000 7f1795135640 1 Processor -- start 2024-03-12T04:21:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.672+0000 7f1795135640 1 -- start start 2024-03-12T04:21:01.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1790100260 0x7f1790101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f1790102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1790102770 0x7f17901a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a5e30 con 0x7f1790105330 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a5fa0 con 0x7f1790100260 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a6160 con 0x7f1790102770 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f1790102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f1790102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60344/0 (socket says 172.21.15.42:60344) 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 -- 172.21.15.42:0/2096704776 learned_addr learned my addr 172.21.15.42:0/2096704776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1790100260 0x7f1790101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 -- 172.21.15.42:0/2096704776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1790102770 msgr2=0x7f17901a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1790102770 0x7f17901a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 -- 172.21.15.42:0/2096704776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1790100260 msgr2=0x7f1790101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1790100260 0x7f1790101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 -- 172.21.15.42:0/2096704776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009900 con 0x7f1790105330 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1790100260 0x7f1790101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f178e59c640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f1790102230 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f178400e9c0 tx=0x7f178400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1777fff640 1 -- 172.21.15.42:0/2096704776 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784009700 con 0x7f1790105330 2024-03-12T04:21:01.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901a6440 con 0x7f1790105330 2024-03-12T04:21:01.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1777fff640 1 -- 172.21.15.42:0/2096704776 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f178400cd80 con 0x7f1790105330 2024-03-12T04:21:01.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1777fff640 1 -- 172.21.15.42:0/2096704776 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784010430 con 0x7f1790105330 2024-03-12T04:21:01.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.673+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a6950 con 0x7f1790105330 2024-03-12T04:21:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.675+0000 7f1777fff640 1 -- 172.21.15.42:0/2096704776 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f1784004f10 con 0x7f1790105330 2024-03-12T04:21:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1777fff640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17780a1bd0 0x7f17780a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 Processor -- start 2024-03-12T04:21:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 -- start start 2024-03-12T04:21:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17780a1bd0 0x7f17780a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1777fff640 1 -- 172.21.15.42:0/2096704776 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1784014070 con 0x7f1790105330 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1250 0x55b0245e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1b70 0x55b0245ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245ea750 0x55b0245ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02459c960 con 0x55b0245ea750 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b024550310 con 0x55b0245e1b70 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f1797ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02454d840 con 0x55b0245e1250 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f178f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245ea750 0x55b0245ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f178f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245ea750 0x55b0245ecb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60350/0 (socket says 172.21.15.42:60350) 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f178f59e640 1 -- 172.21.15.42:0/2578833676 learned_addr learned my addr 172.21.15.42:0/2578833676 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.676+0000 7f178e59c640 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1b70 0x55b0245ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1250 0x55b0245e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178f59e640 1 -- 172.21.15.42:0/2578833676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1250 msgr2=0x55b0245e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178f59e640 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1250 0x55b0245e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17780a1bd0 0x7f17780a4090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1780009b50 tx=0x7f1780009f50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178f59e640 1 -- 172.21.15.42:0/2578833676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1b70 msgr2=0x55b0245ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178f59e640 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1b70 0x55b0245ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178f59e640 1 -- 172.21.15.42:0/2578833676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b02450b060 con 0x55b0245ea750 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f178f59e640 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245ea750 0x55b0245ecb40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1788037040 tx=0x7f1788005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e8bd4a38243030 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f17757fa640 1 -- 172.21.15.42:0/2578833676 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788032440 con 0x55b0245ea750 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f17757fa640 1 -- 172.21.15.42:0/2578833676 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1788032e90 con 0x55b0245ea750 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.677+0000 7f17757fa640 1 -- 172.21.15.42:0/2578833676 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788003970 con 0x55b0245ea750 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- 172.21.15.42:0/2578833676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245ea750 msgr2=0x55b0245ecb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245ea750 0x55b0245ecb40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1788037040 tx=0x7f1788005f60 comp rx=0 tx=0).stop 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- 172.21.15.42:0/2578833676 shutdown_connections 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245ea750 0x55b0245ecb40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1b70 0x55b0245ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2578833676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1250 0x55b0245e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- 172.21.15.42:0/2578833676 >> 172.21.15.42:0/2578833676 conn(0x55b0245db980 msgr2=0x55b0245dbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- 172.21.15.42:0/2578833676 shutdown_connections 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- 172.21.15.42:0/2578833676 wait complete. 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- start start 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 Processor -- start 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b0245e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b024549020 con 0x55b0245e1250 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02421c430 con 0x55b0245ea750 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0241d8e00 con 0x55b0245e1b70 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.678+0000 7f1797ddf740 10 client.0 _ll_register_callbacks cb 0x55b0244f0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60360/0 (socket says 172.21.15.42:60360) 2024-03-12T04:21:01.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 -- 172.21.15.42:0/2964029943 learned_addr learned my addr 172.21.15.42:0/2964029943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178f59e640 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b0245e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178e59c640 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 -- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 msgr2=0x55b0245e7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245e7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 -- 172.21.15.42:0/2964029943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 msgr2=0x55b0245e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b0245e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 -- 172.21.15.42:0/2964029943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179010d710 con 0x55b0245e1250 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f178ed9d640 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245e7710 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1780012720 tx=0x7f1780004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9001ef0e1de5a34e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f1772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f1772ff5640 1 -- 172.21.15.42:0/2964029943 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780004020 con 0x55b0245e1250 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f1772ff5640 1 -- 172.21.15.42:0/2964029943 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1780004180 con 0x55b0245e1250 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0242487e0 con 0x55b0245e1250 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f1772ff5640 1 -- 172.21.15.42:0/2964029943 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1780004340 con 0x55b0245e1250 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f1797ddf740 10 client.17163 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.679+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b0245476a0 con 0x55b0245e1250 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1772ff5640 1 -- 172.21.15.42:0/2964029943 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b0242487e0 con 0x55b0245e1250 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1772ff5640 1 -- 172.21.15.42:0/2964029943 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f178001b050 con 0x55b0245e1250 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1797ddf740 10 client.17163 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1797ddf740 10 client.17163 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1797ddf740 20 client.17163 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b0245faad0 0x55b0245fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1797ddf740 4 client.17163 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b024285720 con 0x55b0245faad0 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f178e59c640 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b0245faad0 0x55b0245fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f178e59c640 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b0245faad0 0x55b0245fcee0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ee38f4ce01c3498 server_cookie=add52771d2cf7c74 in_seq=0 out_seq=0 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.680+0000 7f1772ff5640 10 client.17163 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1772ff5640 1 -- 172.21.15.42:0/2964029943 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b024285720 con 0x55b0245faad0 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1772ff5640 10 client.17163 handle_command_reply: tid=0 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 client.17163 shutdown 2024-03-12T04:21:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 msgr2=0x55b0245e7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245e7710 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1780012720 tx=0x7f1780004730 comp rx=0 tx=0).stop 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 shutdown_connections 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b0245faad0 0x55b0245fcee0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b0245e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1772ff5640 0 client.17163 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 --2- 172.21.15.42:0/2964029943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245e7710 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 >> 172.21.15.42:0/2964029943 conn(0x55b0245db980 msgr2=0x55b0245dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 shutdown_connections 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 1 -- 172.21.15.42:0/2964029943 wait complete. 2024-03-12T04:21:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.681+0000 7f1797ddf740 20 client.17163 trim_cache size 0 max 16384 2024-03-12T04:21:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 Processor -- start 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 -- start start 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1250 0x55b0245e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1b70 0x55b0245e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245ea750 0x55b0245e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0244ce130 con 0x55b0245e1b70 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0245e0c90 con 0x55b0245e1250 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f1797ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0245dcb40 con 0x55b0245ea750 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f178e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1b70 0x55b0245e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f178e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1b70 0x55b0245e7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60370/0 (socket says 172.21.15.42:60370) 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.693+0000 7f178e59c640 1 -- 172.21.15.42:0/67427539 learned_addr learned my addr 172.21.15.42:0/67427539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178ed9d640 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1250 0x55b0245e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178f59e640 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245ea750 0x55b0245e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178e59c640 1 -- 172.21.15.42:0/67427539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245ea750 msgr2=0x55b0245e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178e59c640 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245ea750 0x55b0245e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178e59c640 1 -- 172.21.15.42:0/67427539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1250 msgr2=0x55b0245e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178e59c640 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1250 0x55b0245e9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178e59c640 1 -- 172.21.15.42:0/67427539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0244e3c20 con 0x55b0245e1b70 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178f59e640 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245ea750 0x55b0245e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f178e59c640 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1b70 0x55b0245e7c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f17840cb400 tx=0x7f1784085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66462553759478f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f1774ff9640 1 -- 172.21.15.42:0/67427539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178409e9d0 con 0x55b0245e1b70 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f1774ff9640 1 -- 172.21.15.42:0/67427539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17840a3020 con 0x55b0245e1b70 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.694+0000 7f1774ff9640 1 -- 172.21.15.42:0/67427539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840b31b0 con 0x55b0245e1b70 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- 172.21.15.42:0/67427539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1b70 msgr2=0x55b0245e7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1b70 0x55b0245e7c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f17840cb400 tx=0x7f1784085040 comp rx=0 tx=0).stop 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- 172.21.15.42:0/67427539 shutdown_connections 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245ea750 0x55b0245e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1b70 0x55b0245e7c80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 --2- 172.21.15.42:0/67427539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245e1250 0x55b0245e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- 172.21.15.42:0/67427539 >> 172.21.15.42:0/67427539 conn(0x55b0245db980 msgr2=0x55b0245e2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- 172.21.15.42:0/67427539 shutdown_connections 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- 172.21.15.42:0/67427539 wait complete. 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- start start 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 Processor -- start 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245f0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b02464b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02464b670 con 0x55b0245e1250 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02464b7e0 con 0x55b0245ea750 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02464b9a0 con 0x55b0245e1b70 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f1797ddf740 10 client.0 _ll_register_callbacks cb 0x7f1788037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245f0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60382/0 (socket says 172.21.15.42:60382) 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f178ed9d640 1 -- 172.21.15.42:0/3463884317 learned_addr learned my addr 172.21.15.42:0/3463884317 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.695+0000 7f178e59c640 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f178f59e640 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b02464b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f178ed9d640 1 -- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 msgr2=0x55b0245f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f178ed9d640 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f178ed9d640 1 -- 172.21.15.42:0/3463884317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 msgr2=0x55b02464b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f178ed9d640 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b02464b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f178ed9d640 1 -- 172.21.15.42:0/3463884317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a6950 con 0x55b0245e1250 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f178ed9d640 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245f0930 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1780005940 tx=0x7f178000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bd4c2ab4b52a63 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1772ff5640 1 -- 172.21.15.42:0/3463884317 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780011870 con 0x55b0245e1250 2024-03-12T04:21:01.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1788032e90 con 0x55b0245e1250 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1772ff5640 1 -- 172.21.15.42:0/3463884317 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f17800119d0 con 0x55b0245e1250 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1772ff5640 1 -- 172.21.15.42:0/3463884317 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1780011bd0 con 0x55b0245e1250 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1797ddf740 10 client.17169 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b0245098c0 con 0x55b0245e1250 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1772ff5640 1 -- 172.21.15.42:0/3463884317 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1788032e90 con 0x55b0245e1250 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1772ff5640 1 -- 172.21.15.42:0/3463884317 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f178001b040 con 0x55b0245e1250 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1797ddf740 10 client.17169 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1797ddf740 10 client.17169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1797ddf740 20 client.17169 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.696+0000 7f1797ddf740 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b024653150 0x55b024655570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 4 client.17169 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b0242d4890 con 0x55b024653150 2024-03-12T04:21:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f178e59c640 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b024653150 0x55b024655570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f178e59c640 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b024653150 0x55b024655570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c3f30718010811f server_cookie=88e50a16c660c5f7 in_seq=0 out_seq=0 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1772ff5640 10 client.17169 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1772ff5640 1 -- 172.21.15.42:0/3463884317 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b0242d4890 con 0x55b024653150 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1772ff5640 10 client.17169 handle_command_reply: tid=0 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 client.17169 shutdown 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 msgr2=0x55b0245f0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245f0930 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1780005940 tx=0x7f178000e8d0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 shutdown_connections 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b024653150 0x55b024655570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b0245ea750 0x55b02464b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1772ff5640 0 client.17169 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0245e1b70 0x55b0245f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 --2- 172.21.15.42:0/3463884317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0245e1250 0x55b0245f0930 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.697+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 >> 172.21.15.42:0/3463884317 conn(0x55b0245db980 msgr2=0x55b0245da2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 shutdown_connections 2024-03-12T04:21:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1797ddf740 1 -- 172.21.15.42:0/3463884317 wait complete. 2024-03-12T04:21:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1797ddf740 20 client.17169 trim_cache size 0 max 16384 2024-03-12T04:21:01.708 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17780a1bd0 msgr2=0x7f17780a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17780a1bd0 0x7f17780a4090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1780009b50 tx=0x7f1780009f50 comp rx=0 tx=0).stop 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 msgr2=0x7f1790102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f1790102230 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f178400e9c0 tx=0x7f178400ee80 comp rx=0 tx=0).stop 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 shutdown_connections 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17780a1bd0 0x7f17780a4090 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1790102770 0x7f17901a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790105330 0x7f1790102230 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 --2- 172.21.15.42:0/2096704776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1790100260 0x7f1790101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 >> 172.21.15.42:0/2096704776 conn(0x7f1790077180 msgr2=0x7f17900686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 shutdown_connections 2024-03-12T04:21:01.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.698+0000 7f1795135640 1 -- 172.21.15.42:0/2096704776 wait complete. 2024-03-12T04:21:01.712 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:01.712 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:21:01.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.815+0000 7fbc3353a640 1 -- 172.21.15.42:0/3149912828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 msgr2=0x7fbc2c10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.815+0000 7fbc3353a640 1 --2- 172.21.15.42:0/3149912828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c10cf00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009cd0 tx=0x7fbc1c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- 172.21.15.42:0/3149912828 shutdown_connections 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 --2- 172.21.15.42:0/3149912828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c10cf00 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 --2- 172.21.15.42:0/3149912828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc2c100c10 0x7fbc2c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 --2- 172.21.15.42:0/3149912828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100260 0x7fbc2c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- 172.21.15.42:0/3149912828 >> 172.21.15.42:0/3149912828 conn(0x7fbc2c077180 msgr2=0x7fbc2c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- 172.21.15.42:0/3149912828 shutdown_connections 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- 172.21.15.42:0/3149912828 wait complete. 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 Processor -- start 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- start start 2024-03-12T04:21:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100260 0x7fbc2c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc2c199740 0x7fbc2c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c19a210 con 0x7fbc2c105330 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c19a380 con 0x7fbc2c100260 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.816+0000 7fbc3353a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c19a540 con 0x7fbc2c199740 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c19fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60394/0 (socket says 172.21.15.42:60394) 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 -- 172.21.15.42:0/2571661330 learned_addr learned my addr 172.21.15.42:0/2571661330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc312d6640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100260 0x7fbc2c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc31ad7640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc2c199740 0x7fbc2c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 -- 172.21.15.42:0/2571661330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc2c199740 msgr2=0x7fbc2c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc2c199740 0x7fbc2c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 -- 172.21.15.42:0/2571661330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100260 msgr2=0x7fbc2c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100260 0x7fbc2c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 -- 172.21.15.42:0/2571661330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c009900 con 0x7fbc2c105330 2024-03-12T04:21:01.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc312d6640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100260 0x7fbc2c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:01.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc31ad7640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc2c199740 0x7fbc2c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c19fbc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000c970 tx=0x7fbc2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc2a7fc640 1 -- 172.21.15.42:0/2571661330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20007bb0 con 0x7fbc2c105330 2024-03-12T04:21:01.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c1a64e0 con 0x7fbc2c105330 2024-03-12T04:21:01.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc2a7fc640 1 -- 172.21.15.42:0/2571661330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbc20007d10 con 0x7fbc2c105330 2024-03-12T04:21:01.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc2a7fc640 1 -- 172.21.15.42:0/2571661330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20018660 con 0x7fbc2c105330 2024-03-12T04:21:01.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.817+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1a6970 con 0x7fbc2c105330 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.819+0000 7fbc2a7fc640 1 -- 172.21.15.42:0/2571661330 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fbc2000f450 con 0x7fbc2c105330 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 Processor -- start 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 -- start start 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc2a7fc640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbc140a1ba0 0x7fbc140a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb512630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb51b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb51db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb4cd960 con 0x55e5eb512b70 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb481310 con 0x55e5eb51b750 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc361e8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb47e840 con 0x55e5eb512250 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc2a7fc640 1 -- 172.21.15.42:0/2571661330 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbc200cd880 con 0x7fbc2c105330 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc31ad7640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbc140a1ba0 0x7fbc140a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb51b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb51b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60398/0 (socket says 172.21.15.42:60398) 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc30ad5640 1 -- 172.21.15.42:0/1560776277 learned_addr learned my addr 172.21.15.42:0/1560776277 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.820+0000 7fbc312d6640 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb512630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc30ad5640 1 -- 172.21.15.42:0/1560776277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 msgr2=0x55e5eb512630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb512630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc30ad5640 1 -- 172.21.15.42:0/1560776277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 msgr2=0x55e5eb51db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb51db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc30ad5640 1 -- 172.21.15.42:0/1560776277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5eb43c060 con 0x55e5eb512b70 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc31ad7640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbc140a1ba0 0x7fbc140a4060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c006010 tx=0x7fbc1c0324c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb51b210 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000cfd0 tx=0x7fbc20099e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71410692832a9a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc137fe640 1 -- 172.21.15.42:0/1560776277 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2009a500 con 0x55e5eb512b70 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc137fe640 1 -- 172.21.15.42:0/1560776277 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbc2009e650 con 0x55e5eb512b70 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.821+0000 7fbc137fe640 1 -- 172.21.15.42:0/1560776277 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000f990 con 0x55e5eb512b70 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- 172.21.15.42:0/1560776277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 msgr2=0x55e5eb51b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb51b210 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000cfd0 tx=0x7fbc20099e60 comp rx=0 tx=0).stop 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- 172.21.15.42:0/1560776277 shutdown_connections 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb51db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb51b210 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 --2- 172.21.15.42:0/1560776277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb512630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- 172.21.15.42:0/1560776277 >> 172.21.15.42:0/1560776277 conn(0x55e5eb50c980 msgr2=0x55e5eb50cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- 172.21.15.42:0/1560776277 shutdown_connections 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- 172.21.15.42:0/1560776277 wait complete. 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- start start 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 Processor -- start 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb518790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb518cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb519210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb47a020 con 0x55e5eb512b70 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb14d430 con 0x55e5eb51b750 2024-03-12T04:21:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.822+0000 7fbc361e8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb109e00 con 0x55e5eb512250 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc361e8740 10 client.0 _ll_register_callbacks cb 0x55e5eb421cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb518cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb518cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60406/0 (socket says 172.21.15.42:60406) 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 -- 172.21.15.42:0/264822314 learned_addr learned my addr 172.21.15.42:0/264822314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc312d6640 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb518790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 -- 172.21.15.42:0/264822314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 msgr2=0x55e5eb518790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb518790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 -- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 msgr2=0x55e5eb519210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc31ad7640 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb519210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb519210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 -- 172.21.15.42:0/264822314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1a6970 con 0x55e5eb512b70 2024-03-12T04:21:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb518cd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000c940 tx=0x7fbc2000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f8e207539efe30 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc10ff9640 1 -- 172.21.15.42:0/264822314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200ba0c0 con 0x55e5eb512b70 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc10ff9640 1 -- 172.21.15.42:0/264822314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbc2009a500 con 0x55e5eb512b70 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5eb1797e0 con 0x55e5eb512b70 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc10ff9640 1 -- 172.21.15.42:0/264822314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc20096b70 con 0x55e5eb512b70 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc361e8740 10 client.17181 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e5eb4786a0 con 0x55e5eb512b70 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.823+0000 7fbc10ff9640 1 -- 172.21.15.42:0/264822314 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e5eb1797e0 con 0x55e5eb512b70 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc10ff9640 1 -- 172.21.15.42:0/264822314 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbc20016020 con 0x55e5eb512b70 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc361e8740 10 client.17181 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc361e8740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc361e8740 20 client.17181 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc361e8740 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb52b610 0x55e5eb52da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc361e8740 4 client.17181 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5eb1b6720 con 0x55e5eb52b610 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.824+0000 7fbc312d6640 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb52b610 0x55e5eb52da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc10ff9640 10 client.17181 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc312d6640 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb52b610 0x55e5eb52da00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cdacfd82f9df876 server_cookie=31e07ba34a291e39 in_seq=0 out_seq=0 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc10ff9640 1 -- 172.21.15.42:0/264822314 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5eb1b6720 con 0x55e5eb52b610 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc10ff9640 10 client.17181 handle_command_reply: tid=0 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 client.17181 shutdown 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 msgr2=0x55e5eb518cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb518cd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000c940 tx=0x7fbc2000f770 comp rx=0 tx=0).stop 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 shutdown_connections 2024-03-12T04:21:01.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb52b610 0x55e5eb52da00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb519210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb518cd0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc10ff9640 0 client.17181 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 --2- 172.21.15.42:0/264822314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb518790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 >> 172.21.15.42:0/264822314 conn(0x55e5eb50c980 msgr2=0x55e5eb50e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 shutdown_connections 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.825+0000 7fbc361e8740 1 -- 172.21.15.42:0/264822314 wait complete. 2024-03-12T04:21:01.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.826+0000 7fbc361e8740 20 client.17181 trim_cache size 0 max 16384 2024-03-12T04:21:01.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 Processor -- start 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 -- start start 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512250 0x55e5eb5178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb512b70 0x55e5eb519030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb51b750 0x55e5eb50ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb3ff130 con 0x55e5eb512250 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb511eb0 con 0x55e5eb512b70 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.837+0000 7fbc361e8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb532c30 con 0x55e5eb51b750 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512250 0x55e5eb5178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512250 0x55e5eb5178f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60408/0 (socket says 172.21.15.42:60408) 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 -- 172.21.15.42:0/726603324 learned_addr learned my addr 172.21.15.42:0/726603324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc31ad7640 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb51b750 0x55e5eb50ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 -- 172.21.15.42:0/726603324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb51b750 msgr2=0x55e5eb50ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb51b750 0x55e5eb50ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 -- 172.21.15.42:0/726603324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb512b70 msgr2=0x55e5eb519030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb512b70 0x55e5eb519030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 -- 172.21.15.42:0/726603324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5eb414c20 con 0x55e5eb512250 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc31ad7640 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb51b750 0x55e5eb50ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc312d6640 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512250 0x55e5eb5178f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x55e5eb52e330 tx=0x7fbc24013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6181606d679e76 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc12ffd640 1 -- 172.21.15.42:0/726603324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24020030 con 0x55e5eb512250 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc12ffd640 1 -- 172.21.15.42:0/726603324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbc240132d0 con 0x55e5eb512250 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.838+0000 7fbc12ffd640 1 -- 172.21.15.42:0/726603324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24017040 con 0x55e5eb512250 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 -- 172.21.15.42:0/726603324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512250 msgr2=0x55e5eb5178f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512250 0x55e5eb5178f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x55e5eb52e330 tx=0x7fbc24013830 comp rx=0 tx=0).stop 2024-03-12T04:21:01.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 -- 172.21.15.42:0/726603324 shutdown_connections 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb51b750 0x55e5eb50ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb512b70 0x55e5eb519030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 --2- 172.21.15.42:0/726603324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512250 0x55e5eb5178f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 -- 172.21.15.42:0/726603324 >> 172.21.15.42:0/726603324 conn(0x55e5eb50c980 msgr2=0x55e5eb50b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 -- 172.21.15.42:0/726603324 shutdown_connections 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 -- 172.21.15.42:0/726603324 wait complete. 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 -- start start 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.839+0000 7fbc361e8740 1 Processor -- start 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb521190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb5216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb521c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb581c00 con 0x55e5eb512b70 2024-03-12T04:21:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb581d70 con 0x55e5eb51b750 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eb581f30 con 0x55e5eb512250 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 10 client.0 _ll_register_callbacks cb 0x7fbc200960e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb5216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb5216d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60420/0 (socket says 172.21.15.42:60420) 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 -- 172.21.15.42:0/441029126 learned_addr learned my addr 172.21.15.42:0/441029126 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc312d6640 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb521190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 -- 172.21.15.42:0/441029126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 msgr2=0x55e5eb521190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb521190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 -- 172.21.15.42:0/441029126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 msgr2=0x55e5eb521c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb521c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 -- 172.21.15.42:0/441029126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2400bac0 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc312d6640 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb521190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc30ad5640 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb5216d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbc20096910 tx=0x7fbc20098a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c172cfbfad0aff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc10ff9640 1 -- 172.21.15.42:0/441029126 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2009a2d0 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc10ff9640 1 -- 172.21.15.42:0/441029126 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbc2009a500 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc2009e650 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc10ff9640 1 -- 172.21.15.42:0/441029126 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc2009e170 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 10 client.17187 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.840+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e5eb43a8c0 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc10ff9640 1 -- 172.21.15.42:0/441029126 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbc2009e650 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc10ff9640 1 -- 172.21.15.42:0/441029126 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbc2009f570 con 0x55e5eb512b70 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc361e8740 10 client.17187 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc361e8740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc361e8740 20 client.17187 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc361e8740 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb583330 0x55e5eb585720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc361e8740 4 client.17187 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5eb205890 con 0x55e5eb583330 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc312d6640 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb583330 0x55e5eb585720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc10ff9640 10 client.17187 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.841+0000 7fbc312d6640 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb583330 0x55e5eb585720 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85103a8c44a8e480 server_cookie=618cc202b9f90887 in_seq=0 out_seq=0 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc10ff9640 1 -- 172.21.15.42:0/441029126 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5eb205890 con 0x55e5eb583330 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc10ff9640 10 client.17187 handle_command_reply: tid=0 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 client.17187 shutdown 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 msgr2=0x55e5eb5216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb5216d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbc20096910 tx=0x7fbc20098a10 comp rx=0 tx=0).stop 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 shutdown_connections 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e5eb583330 0x55e5eb585720 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5eb51b750 0x55e5eb521c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e5eb512b70 0x55e5eb5216d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc10ff9640 0 client.17187 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 --2- 172.21.15.42:0/441029126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5eb512250 0x55e5eb521190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 >> 172.21.15.42:0/441029126 conn(0x55e5eb50c980 msgr2=0x55e5eb50bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 shutdown_connections 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 1 -- 172.21.15.42:0/441029126 wait complete. 2024-03-12T04:21:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc361e8740 20 client.17187 trim_cache size 0 max 16384 2024-03-12T04:21:01.855 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbc140a1ba0 msgr2=0x7fbc140a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbc140a1ba0 0x7fbc140a4060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c006010 tx=0x7fbc1c0324c0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 msgr2=0x7fbc2c19fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c19fbc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000c970 tx=0x7fbc2000ce30 comp rx=0 tx=0).stop 2024-03-12T04:21:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 shutdown_connections 2024-03-12T04:21:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbc140a1ba0 0x7fbc140a4060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc2c199740 0x7fbc2c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc2c105330 0x7fbc2c19fbc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 --2- 172.21.15.42:0/2571661330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100260 0x7fbc2c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 >> 172.21.15.42:0/2571661330 conn(0x7fbc2c077180 msgr2=0x7fbc2c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 shutdown_connections 2024-03-12T04:21:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:01.842+0000 7fbc3353a640 1 -- 172.21.15.42:0/2571661330 wait complete. 2024-03-12T04:21:02.858 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:21:02.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:21:02.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:21:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.973+0000 7f011748a640 1 -- 172.21.15.42:0/3036310069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110100260 msgr2=0x7f0110100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.973+0000 7f011748a640 1 --2- 172.21.15.42:0/3036310069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110100260 0x7f0110100640 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0100009cd0 tx=0x7f010002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- 172.21.15.42:0/3036310069 shutdown_connections 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 --2- 172.21.15.42:0/3036310069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0110105330 0x7f011010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 --2- 172.21.15.42:0/3036310069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0110100c10 0x7f0110104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 --2- 172.21.15.42:0/3036310069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110100260 0x7f0110100640 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- 172.21.15.42:0/3036310069 >> 172.21.15.42:0/3036310069 conn(0x7f0110077180 msgr2=0x7f0110077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- 172.21.15.42:0/3036310069 shutdown_connections 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- 172.21.15.42:0/3036310069 wait complete. 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 Processor -- start 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- start start 2024-03-12T04:21:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0110100260 0x7f01101985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110105330 0x7f0110198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01101990b0 0x7f01101a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01101a3d00 con 0x7f0110105330 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01101a3e70 con 0x7f01101990b0 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.974+0000 7f011748a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01101a4030 con 0x7f0110100260 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110105330 0x7f0110198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110105330 0x7f0110198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60438/0 (socket says 172.21.15.42:60438) 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 -- 172.21.15.42:0/3514762301 learned_addr learned my addr 172.21.15.42:0/3514762301 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0115a27640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01101990b0 0x7f01101a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0115226640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0110100260 0x7f01101985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 -- 172.21.15.42:0/3514762301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0110100260 msgr2=0x7f01101985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0110100260 0x7f01101985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 -- 172.21.15.42:0/3514762301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01101990b0 msgr2=0x7f01101a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01101990b0 0x7f01101a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 -- 172.21.15.42:0/3514762301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009900 con 0x7f0110105330 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0115226640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0110100260 0x7f01101985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f0114a25640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110105330 0x7f0110198b20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f010400b450 tx=0x7f010400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f010e7fc640 1 -- 172.21.15.42:0/3514762301 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104011890 con 0x7f0110105330 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f010e7fc640 1 -- 172.21.15.42:0/3514762301 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01040103e0 con 0x7f0110105330 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01101a4310 con 0x7f0110105330 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f010e7fc640 1 -- 172.21.15.42:0/3514762301 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104011ea0 con 0x7f0110105330 2024-03-12T04:21:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.975+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01101a4820 con 0x7f0110105330 2024-03-12T04:21:02.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.978+0000 7f010e7fc640 1 -- 172.21.15.42:0/3514762301 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0104010550 con 0x7f0110105330 2024-03-12T04:21:02.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 Processor -- start 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 -- start start 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559464801e0 0x5559464805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x5559464896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x55594648a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559463eef90 con 0x555946480b90 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559463ec4c0 con 0x555946489cf0 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f011a056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559463e7ca0 con 0x5559464801e0 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x5559464896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x5559464896d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60450/0 (socket says 172.21.15.42:60450) 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 -- 172.21.15.42:0/1212475238 learned_addr learned my addr 172.21.15.42:0/1212475238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0114a25640 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x55594648a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f010e7fc640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f00f80a1ba0 0x7f00f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115a27640 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559464801e0 0x5559464805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 -- 172.21.15.42:0/1212475238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559464801e0 msgr2=0x5559464805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559464801e0 0x5559464805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 -- 172.21.15.42:0/1212475238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 msgr2=0x55594648a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x55594648a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115226640 1 -- 172.21.15.42:0/1212475238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559463aa0b0 con 0x555946480b90 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0114a25640 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x55594648a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115a27640 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559464801e0 0x5559464805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f0115a27640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f00f80a1ba0 0x7f00f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.979+0000 7f010e7fc640 1 -- 172.21.15.42:0/3514762301 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f01040cd490 con 0x7f0110105330 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.980+0000 7f0115226640 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x5559464896d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0100037040 tx=0x7f0100032500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef32c2681e32a16 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.980+0000 7f00f77fe640 1 -- 172.21.15.42:0/1212475238 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100032ed0 con 0x555946480b90 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.980+0000 7f0115a27640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f00f80a1ba0 0x7f00f80a4060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f01080046c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.980+0000 7f00f77fe640 1 -- 172.21.15.42:0/1212475238 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0100034a60 con 0x555946480b90 2024-03-12T04:21:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.980+0000 7f00f77fe640 1 -- 172.21.15.42:0/1212475238 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100003970 con 0x555946480b90 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.980+0000 7f011a056740 1 -- 172.21.15.42:0/1212475238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 msgr2=0x5559464896d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.980+0000 7f011a056740 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x5559464896d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0100037040 tx=0x7f0100032500 comp rx=0 tx=0).stop 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- 172.21.15.42:0/1212475238 shutdown_connections 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x55594648a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x5559464896d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 --2- 172.21.15.42:0/1212475238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559464801e0 0x5559464805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- 172.21.15.42:0/1212475238 >> 172.21.15.42:0/1212475238 conn(0x55594647a840 msgr2=0x55594647ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- 172.21.15.42:0/1212475238 shutdown_connections 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- 172.21.15.42:0/1212475238 wait complete. 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- start start 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 Processor -- start 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559464801e0 0x55594648d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x55594648dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 0x55594648e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594638cf40 con 0x555946480b90 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559460bb4b0 con 0x5559464801e0 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594636d670 con 0x555946489cf0 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.981+0000 7f011a056740 10 client.0 _ll_register_callbacks cb 0x5559463daf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x55594648dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x55594648dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60462/0 (socket says 172.21.15.42:60462) 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 -- 172.21.15.42:0/2907070171 learned_addr learned my addr 172.21.15.42:0/2907070171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0115226640 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559464801e0 0x55594648d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 -- 172.21.15.42:0/2907070171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 msgr2=0x55594648e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 0x55594648e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 -- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559464801e0 msgr2=0x55594648d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559464801e0 0x55594648d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 -- 172.21.15.42:0/2907070171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011010d710 con 0x555946480b90 2024-03-12T04:21:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0115226640 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559464801e0 0x55594648d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f0114a25640 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x55594648dc70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0104096b90 tx=0x7f01040a1420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da066f9c51022b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f00f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f00f4ff9640 1 -- 172.21.15.42:0/2907070171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01040cb050 con 0x555946480b90 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f00f4ff9640 1 -- 172.21.15.42:0/2907070171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f010409a1c0 con 0x555946480b90 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559463c1f40 con 0x555946480b90 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f00f4ff9640 1 -- 172.21.15.42:0/2907070171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f010409e120 con 0x555946480b90 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f011a056740 10 client.17199 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.982+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5559463e6320 con 0x555946480b90 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f00f4ff9640 1 -- 172.21.15.42:0/2907070171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5559463c1f40 con 0x555946480b90 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f00f4ff9640 1 -- 172.21.15.42:0/2907070171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0104099610 con 0x555946480b90 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f011a056740 10 client.17199 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f011a056740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f011a056740 20 client.17199 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f011a056740 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55594649e230 0x5559464a0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f011a056740 4 client.17199 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559463de310 con 0x55594649e230 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f0115226640 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55594649e230 0x5559464a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f00f4ff9640 10 client.17199 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.983+0000 7f0115226640 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55594649e230 0x5559464a0670 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18137a16ed5830b server_cookie=1df45eb8351c2b36 in_seq=0 out_seq=0 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f00f4ff9640 1 -- 172.21.15.42:0/2907070171 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559463de310 con 0x55594649e230 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f00f4ff9640 10 client.17199 handle_command_reply: tid=0 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 client.17199 shutdown 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 msgr2=0x55594648dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x55594648dc70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0104096b90 tx=0x7f01040a1420 comp rx=0 tx=0).stop 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 shutdown_connections 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55594649e230 0x5559464a0670 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 0x55594648e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555946480b90 0x55594648dc70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 --2- 172.21.15.42:0/2907070171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559464801e0 0x55594648d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f00f4ff9640 0 client.17199 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 >> 172.21.15.42:0/2907070171 conn(0x55594647a840 msgr2=0x55594647c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 shutdown_connections 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.984+0000 7f011a056740 1 -- 172.21.15.42:0/2907070171 wait complete. 2024-03-12T04:21:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:02.985+0000 7f011a056740 20 client.17199 trim_cache size 0 max 16384 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 Processor -- start 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 -- start start 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x55594648fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946480b90 0x55594648b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 0x55594648dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559464761b0 con 0x5559464801e0 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555946474d20 con 0x555946480b90 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f011a056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559464a58e0 con 0x555946489cf0 2024-03-12T04:21:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x55594648fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x55594648fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60474/0 (socket says 172.21.15.42:60474) 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 -- 172.21.15.42:0/241248715 learned_addr learned my addr 172.21.15.42:0/241248715 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 -- 172.21.15.42:0/241248715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 msgr2=0x55594648dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 0x55594648dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0114a25640 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946480b90 0x55594648b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 -- 172.21.15.42:0/241248715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946480b90 msgr2=0x55594648b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946480b90 0x55594648b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 -- 172.21.15.42:0/241248715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555946382c00 con 0x5559464801e0 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0114a25640 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946480b90 0x55594648b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.000+0000 7f0115226640 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x55594648fe60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f010002c130 tx=0x7f0100032500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7c6c0cb1434940 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f00f6ffd640 1 -- 172.21.15.42:0/241248715 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010004d040 con 0x5559464801e0 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f00f6ffd640 1 -- 172.21.15.42:0/241248715 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0100005cf0 con 0x5559464801e0 2024-03-12T04:21:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f00f6ffd640 1 -- 172.21.15.42:0/241248715 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100034a60 con 0x5559464801e0 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 -- 172.21.15.42:0/241248715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 msgr2=0x55594648fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x55594648fe60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f010002c130 tx=0x7f0100032500 comp rx=0 tx=0).stop 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 -- 172.21.15.42:0/241248715 shutdown_connections 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946489cf0 0x55594648dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946480b90 0x55594648b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 --2- 172.21.15.42:0/241248715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x55594648fe60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 -- 172.21.15.42:0/241248715 >> 172.21.15.42:0/241248715 conn(0x55594647a840 msgr2=0x5559464884d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 -- 172.21.15.42:0/241248715 shutdown_connections 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 -- 172.21.15.42:0/241248715 wait complete. 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.001+0000 7f011a056740 1 -- start start 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 1 Processor -- start 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x555946493800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946480b90 0x555946493d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x555946494280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559464f4760 con 0x5559464801e0 2024-03-12T04:21:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559464f48d0 con 0x555946489cf0 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559464f4a90 con 0x555946480b90 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f011a056740 10 client.0 _ll_register_callbacks cb 0x7f0100037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x555946493800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x555946493800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60490/0 (socket says 172.21.15.42:60490) 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 -- 172.21.15.42:0/1583716558 learned_addr learned my addr 172.21.15.42:0/1583716558 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0114a25640 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946480b90 0x555946493d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115a27640 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x555946494280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 -- 172.21.15.42:0/1583716558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946480b90 msgr2=0x555946493d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946480b90 0x555946493d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 -- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 msgr2=0x555946494280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x555946494280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.002+0000 7f0115226640 1 -- 172.21.15.42:0/1583716558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559463aa0b0 con 0x5559464801e0 2024-03-12T04:21:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f0115226640 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x555946493800 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f010002c630 tx=0x7f0100032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0b6af8c70b19eb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f00f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f00f4ff9640 1 -- 172.21.15.42:0/1583716558 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010004d050 con 0x5559464801e0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f00f4ff9640 1 -- 172.21.15.42:0/1583716558 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0100050d90 con 0x5559464801e0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55594640d900 con 0x5559464801e0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f00f4ff9640 1 -- 172.21.15.42:0/1583716558 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0100034440 con 0x5559464801e0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 10 client.17205 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5559463a8910 con 0x5559464801e0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f00f4ff9640 1 -- 172.21.15.42:0/1583716558 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0100067020 con 0x5559464801e0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f00f4ff9640 1 -- 172.21.15.42:0/1583716558 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f010003e410 con 0x5559464801e0 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 10 client.17205 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 20 client.17205 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559464f5da0 0x5559464f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 4 client.17205 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:21:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.003+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5559460e7860 con 0x5559464f5da0 2024-03-12T04:21:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.004+0000 7f0114a25640 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559464f5da0 0x5559464f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.004+0000 7f00f4ff9640 10 client.17205 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.004+0000 7f0114a25640 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559464f5da0 0x5559464f8190 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6de4cb55d8679db server_cookie=40a9d9ac1b7d38a5 in_seq=0 out_seq=0 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f00f4ff9640 1 -- 172.21.15.42:0/1583716558 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5559460e7860 con 0x5559464f5da0 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f00f4ff9640 10 client.17205 handle_command_reply: tid=0 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 client.17205 shutdown 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 msgr2=0x555946493800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x555946493800 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f010002c630 tx=0x7f0100032900 comp rx=0 tx=0).stop 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 shutdown_connections 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5559464f5da0 0x5559464f8190 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555946489cf0 0x555946494280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555946480b90 0x555946493d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 --2- 172.21.15.42:0/1583716558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559464801e0 0x555946493800 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f00f4ff9640 0 client.17205 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 >> 172.21.15.42:0/1583716558 conn(0x55594647a840 msgr2=0x55594648bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 shutdown_connections 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 1 -- 172.21.15.42:0/1583716558 wait complete. 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.005+0000 7f011a056740 20 client.17205 trim_cache size 0 max 16384 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "aaede730-3c23-44ce-93ce-41040b247b8a", 2024-03-12T04:21:03.014 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:21:03.015 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:21:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f00f80a1ba0 msgr2=0x7f00f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f00f80a1ba0 0x7f00f80a4060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f01080046c0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110105330 msgr2=0x7f0110198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110105330 0x7f0110198b20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f010400b450 tx=0x7f010400b910 comp rx=0 tx=0).stop 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 shutdown_connections 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f00f80a1ba0 0x7f00f80a4060 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01101990b0 0x7f01101a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0110105330 0x7f0110198b20 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 --2- 172.21.15.42:0/3514762301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0110100260 0x7f01101985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 >> 172.21.15.42:0/3514762301 conn(0x7f0110077180 msgr2=0x7f01100fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 shutdown_connections 2024-03-12T04:21:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.006+0000 7f011748a640 1 -- 172.21.15.42:0/3514762301 wait complete. 2024-03-12T04:21:03.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:21:03.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/1644835159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a48101d10 msgr2=0x7f1a48105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/1644835159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a48101d10 0x7f1a48105d50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009930 tx=0x7f1a3802bc30 comp rx=0 tx=0).stop 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/1644835159 shutdown_connections 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/1644835159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a48106430 0x7f1a4810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/1644835159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a48101d10 0x7f1a48105d50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/1644835159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a48101360 0x7f1a48101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/1644835159 >> 172.21.15.42:0/1644835159 conn(0x7f1a480fcc20 msgr2=0x7f1a480ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/1644835159 shutdown_connections 2024-03-12T04:21:03.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.122+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/1644835159 wait complete. 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 Processor -- start 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 -- start start 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a48101360 0x7f1a4819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a48106430 0x7f1a4819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a481975e0 0x7f1a48197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48197fd0 con 0x7f1a481975e0 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48198140 con 0x7f1a48106430 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4fb5e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48198300 con 0x7f1a48101360 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a481975e0 0x7f1a48197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a481975e0 0x7f1a48197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60496/0 (socket says 172.21.15.42:60496) 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/688917276 learned_addr learned my addr 172.21.15.42:0/688917276 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a48101360 0x7f1a4819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a48106430 0x7f1a4819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/688917276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a48101360 msgr2=0x7f1a4819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a48101360 0x7f1a4819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/688917276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a48106430 msgr2=0x7f1a4819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a48106430 0x7f1a4819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/688917276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a38009560 con 0x7f1a481975e0 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.123+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a481975e0 0x7f1a48197a90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000cce0 tx=0x7f1a4000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.124+0000 7f1a3effd640 1 -- 172.21.15.42:0/688917276 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000f450 con 0x7f1a481975e0 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.124+0000 7f1a3effd640 1 -- 172.21.15.42:0/688917276 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a4000fa60 con 0x7f1a481975e0 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.124+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a481a42a0 con 0x7f1a481975e0 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.124+0000 7f1a3effd640 1 -- 172.21.15.42:0/688917276 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40018b10 con 0x7f1a481975e0 2024-03-12T04:21:03.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.124+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a481a4730 con 0x7f1a481975e0 2024-03-12T04:21:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.126+0000 7f1a3effd640 1 -- 172.21.15.42:0/688917276 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f1a4000fbd0 con 0x7f1a481975e0 2024-03-12T04:21:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.126+0000 7f1a5280d740 1 Processor -- start 2024-03-12T04:21:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.126+0000 7f1a5280d740 1 -- start start 2024-03-12T04:21:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.126+0000 7f1a3effd640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1a340a1ba0 0x7f1a340a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060f2b0 0x55aad060f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060fbd0 0x55aad0618270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad061aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a5280d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad057e280 con 0x55aad060f2b0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a5280d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad057b7b0 con 0x55aad06187b0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a5280d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad0576f90 con 0x55aad060fbd0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1a340a1ba0 0x7f1a340a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060f2b0 0x55aad060f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a3effd640 1 -- 172.21.15.42:0/688917276 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1a400cdc50 con 0x7f1a481975e0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060f2b0 0x55aad060f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60504/0 (socket says 172.21.15.42:60504) 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 -- 172.21.15.42:0/1154638917 learned_addr learned my addr 172.21.15.42:0/1154638917 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad061aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 -- 172.21.15.42:0/1154638917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060fbd0 msgr2=0x55aad0618270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060fbd0 0x55aad0618270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 -- 172.21.15.42:0/1154638917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 msgr2=0x55aad061aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad061aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 -- 172.21.15.42:0/1154638917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aad0535570 con 0x55aad060f2b0 2024-03-12T04:21:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad061aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1a340a1ba0 0x7f1a340a4060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009900 tx=0x7f1a38005be0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060f2b0 0x55aad060f690 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400cbe0 tx=0x7f1a4400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708e6568670f8568 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a33fff640 1 -- 172.21.15.42:0/1154638917 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44010630 con 0x55aad060f2b0 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a33fff640 1 -- 172.21.15.42:0/1154638917 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a44010c40 con 0x55aad060f2b0 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.127+0000 7f1a33fff640 1 -- 172.21.15.42:0/1154638917 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4401aa80 con 0x55aad060f2b0 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 -- 172.21.15.42:0/1154638917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060f2b0 msgr2=0x55aad060f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060f2b0 0x55aad060f690 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400cbe0 tx=0x7f1a4400e720 comp rx=0 tx=0).stop 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 -- 172.21.15.42:0/1154638917 shutdown_connections 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad061aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060fbd0 0x55aad0618270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1154638917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060f2b0 0x55aad060f690 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 -- 172.21.15.42:0/1154638917 >> 172.21.15.42:0/1154638917 conn(0x55aad0609990 msgr2=0x55aad0609d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 -- 172.21.15.42:0/1154638917 shutdown_connections 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.128+0000 7f1a5280d740 1 -- 172.21.15.42:0/1154638917 wait complete. 2024-03-12T04:21:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 -- start start 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 Processor -- start 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad06181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad0615530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad0615b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad024a430 con 0x55aad060fbd0 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad0206e00 con 0x55aad06187b0 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad04fbfd0 con 0x55aad060f2b0 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad0615530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a5280d740 10 client.0 _ll_register_callbacks cb 0x55aad0551fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad0615530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60518/0 (socket says 172.21.15.42:60518) 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/735848181 learned_addr learned my addr 172.21.15.42:0/735848181 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad06181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/735848181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 msgr2=0x55aad06181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad06181a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad0615b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 msgr2=0x55aad0615b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad0615b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.129+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/735848181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4400c810 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad0615530 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1a3802c710 tx=0x7f1a38009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fbda459182659b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a317fa640 1 -- 172.21.15.42:0/735848181 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38003b40 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a317fa640 1 -- 172.21.15.42:0/735848181 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1a38003ca0 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aad02767e0 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a317fa640 1 -- 172.21.15.42:0/735848181 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a38003e60 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 10 client.17217 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aad0575610 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a317fa640 1 -- 172.21.15.42:0/735848181 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55aad02767e0 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a317fa640 1 -- 172.21.15.42:0/735848181 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1a38040570 con 0x55aad060fbd0 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 10 client.17217 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 20 client.17217 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0628930 0x55aad062ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 4 client.17217 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.130+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aad02b3720 con 0x55aad0628930 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.131+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0628930 0x55aad062ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.131+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0628930 0x55aad062ad50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4415b17ac81a14f server_cookie=59d45ba1a249d2d in_seq=0 out_seq=0 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.131+0000 7f1a317fa640 10 client.17217 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a317fa640 1 -- 172.21.15.42:0/735848181 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aad02b3720 con 0x55aad0628930 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a317fa640 10 client.17217 handle_command_reply: tid=0 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 client.17217 shutdown 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 msgr2=0x55aad0615530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad0615530 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1a3802c710 tx=0x7f1a38009260 comp rx=0 tx=0).stop 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 shutdown_connections 2024-03-12T04:21:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0628930 0x55aad062ad50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad06187b0 0x55aad0615b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad0615530 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a317fa640 0 client.17217 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 --2- 172.21.15.42:0/735848181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad06181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 >> 172.21.15.42:0/735848181 conn(0x55aad0609990 msgr2=0x55aad060b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 shutdown_connections 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 1 -- 172.21.15.42:0/735848181 wait complete. 2024-03-12T04:21:03.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.132+0000 7f1a5280d740 20 client.17217 trim_cache size 0 max 16384 2024-03-12T04:21:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 Processor -- start 2024-03-12T04:21:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 -- start start 2024-03-12T04:21:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad0617c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060fbd0 0x55aad06181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad06187b0 0x55aad0614dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad06050f0 con 0x55aad06187b0 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad060e450 con 0x55aad060fbd0 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a5280d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad06052b0 con 0x55aad060f2b0 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a4e0fb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad06187b0 0x55aad0614dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a4e0fb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad06187b0 0x55aad0614dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60528/0 (socket says 172.21.15.42:60528) 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.150+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/1244414350 learned_addr learned my addr 172.21.15.42:0/1244414350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060fbd0 0x55aad06181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad0617c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/1244414350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 msgr2=0x55aad0617c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad0617c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/1244414350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060fbd0 msgr2=0x55aad06181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060fbd0 0x55aad06181a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4e0fb640 1 -- 172.21.15.42:0/1244414350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aad0511ac0 con 0x55aad06187b0 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad0617c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a4e0fb640 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad06187b0 0x55aad0614dc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1a400cb450 tx=0x7f1a40007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19ed2dd1dd2ce44 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a337fe640 1 -- 172.21.15.42:0/1244414350 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40079040 con 0x55aad06187b0 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a337fe640 1 -- 172.21.15.42:0/1244414350 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a4000d750 con 0x55aad06187b0 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.151+0000 7f1a337fe640 1 -- 172.21.15.42:0/1244414350 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a400a3070 con 0x55aad06187b0 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 -- 172.21.15.42:0/1244414350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad06187b0 msgr2=0x55aad0614dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad06187b0 0x55aad0614dc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1a400cb450 tx=0x7f1a40007870 comp rx=0 tx=0).stop 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 -- 172.21.15.42:0/1244414350 shutdown_connections 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad06187b0 0x55aad0614dc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060fbd0 0x55aad06181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1244414350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad060f2b0 0x55aad0617c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 -- 172.21.15.42:0/1244414350 >> 172.21.15.42:0/1244414350 conn(0x55aad0609990 msgr2=0x55aad0613e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 -- 172.21.15.42:0/1244414350 shutdown_connections 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 -- 172.21.15.42:0/1244414350 wait complete. 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 -- start start 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.152+0000 7f1a5280d740 1 Processor -- start 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060f2b0 0x55aad061c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad061c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad06187b0 0x55aad067ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad061d050 con 0x55aad060fbd0 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad061d1c0 con 0x55aad060f2b0 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aad067f370 con 0x55aad06187b0 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 10 client.0 _ll_register_callbacks cb 0x7f1a4400c370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad061c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad061c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60540/0 (socket says 172.21.15.42:60540) 2024-03-12T04:21:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/1211640999 learned_addr learned my addr 172.21.15.42:0/1211640999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060f2b0 0x55aad061c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/1211640999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad06187b0 msgr2=0x55aad067ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad06187b0 0x55aad067ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060f2b0 msgr2=0x55aad061c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060f2b0 0x55aad061c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 -- 172.21.15.42:0/1211640999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a481a4730 con 0x55aad060fbd0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a4d0f9640 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad061c8b0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1a38000c00 tx=0x7f1a38004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489ab7f3a8dc17f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a317fa640 1 -- 172.21.15.42:0/1211640999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38040aa0 con 0x55aad060fbd0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a44010c40 con 0x55aad060fbd0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.153+0000 7f1a317fa640 1 -- 172.21.15.42:0/1211640999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1a38040c00 con 0x55aad060fbd0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a317fa640 1 -- 172.21.15.42:0/1211640999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a38040de0 con 0x55aad060fbd0 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 10 client.17223 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aad05376a0 con 0x55aad060fbd0 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a317fa640 1 -- 172.21.15.42:0/1211640999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a3806f020 con 0x55aad060fbd0 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a317fa640 1 -- 172.21.15.42:0/1211640999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1a38005370 con 0x55aad060fbd0 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 10 client.17223 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 20 client.17223 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0680820 0x55aad0682c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 4 client.17223 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aad0302890 con 0x55aad0680820 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.154+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0680820 0x55aad0682c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a4d8fa640 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0680820 0x55aad0682c40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d467c03f7d617b38 server_cookie=b9b8ebcb50201705 in_seq=0 out_seq=0 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a317fa640 10 client.17223 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a317fa640 1 -- 172.21.15.42:0/1211640999 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aad0302890 con 0x55aad0680820 2024-03-12T04:21:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a317fa640 10 client.17223 handle_command_reply: tid=0 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 client.17223 shutdown 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 msgr2=0x55aad061c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad061c8b0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1a38000c00 tx=0x7f1a38004980 comp rx=0 tx=0).stop 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 shutdown_connections 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55aad0680820 0x55aad0682c40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aad06187b0 0x55aad067ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aad060fbd0 0x55aad061c8b0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a317fa640 0 client.17223 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 --2- 172.21.15.42:0/1211640999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aad060f2b0 0x55aad061c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 >> 172.21.15.42:0/1211640999 conn(0x55aad0609990 msgr2=0x55aad0614dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 shutdown_connections 2024-03-12T04:21:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 1 -- 172.21.15.42:0/1211640999 wait complete. 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.155+0000 7f1a5280d740 20 client.17223 trim_cache size 0 max 16384 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "aaede730-3c23-44ce-93ce-41040b247b8a": { 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "aaede730-3c23-44ce-93ce-41040b247b8a", 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:21:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:21:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:21:03.167 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:21:03.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1a340a1ba0 msgr2=0x7f1a340a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1a340a1ba0 0x7f1a340a4060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009900 tx=0x7f1a38005be0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a481975e0 msgr2=0x7f1a48197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a481975e0 0x7f1a48197a90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000cce0 tx=0x7f1a4000a620 comp rx=0 tx=0).stop 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 shutdown_connections 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1a340a1ba0 0x7f1a340a4060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a481975e0 0x7f1a48197a90 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a48106430 0x7f1a4819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 --2- 172.21.15.42:0/688917276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a48101360 0x7f1a4819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 >> 172.21.15.42:0/688917276 conn(0x7f1a480fcc20 msgr2=0x7f1a480fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 shutdown_connections 2024-03-12T04:21:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:03.156+0000 7f1a4fb5e640 1 -- 172.21.15.42:0/688917276 wait complete. 2024-03-12T04:21:03.169 INFO:tasks.cephfs.filesystem:scrub status for tag:aaede730-3c23-44ce-93ce-41040b247b8a - {'path': '/', 'tag': 'aaede730-3c23-44ce-93ce-41040b247b8a', 'options': 'recursive,force'} 2024-03-12T04:21:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:03 smithi119 ceph-mon[36119]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:03 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:21:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:03 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:03 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:21:03.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:03 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:03 smithi136 ceph-mon[35571]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:03 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:21:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:03 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:03 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:21:03.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:03 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:03.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:03 smithi042 ceph-mon[32260]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:03.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:03 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:21:03.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:03 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:03.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:03 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:21:03.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:03 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:04.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:04 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:21:04.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:04 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:21:04.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:04 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:21:05.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:05 smithi119 ceph-mon[36119]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:05.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:05 smithi136 ceph-mon[35571]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:05.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:05 smithi042 ceph-mon[32260]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:07.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:07 smithi119 ceph-mon[36119]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:07 smithi136 ceph-mon[35571]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:07 smithi042 ceph-mon[32260]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:08.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:08 smithi042 ceph-mon[32260]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:08.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:08 smithi119 ceph-mon[36119]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:08 smithi136 ceph-mon[35571]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:10 smithi042 ceph-mon[32260]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:10.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:10 smithi119 ceph-mon[36119]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:10.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:10 smithi136 ceph-mon[35571]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:12.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:12 smithi042 ceph-mon[32260]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:12.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:12 smithi119 ceph-mon[36119]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:12 smithi136 ceph-mon[35571]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T04:21:14.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:14 smithi042 ceph-mon[32260]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:14 smithi119 ceph-mon[36119]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:14 smithi136 ceph-mon[35571]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-12T04:21:16.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:16 smithi119 ceph-mon[36119]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:21:16.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:16 smithi136 ceph-mon[35571]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:21:16.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:16 smithi042 ceph-mon[32260]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-12T04:21:18.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:18 smithi119 ceph-mon[36119]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:21:18.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:18 smithi136 ceph-mon[35571]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:21:18.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:18 smithi042 ceph-mon[32260]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-12T04:21:20.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:20 smithi119 ceph-mon[36119]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:21:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:20 smithi136 ceph-mon[35571]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:21:20.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:20 smithi042 ceph-mon[32260]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:21:22.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:22 smithi119 ceph-mon[36119]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:21:22.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:22 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:21:22.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:22 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:21:22.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:22 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:21:22.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:22 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:21:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:22 smithi136 ceph-mon[35571]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:21:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:22 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:21:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:22 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:21:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:22 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:21:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:22 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:21:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:22 smithi042 ceph-mon[32260]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-12T04:21:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:22 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:21:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:22 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:21:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:22 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:21:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:22 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:21:24.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:24 smithi119 ceph-mon[36119]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:21:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:24 smithi136 ceph-mon[35571]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:21:24.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:24 smithi042 ceph-mon[32260]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:21:26.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:26 smithi119 ceph-mon[36119]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:26 smithi136 ceph-mon[35571]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:26.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:26 smithi042 ceph-mon[32260]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:28.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:28 smithi119 ceph-mon[36119]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:28 smithi136 ceph-mon[35571]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:28.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:28 smithi042 ceph-mon[32260]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:30.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:30 smithi119 ceph-mon[36119]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:30.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:30 smithi136 ceph-mon[35571]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:30.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:30 smithi042 ceph-mon[32260]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:32.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:32 smithi119 ceph-mon[36119]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:32.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:32 smithi136 ceph-mon[35571]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:32.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:32 smithi042 ceph-mon[32260]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:33.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:21:33.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:21:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 -- 172.21.15.42:0/1237235441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c1002d0 msgr2=0x7f8e3c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 --2- 172.21.15.42:0/1237235441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c1002d0 0x7f8e3c100730 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009900 tx=0x7f8e2c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:21:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 -- 172.21.15.42:0/1237235441 shutdown_connections 2024-03-12T04:21:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 --2- 172.21.15.42:0/1237235441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c100c70 0x7f8e3c10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 --2- 172.21.15.42:0/1237235441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c1002d0 0x7f8e3c100730 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 --2- 172.21.15.42:0/1237235441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e3c1060b0 0x7f8e3c106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 -- 172.21.15.42:0/1237235441 >> 172.21.15.42:0/1237235441 conn(0x7f8e3c076e20 msgr2=0x7f8e3c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 -- 172.21.15.42:0/1237235441 shutdown_connections 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.287+0000 7f8e43c49640 1 -- 172.21.15.42:0/1237235441 wait complete. 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 Processor -- start 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 -- start start 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c1002d0 0x7f8e3c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c100c70 0x7f8e3c102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e3c1060b0 0x7f8e3c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1a5ae0 con 0x7f8e3c100c70 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1a5c50 con 0x7f8e3c1002d0 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e43c49640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1a5e10 con 0x7f8e3c1060b0 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e411e4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c100c70 0x7f8e3c102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e411e4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c100c70 0x7f8e3c102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35558/0 (socket says 172.21.15.42:35558) 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e411e4640 1 -- 172.21.15.42:0/4256431651 learned_addr learned my addr 172.21.15.42:0/4256431651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e419e5640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c1002d0 0x7f8e3c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.288+0000 7f8e421e6640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e3c1060b0 0x7f8e3c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e411e4640 1 -- 172.21.15.42:0/4256431651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e3c1060b0 msgr2=0x7f8e3c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e411e4640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e3c1060b0 0x7f8e3c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e411e4640 1 -- 172.21.15.42:0/4256431651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c1002d0 msgr2=0x7f8e3c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e411e4640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c1002d0 0x7f8e3c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e411e4640 1 -- 172.21.15.42:0/4256431651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c009560 con 0x7f8e3c100c70 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e421e6640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e3c1060b0 0x7f8e3c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e411e4640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c100c70 0x7f8e3c102230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c006010 tx=0x7f8e2c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e32ffd640 1 -- 172.21.15.42:0/4256431651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0044c0 con 0x7f8e3c100c70 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e32ffd640 1 -- 172.21.15.42:0/4256431651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e2c02de90 con 0x7f8e3c100c70 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c1a6090 con 0x7f8e3c100c70 2024-03-12T04:21:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e32ffd640 1 -- 172.21.15.42:0/4256431651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0053c0 con 0x7f8e3c100c70 2024-03-12T04:21:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.289+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c1a65f0 con 0x7f8e3c100c70 2024-03-12T04:21:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.291+0000 7f8e32ffd640 1 -- 172.21.15.42:0/4256431651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8e2c005520 con 0x7f8e3c100c70 2024-03-12T04:21:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 Processor -- start 2024-03-12T04:21:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 -- start start 2024-03-12T04:21:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d024690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286cf93280 con 0x56286d0242b0 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286cf907b0 con 0x56286d02d7b0 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e46816740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286cf8bf90 con 0x56286d024bd0 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e32ffd640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8e280a1c20 0x7f8e280a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e421e6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d024690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e421e6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d024690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35574/0 (socket says 172.21.15.42:35574) 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e421e6640 1 -- 172.21.15.42:0/1910638683 learned_addr learned my addr 172.21.15.42:0/1910638683 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e32ffd640 1 -- 172.21.15.42:0/4256431651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8e2c0ef800 con 0x7f8e3c100c70 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e419e5640 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e421e6640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8e280a1c20 0x7f8e280a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.292+0000 7f8e411e4640 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e419e5640 1 -- 172.21.15.42:0/1910638683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 msgr2=0x56286d02fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e419e5640 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e419e5640 1 -- 172.21.15.42:0/1910638683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 msgr2=0x56286d024690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e419e5640 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d024690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e419e5640 1 -- 172.21.15.42:0/1910638683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286cf4a570 con 0x56286d024bd0 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e411e4640 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e421e6640 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d024690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e419e5640 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02d270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e38004630 tx=0x7f8e3800bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b99f29121a1a50f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e421e6640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8e280a1c20 0x7f8e280a40e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400aed0 tx=0x7f8e34009ad0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e27fff640 1 -- 172.21.15.42:0/1910638683 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003bd0 con 0x56286d024bd0 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e27fff640 1 -- 172.21.15.42:0/1910638683 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e38003d30 con 0x56286d024bd0 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.293+0000 7f8e27fff640 1 -- 172.21.15.42:0/1910638683 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004d10 con 0x56286d024bd0 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 -- 172.21.15.42:0/1910638683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 msgr2=0x56286d02d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02d270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e38004630 tx=0x7f8e3800bb20 comp rx=0 tx=0).stop 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 -- 172.21.15.42:0/1910638683 shutdown_connections 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02d270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 --2- 172.21.15.42:0/1910638683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d024690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 -- 172.21.15.42:0/1910638683 >> 172.21.15.42:0/1910638683 conn(0x56286d01e990 msgr2=0x56286d01ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 -- 172.21.15.42:0/1910638683 shutdown_connections 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 -- 172.21.15.42:0/1910638683 wait complete. 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.294+0000 7f8e46816740 1 -- start start 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 1 Processor -- start 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d02d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286cc5f430 con 0x56286d0242b0 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286cc1be00 con 0x56286d02d7b0 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286cf10fd0 con 0x56286d024bd0 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e46816740 10 client.0 _ll_register_callbacks cb 0x56286cf66fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d02d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d02d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35582/0 (socket says 172.21.15.42:35582) 2024-03-12T04:21:33.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 -- 172.21.15.42:0/237201885 learned_addr learned my addr 172.21.15.42:0/237201885 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e411e4640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e421e6640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 -- 172.21.15.42:0/237201885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 msgr2=0x56286d02a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 -- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 msgr2=0x56286d02ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e419e5640 1 -- 172.21.15.42:0/237201885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e38004290 con 0x56286d0242b0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e421e6640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.295+0000 7f8e411e4640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e419e5640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d02d1d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8e380049b0 tx=0x7f8e3800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd66a544e65c239 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e257fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e257fa640 1 -- 172.21.15.42:0/237201885 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38011850 con 0x56286d0242b0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e257fa640 1 -- 172.21.15.42:0/237201885 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8e380119b0 con 0x56286d0242b0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56286cc8b7e0 con 0x56286d0242b0 2024-03-12T04:21:33.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e257fa640 1 -- 172.21.15.42:0/237201885 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e38011b70 con 0x56286d0242b0 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 10 client.17235 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56286cf8a610 con 0x56286d0242b0 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e257fa640 1 -- 172.21.15.42:0/237201885 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e38031020 con 0x56286d0242b0 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e257fa640 1 -- 172.21.15.42:0/237201885 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8e38019420 con 0x56286d0242b0 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 10 client.17235 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 20 client.17235 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d03d800 0x56286d03fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 4 client.17235 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.296+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56286ccc8720 con 0x56286d03d800 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.297+0000 7f8e411e4640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d03d800 0x56286d03fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.297+0000 7f8e411e4640 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d03d800 0x56286d03fc10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1262c7fbdda5b4 server_cookie=d9d8fee6032cd06e in_seq=0 out_seq=0 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.297+0000 7f8e257fa640 10 client.17235 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:33.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e257fa640 1 -- 172.21.15.42:0/237201885 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56286ccc8720 con 0x56286d03d800 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e257fa640 10 client.17235 handle_command_reply: tid=0 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 client.17235 shutdown 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 msgr2=0x56286d02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d02d1d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8e380049b0 tx=0x7f8e3800bfa0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 shutdown_connections 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d03d800 0x56286d03fc10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d02ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e257fa640 0 client.17235 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 --2- 172.21.15.42:0/237201885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d02d1d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 >> 172.21.15.42:0/237201885 conn(0x56286d01e990 msgr2=0x56286d020730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 shutdown_connections 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 1 -- 172.21.15.42:0/237201885 wait complete. 2024-03-12T04:21:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.298+0000 7f8e46816740 20 client.17235 trim_cache size 0 max 16384 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.313+0000 7f8e46816740 1 Processor -- start 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.313+0000 7f8e46816740 1 -- start start 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.313+0000 7f8e46816740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d03fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.313+0000 7f8e46816740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.313+0000 7f8e46816740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d0258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e46816740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d044e60 con 0x56286d0242b0 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e46816740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d0249c0 con 0x56286d02d7b0 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e46816740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d020c20 con 0x56286d024bd0 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d03fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d03fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35598/0 (socket says 172.21.15.42:35598) 2024-03-12T04:21:33.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 -- 172.21.15.42:0/2926826901 learned_addr learned my addr 172.21.15.42:0/2926826901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e421e6640 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d0258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 -- 172.21.15.42:0/2926826901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 msgr2=0x56286d02cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e411e4640 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 -- 172.21.15.42:0/2926826901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 msgr2=0x56286d0258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d0258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 -- 172.21.15.42:0/2926826901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286cf26ac0 con 0x56286d0242b0 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e421e6640 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d0258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e419e5640 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d03fbf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005880 tx=0x7f8e38019920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50d02173dd80c52 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e277fe640 1 -- 172.21.15.42:0/2926826901 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38005c50 con 0x56286d0242b0 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e277fe640 1 -- 172.21.15.42:0/2926826901 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e38031030 con 0x56286d0242b0 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.314+0000 7f8e277fe640 1 -- 172.21.15.42:0/2926826901 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38011d60 con 0x56286d0242b0 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 -- 172.21.15.42:0/2926826901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 msgr2=0x56286d03fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d03fbf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005880 tx=0x7f8e38019920 comp rx=0 tx=0).stop 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 -- 172.21.15.42:0/2926826901 shutdown_connections 2024-03-12T04:21:33.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d02d7b0 0x56286d0258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d024bd0 0x56286d02cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 --2- 172.21.15.42:0/2926826901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d0242b0 0x56286d03fbf0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 -- 172.21.15.42:0/2926826901 >> 172.21.15.42:0/2926826901 conn(0x56286d01e990 msgr2=0x56286d01ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 -- 172.21.15.42:0/2926826901 shutdown_connections 2024-03-12T04:21:33.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 -- 172.21.15.42:0/2926826901 wait complete. 2024-03-12T04:21:33.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 -- start start 2024-03-12T04:21:33.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.315+0000 7f8e46816740 1 Processor -- start 2024-03-12T04:21:33.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e46816740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d0242b0 0x56286d033290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e46816740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d024bd0 0x56286d0337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e46816740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d02d7b0 0x56286d033d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e46816740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d093eb0 con 0x56286d02d7b0 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e46816740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d094020 con 0x56286d024bd0 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e46816740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d0941e0 con 0x56286d0242b0 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e46816740 10 client.0 _ll_register_callbacks cb 0x7f8e38004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d02d7b0 0x56286d033d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d02d7b0 0x56286d033d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35614/0 (socket says 172.21.15.42:35614) 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 -- 172.21.15.42:0/2587966448 learned_addr learned my addr 172.21.15.42:0/2587966448 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e411e4640 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d024bd0 0x56286d0337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e419e5640 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d0242b0 0x56286d033290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 -- 172.21.15.42:0/2587966448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d0242b0 msgr2=0x56286d033290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d0242b0 0x56286d033290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 -- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d024bd0 msgr2=0x56286d0337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d024bd0 0x56286d0337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 -- 172.21.15.42:0/2587966448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56286cf8a610 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e421e6640 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d02d7b0 0x56286d033d10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400ac70 tx=0x7f8e34019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbe8cd17b253f67 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e257fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.316+0000 7f8e257fa640 1 -- 172.21.15.42:0/2587966448 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34028070 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e257fa640 1 -- 172.21.15.42:0/2587966448 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8e34019e00 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e38003d30 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e257fa640 1 -- 172.21.15.42:0/2587966448 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e3401e050 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 10 client.17241 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56286cf4c6a0 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e257fa640 1 -- 172.21.15.42:0/2587966448 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e38003d30 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e257fa640 1 -- 172.21.15.42:0/2587966448 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8e3402b020 con 0x56286d02d7b0 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 10 client.17241 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 20 client.17241 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d095a10 0x56286d097e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 4 client.17241 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56286cd17890 con 0x56286d095a10 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.317+0000 7f8e419e5640 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d095a10 0x56286d097e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e257fa640 10 client.17241 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e419e5640 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d095a10 0x56286d097e30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ccea03a6ff3071 server_cookie=b2ce52c844034c0f in_seq=0 out_seq=0 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e257fa640 1 -- 172.21.15.42:0/2587966448 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56286cd17890 con 0x56286d095a10 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e257fa640 10 client.17241 handle_command_reply: tid=0 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 client.17241 shutdown 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d02d7b0 msgr2=0x56286d033d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d02d7b0 0x56286d033d10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400ac70 tx=0x7f8e34019000 comp rx=0 tx=0).stop 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 shutdown_connections 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56286d095a10 0x56286d097e30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56286d02d7b0 0x56286d033d10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286d024bd0 0x56286d0337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e257fa640 0 client.17241 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 --2- 172.21.15.42:0/2587966448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56286d0242b0 0x56286d033290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 >> 172.21.15.42:0/2587966448 conn(0x56286d01e990 msgr2=0x56286d0258d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 shutdown_connections 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 1 -- 172.21.15.42:0/2587966448 wait complete. 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.318+0000 7f8e46816740 20 client.17241 trim_cache size 0 max 16384 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:21:33.327 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:21:33.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8e280a1c20 msgr2=0x7f8e280a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8e280a1c20 0x7f8e280a40e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400aed0 tx=0x7f8e34009ad0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c100c70 msgr2=0x7f8e3c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c100c70 0x7f8e3c102230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c006010 tx=0x7f8e2c0040c0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 shutdown_connections 2024-03-12T04:21:33.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8e280a1c20 0x7f8e280a40e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e3c1060b0 0x7f8e3c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e3c100c70 0x7f8e3c102230 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 --2- 172.21.15.42:0/4256431651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c1002d0 0x7f8e3c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 >> 172.21.15.42:0/4256431651 conn(0x7f8e3c076e20 msgr2=0x7f8e3c0699b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 shutdown_connections 2024-03-12T04:21:33.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.319+0000 7f8e43c49640 1 -- 172.21.15.42:0/4256431651 wait complete. 2024-03-12T04:21:33.332 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:21:33.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:21:33.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 -- 172.21.15.42:0/346217469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34106450 msgr2=0x7fbb3410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 --2- 172.21.15.42:0/346217469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34106450 0x7fbb3410e020 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009cd0 tx=0x7fbb2c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 -- 172.21.15.42:0/346217469 shutdown_connections 2024-03-12T04:21:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 --2- 172.21.15.42:0/346217469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34106450 0x7fbb3410e020 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 --2- 172.21.15.42:0/346217469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb34101d30 0x7fbb34105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 --2- 172.21.15.42:0/346217469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb34101380 0x7fbb34101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 -- 172.21.15.42:0/346217469 >> 172.21.15.42:0/346217469 conn(0x7fbb340fcc20 msgr2=0x7fbb340ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.432+0000 7fbb39148640 1 -- 172.21.15.42:0/346217469 shutdown_connections 2024-03-12T04:21:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.433+0000 7fbb39148640 1 -- 172.21.15.42:0/346217469 wait complete. 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.433+0000 7fbb39148640 1 Processor -- start 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.433+0000 7fbb39148640 1 -- start start 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34101380 0x7fbb3419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb34106450 0x7fbb3419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb34197650 0x7fbb34197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34198100 con 0x7fbb34101380 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34198270 con 0x7fbb34197650 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34198430 con 0x7fbb34106450 2024-03-12T04:21:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34101380 0x7fbb3419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34101380 0x7fbb3419d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35630/0 (socket says 172.21.15.42:35630) 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 -- 172.21.15.42:0/982878799 learned_addr learned my addr 172.21.15.42:0/982878799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb3259c640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb34106450 0x7fbb3419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 -- 172.21.15.42:0/982878799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb34106450 msgr2=0x7fbb3419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb34106450 0x7fbb3419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 -- 172.21.15.42:0/982878799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb34197650 msgr2=0x7fbb34197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb34197650 0x7fbb34197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 -- 172.21.15.42:0/982878799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009900 con 0x7fbb34101380 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb32d9d640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34101380 0x7fbb3419d500 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c970 tx=0x7fbb2400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb1bfff640 1 -- 172.21.15.42:0/982878799 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24007bb0 con 0x7fbb34101380 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb1bfff640 1 -- 172.21.15.42:0/982878799 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb24007d10 con 0x7fbb34101380 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb341a4370 con 0x7fbb34101380 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb1bfff640 1 -- 172.21.15.42:0/982878799 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24004e80 con 0x7fbb34101380 2024-03-12T04:21:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.434+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb341a4830 con 0x7fbb34101380 2024-03-12T04:21:33.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.435+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb340683c0 con 0x7fbb34101380 2024-03-12T04:21:33.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.436+0000 7fbb1bfff640 1 -- 172.21.15.42:0/982878799 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fbb24005020 con 0x7fbb34101380 2024-03-12T04:21:33.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.437+0000 7fbb1bfff640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb1c0a1ba0 0x7fbb1c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.437+0000 7fbb3259c640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb1c0a1ba0 0x7fbb1c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.437+0000 7fbb1bfff640 1 -- 172.21.15.42:0/982878799 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbb240cd560 con 0x7fbb34101380 2024-03-12T04:21:33.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.437+0000 7fbb3259c640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb1c0a1ba0 0x7fbb1c0a4060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb34100510 tx=0x7fbb2800b390 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.441+0000 7fbb1bfff640 1 -- 172.21.15.42:0/982878799 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb24096480 con 0x7fbb34101380 2024-03-12T04:21:33.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.662+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb34198b60 con 0x7fbb34101380 2024-03-12T04:21:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.663+0000 7fbb1bfff640 1 -- 172.21.15.42:0/982878799 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fbb24099b40 con 0x7fbb34101380 2024-03-12T04:21:33.665 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:21:33.666 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:21:33.666 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb1c0a1ba0 msgr2=0x7fbb1c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb1c0a1ba0 0x7fbb1c0a4060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb34100510 tx=0x7fbb2800b390 comp rx=0 tx=0).stop 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34101380 msgr2=0x7fbb3419d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34101380 0x7fbb3419d500 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c970 tx=0x7fbb2400ce30 comp rx=0 tx=0).stop 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 shutdown_connections 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb1c0a1ba0 0x7fbb1c0a4060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb34197650 0x7fbb34197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb34106450 0x7fbb3419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 --2- 172.21.15.42:0/982878799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb34101380 0x7fbb3419d500 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 >> 172.21.15.42:0/982878799 conn(0x7fbb340fcc20 msgr2=0x7fbb340fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 shutdown_connections 2024-03-12T04:21:33.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.665+0000 7fbb39148640 1 -- 172.21.15.42:0/982878799 wait complete. 2024-03-12T04:21:33.678 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:33.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:21:33.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 -- 172.21.15.42:0/4117961407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc174069aa0 msgr2=0x7fc174069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 --2- 172.21.15.42:0/4117961407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc174069aa0 0x7fc174069e80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009930 tx=0x7fc15c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:21:33.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 -- 172.21.15.42:0/4117961407 shutdown_connections 2024-03-12T04:21:33.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 --2- 172.21.15.42:0/4117961407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1741064e0 0x7fc17410e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 --2- 172.21.15.42:0/4117961407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc17406a450 0x7fc174105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 --2- 172.21.15.42:0/4117961407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc174069aa0 0x7fc174069e80 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 -- 172.21.15.42:0/4117961407 >> 172.21.15.42:0/4117961407 conn(0x7fc174077150 msgr2=0x7fc174077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 -- 172.21.15.42:0/4117961407 shutdown_connections 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 -- 172.21.15.42:0/4117961407 wait complete. 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.781+0000 7fc179d79640 1 Processor -- start 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc179d79640 1 -- start start 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc179d79640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc174069aa0 0x7fc174100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc179d79640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1741064e0 0x7fc1741005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc179d79640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc174100ae0 0x7fc1741a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc179d79640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1741a5e80 con 0x7fc1741064e0 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc179d79640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1741a5ff0 con 0x7fc174100ae0 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc179d79640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1741a61b0 con 0x7fc174069aa0 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1741064e0 0x7fc1741005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1741064e0 0x7fc1741005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35644/0 (socket says 172.21.15.42:35644) 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 -- 172.21.15.42:0/2982570298 learned_addr learned my addr 172.21.15.42:0/2982570298 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc1737fe640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc174069aa0 0x7fc174100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc173fff640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc174100ae0 0x7fc1741a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 -- 172.21.15.42:0/2982570298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc174069aa0 msgr2=0x7fc174100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc174069aa0 0x7fc174100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 -- 172.21.15.42:0/2982570298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc174100ae0 msgr2=0x7fc1741a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc174100ae0 0x7fc1741a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 -- 172.21.15.42:0/2982570298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c009560 con 0x7fc1741064e0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.782+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1741064e0 0x7fc1741005a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d750 tx=0x7fc16400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.783+0000 7fc170ff9640 1 -- 172.21.15.42:0/2982570298 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164009c30 con 0x7fc1741064e0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.783+0000 7fc170ff9640 1 -- 172.21.15.42:0/2982570298 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc16400de60 con 0x7fc1741064e0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.783+0000 7fc170ff9640 1 -- 172.21.15.42:0/2982570298 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164004e00 con 0x7fc1741064e0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.783+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1741a6490 con 0x7fc1741064e0 2024-03-12T04:21:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.783+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1741a6950 con 0x7fc1741064e0 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.785+0000 7fc170ff9640 1 -- 172.21.15.42:0/2982570298 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fc164004f60 con 0x7fc1741064e0 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.785+0000 7fc17ca1f740 1 Processor -- start 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.785+0000 7fc17ca1f740 1 -- start start 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc170ff9640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc1600a1bd0 0x7fc1600a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775c250 0x55811775c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775cb70 0x558117765210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x558117767b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558117717960 con 0x55811775cb70 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581176cb310 con 0x55811775c250 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581176c8840 con 0x558117765750 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775cb70 0x558117765210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775cb70 0x558117765210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35648/0 (socket says 172.21.15.42:35648) 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 -- 172.21.15.42:0/2850602058 learned_addr learned my addr 172.21.15.42:0/2850602058 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc173fff640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc1600a1bd0 0x7fc1600a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc170ff9640 1 -- 172.21.15.42:0/2982570298 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc1640cd120 con 0x7fc1741064e0 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc1737fe640 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775c250 0x55811775c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 -- 172.21.15.42:0/2850602058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 msgr2=0x558117767b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x558117767b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 -- 172.21.15.42:0/2850602058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775c250 msgr2=0x55811775c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775c250 0x55811775c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.786+0000 7fc172ffd640 1 -- 172.21.15.42:0/2850602058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558117686060 con 0x55811775cb70 2024-03-12T04:21:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.787+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775cb70 0x558117765210 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc164009610 tx=0x7fc164065580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d2263bac5391af server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.787+0000 7fc169ffb640 1 -- 172.21.15.42:0/2850602058 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164099700 con 0x55811775cb70 2024-03-12T04:21:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.787+0000 7fc169ffb640 1 -- 172.21.15.42:0/2850602058 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1640c46c0 con 0x55811775cb70 2024-03-12T04:21:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.787+0000 7fc169ffb640 1 -- 172.21.15.42:0/2850602058 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640980a0 con 0x55811775cb70 2024-03-12T04:21:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2850602058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775cb70 msgr2=0x558117765210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc173fff640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc1600a1bd0 0x7fc1600a4090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x558117757b10 tx=0x7fc16c009330 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775cb70 0x558117765210 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc164009610 tx=0x7fc164065580 comp rx=0 tx=0).stop 2024-03-12T04:21:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2850602058 shutdown_connections 2024-03-12T04:21:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x558117767b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775cb70 0x558117765210 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2850602058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775c250 0x55811775c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2850602058 >> 172.21.15.42:0/2850602058 conn(0x558117756980 msgr2=0x558117756d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2850602058 shutdown_connections 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2850602058 wait complete. 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.788+0000 7fc17ca1f740 1 -- start start 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 1 Processor -- start 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117762800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 0x558117762d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x558117763280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581176c4020 con 0x55811775c250 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558117397430 con 0x55811775cb70 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558117353e00 con 0x558117765750 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc17ca1f740 10 client.0 _ll_register_callbacks cb 0x55811766bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117762800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117762800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35662/0 (socket says 172.21.15.42:35662) 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 -- 172.21.15.42:0/2615847329 learned_addr learned my addr 172.21.15.42:0/2615847329 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 0x558117762d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc173fff640 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x558117763280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 -- 172.21.15.42:0/2615847329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 msgr2=0x558117763280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x558117763280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 -- 172.21.15.42:0/2615847329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 msgr2=0x558117762d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 0x558117762d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.789+0000 7fc1737fe640 1 -- 172.21.15.42:0/2615847329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc164016e70 con 0x55811775c250 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc1737fe640 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117762800 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006010 tx=0x7fc15c02cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505c8630136ab2e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc152ffd640 1 -- 172.21.15.42:0/2615847329 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c004250 con 0x55811775c250 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc152ffd640 1 -- 172.21.15.42:0/2615847329 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc15c045070 con 0x55811775c250 2024-03-12T04:21:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581173c37e0 con 0x55811775c250 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc152ffd640 1 -- 172.21.15.42:0/2615847329 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc15c0043b0 con 0x55811775c250 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 10 client.17259 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5581176c26a0 con 0x55811775c250 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc152ffd640 1 -- 172.21.15.42:0/2615847329 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5581173c37e0 con 0x55811775c250 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc152ffd640 1 -- 172.21.15.42:0/2615847329 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc15c03c040 con 0x55811775c250 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 10 client.17259 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 10 client.17259 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 20 client.17259 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558117775b70 0x558117777f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 4 client.17259 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558117400720 con 0x558117775b70 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.790+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558117775b70 0x558117777f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.791+0000 7fc152ffd640 10 client.17259 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.791+0000 7fc172ffd640 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558117775b70 0x558117777f80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=568de7f722b9d1c7 server_cookie=143bbe00eb7fc5b2 in_seq=0 out_seq=0 2024-03-12T04:21:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.791+0000 7fc152ffd640 1 -- 172.21.15.42:0/2615847329 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558117400720 con 0x558117775b70 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.791+0000 7fc152ffd640 10 client.17259 handle_command_reply: tid=0 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.791+0000 7fc17ca1f740 1 client.17259 shutdown 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.791+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 msgr2=0x558117762800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.791+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117762800 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006010 tx=0x7fc15c02cae0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 shutdown_connections 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x558117763280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 0x558117762d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558117775b70 0x558117777f80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/2615847329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117762800 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc152ffd640 0 client.17259 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 >> 172.21.15.42:0/2615847329 conn(0x558117756980 msgr2=0x5581177586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 shutdown_connections 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 1 -- 172.21.15.42:0/2615847329 wait complete. 2024-03-12T04:21:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.792+0000 7fc17ca1f740 20 client.17259 trim_cache size 0 max 16384 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.803+0000 7fc17ca1f740 1 Processor -- start 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc17ca1f740 1 -- start start 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117764f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 0x5581177630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x55811775db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558117649130 con 0x55811775c250 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811775beb0 con 0x55811775cb70 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811777d1d0 con 0x558117765750 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117764f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117764f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35672/0 (socket says 172.21.15.42:35672) 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 -- 172.21.15.42:0/3238367227 learned_addr learned my addr 172.21.15.42:0/3238367227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc173fff640 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x55811775db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 -- 172.21.15.42:0/3238367227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 msgr2=0x55811775db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x55811775db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 -- 172.21.15.42:0/3238367227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 msgr2=0x5581177630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 0x5581177630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 -- 172.21.15.42:0/3238367227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55811765ec20 con 0x55811775c250 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc173fff640 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x55811775db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1737fe640 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117764f30 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02bc70 tx=0x7fc15c004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ccdd8daa9ad2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1697fa640 1 -- 172.21.15.42:0/3238367227 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c005d50 con 0x55811775c250 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1697fa640 1 -- 172.21.15.42:0/3238367227 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc15c043070 con 0x55811775c250 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.804+0000 7fc1697fa640 1 -- 172.21.15.42:0/3238367227 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c003850 con 0x55811775c250 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3238367227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 msgr2=0x558117764f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117764f30 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02bc70 tx=0x7fc15c004fe0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3238367227 shutdown_connections 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558117765750 0x55811775db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55811775cb70 0x5581177630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3238367227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x558117764f30 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3238367227 >> 172.21.15.42:0/3238367227 conn(0x558117756980 msgr2=0x558117751a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3238367227 shutdown_connections 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3238367227 wait complete. 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 -- start start 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.805+0000 7fc17ca1f740 1 Processor -- start 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x55811776af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55811775cb70 0x55811776b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558117765750 0x55811776b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811776cee0 con 0x55811775c250 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581177cc0b0 con 0x558117765750 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581177cc220 con 0x55811775cb70 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 10 client.0 _ll_register_callbacks cb 0x7fc164004740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x55811776af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x55811776af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35678/0 (socket says 172.21.15.42:35678) 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 -- 172.21.15.42:0/3075112109 learned_addr learned my addr 172.21.15.42:0/3075112109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc173fff640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558117765750 0x55811776b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc172ffd640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55811775cb70 0x55811776b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 -- 172.21.15.42:0/3075112109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55811775cb70 msgr2=0x55811776b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55811775cb70 0x55811776b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 -- 172.21.15.42:0/3075112109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558117765750 msgr2=0x55811776b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558117765750 0x55811776b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 -- 172.21.15.42:0/3075112109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581176c26a0 con 0x55811775c250 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc173fff640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558117765750 0x55811776b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc1737fe640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x55811776af40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02c240 tx=0x7fc15c005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05a7a6ce6d03ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc152ffd640 1 -- 172.21.15.42:0/3075112109 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c004250 con 0x55811775c250 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc152ffd640 1 -- 172.21.15.42:0/3075112109 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc15c045040 con 0x55811775c250 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.806+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1640c46c0 con 0x55811775c250 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc152ffd640 1 -- 172.21.15.42:0/3075112109 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc15c0050d0 con 0x55811775c250 2024-03-12T04:21:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 10 client.17265 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5581176848c0 con 0x55811775c250 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc152ffd640 1 -- 172.21.15.42:0/3075112109 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1640c46c0 con 0x55811775c250 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc152ffd640 1 -- 172.21.15.42:0/3075112109 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc15c03c070 con 0x55811775c250 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 10 client.17265 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 10 client.17265 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 20 client.17265 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5581177cd5e0 0x5581177cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 4 client.17265 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55811744f890 con 0x5581177cd5e0 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc172ffd640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5581177cd5e0 0x5581177cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc152ffd640 10 client.17265 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.807+0000 7fc172ffd640 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5581177cd5e0 0x5581177cfa00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea8a8fc833a4aa87 server_cookie=3ca9688139733b75 in_seq=0 out_seq=0 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc152ffd640 1 -- 172.21.15.42:0/3075112109 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55811744f890 con 0x5581177cd5e0 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc152ffd640 10 client.17265 handle_command_reply: tid=0 2024-03-12T04:21:33.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 client.17265 shutdown 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 msgr2=0x55811776af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x55811776af40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02c240 tx=0x7fc15c005d70 comp rx=0 tx=0).stop 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 shutdown_connections 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558117765750 0x55811776b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55811775cb70 0x55811776b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5581177cd5e0 0x5581177cfa00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 --2- 172.21.15.42:0/3075112109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55811775c250 0x55811776af40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc152ffd640 0 client.17265 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 >> 172.21.15.42:0/3075112109 conn(0x558117756980 msgr2=0x55811775db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 shutdown_connections 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 1 -- 172.21.15.42:0/3075112109 wait complete. 2024-03-12T04:21:33.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc17ca1f740 20 client.17265 trim_cache size 0 max 16384 2024-03-12T04:21:33.818 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:33.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc1600a1bd0 msgr2=0x7fc1600a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc1600a1bd0 0x7fc1600a4090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x558117757b10 tx=0x7fc16c009330 comp rx=0 tx=0).stop 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1741064e0 msgr2=0x7fc1741005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1741064e0 0x7fc1741005a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d750 tx=0x7fc16400dc10 comp rx=0 tx=0).stop 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 shutdown_connections 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc1600a1bd0 0x7fc1600a4090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc174100ae0 0x7fc1741a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1741064e0 0x7fc1741005a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 --2- 172.21.15.42:0/2982570298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc174069aa0 0x7fc174100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.808+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 >> 172.21.15.42:0/2982570298 conn(0x7fc174077150 msgr2=0x7fc1740fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.809+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 shutdown_connections 2024-03-12T04:21:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.809+0000 7fc179d79640 1 -- 172.21.15.42:0/2982570298 wait complete. 2024-03-12T04:21:33.822 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:33.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:21:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 -- 172.21.15.42:0/1387808080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 msgr2=0x7f2cc4100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/1387808080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4100680 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009930 tx=0x7f2cb802bc30 comp rx=0 tx=0).stop 2024-03-12T04:21:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 -- 172.21.15.42:0/1387808080 shutdown_connections 2024-03-12T04:21:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/1387808080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cc4105370 0x7f2cc410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/1387808080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc4100c50 0x7f2cc4104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/1387808080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4100680 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 -- 172.21.15.42:0/1387808080 >> 172.21.15.42:0/1387808080 conn(0x7f2cc4076400 msgr2=0x7f2cc4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 -- 172.21.15.42:0/1387808080 shutdown_connections 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 -- 172.21.15.42:0/1387808080 wait complete. 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 Processor -- start 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.924+0000 7f2ccb20e640 1 -- start start 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2ccb20e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2ccb20e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc4105370 0x7f2cc4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2ccb20e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cc4199050 0x7f2cc41a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2ccb20e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41a3ca0 con 0x7f2cc41002a0 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2ccb20e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41a3e10 con 0x7f2cc4105370 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2ccb20e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41a3fd0 con 0x7f2cc4199050 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35684/0 (socket says 172.21.15.42:35684) 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 -- 172.21.15.42:0/818503542 learned_addr learned my addr 172.21.15.42:0/818503542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc4105370 0x7f2cc4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cc4199050 0x7f2cc41a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 -- 172.21.15.42:0/818503542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cc4199050 msgr2=0x7f2cc41a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cc4199050 0x7f2cc41a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 -- 172.21.15.42:0/818503542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc4105370 msgr2=0x7f2cc4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc4105370 0x7f2cc4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 -- 172.21.15.42:0/818503542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8009560 con 0x7f2cc41002a0 2024-03-12T04:21:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4198530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005ab0 tx=0x7f2cb80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc1ffb640 1 -- 172.21.15.42:0/818503542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80044c0 con 0x7f2cc41002a0 2024-03-12T04:21:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc41a4250 con 0x7f2cc41002a0 2024-03-12T04:21:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc1ffb640 1 -- 172.21.15.42:0/818503542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb8003aa0 con 0x7f2cc41002a0 2024-03-12T04:21:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.925+0000 7f2cc1ffb640 1 -- 172.21.15.42:0/818503542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8005540 con 0x7f2cc41002a0 2024-03-12T04:21:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.926+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc41a4760 con 0x7f2cc41002a0 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2cc1ffb640 1 -- 172.21.15.42:0/818503542 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f2cb8003c10 con 0x7f2cc41002a0 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 Processor -- start 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2cc1ffb640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2cb40a1ba0 0x7f2cb40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 -- start start 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2cb40a1ba0 0x7f2cb40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2cc1ffb640 1 -- 172.21.15.42:0/818503542 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2cb802de40 con 0x7f2cc41002a0 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19273960 con 0x55de192b8b70 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19227310 con 0x55de192b8250 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.928+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19224840 con 0x55de192c1750 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35700/0 (socket says 172.21.15.42:35700) 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 -- 172.21.15.42:0/4179831780 learned_addr learned my addr 172.21.15.42:0/4179831780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 -- 172.21.15.42:0/4179831780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 msgr2=0x55de192c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 -- 172.21.15.42:0/4179831780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 msgr2=0x55de192b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 -- 172.21.15.42:0/4179831780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de191e2060 con 0x55de192b8b70 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2cb40a1ba0 0x7f2cb40a4060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0092f0 tx=0x7f2cbc010740 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192c1210 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000bc70 tx=0x7f2cb00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344b1d858dd29207 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2caeffd640 1 -- 172.21.15.42:0/4179831780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0005220 con 0x55de192b8b70 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2caeffd640 1 -- 172.21.15.42:0/4179831780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb0015070 con 0x55de192b8b70 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.929+0000 7f2caeffd640 1 -- 172.21.15.42:0/4179831780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0011640 con 0x55de192b8b70 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4179831780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 msgr2=0x55de192c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192c1210 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000bc70 tx=0x7f2cb00107a0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4179831780 shutdown_connections 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192c1210 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4179831780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4179831780 >> 172.21.15.42:0/4179831780 conn(0x55de192b2980 msgr2=0x55de192b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4179831780 shutdown_connections 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4179831780 wait complete. 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 -- start start 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.930+0000 7f2ccdde8740 1 Processor -- start 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19220020 con 0x55de192b8b70 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de18ef3430 con 0x55de192b8250 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de18eafe00 con 0x55de192c1750 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 10 client.0 _ll_register_callbacks cb 0x55de191c7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192bec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35706/0 (socket says 172.21.15.42:35706) 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 -- 172.21.15.42:0/1103504691 learned_addr learned my addr 172.21.15.42:0/1103504691 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 -- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 msgr2=0x55de192bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192bf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 -- 172.21.15.42:0/1103504691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 msgr2=0x55de192be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 -- 172.21.15.42:0/1103504691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb000b8a0 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192bec90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0011010 tx=0x7f2cb0004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26190411d0539e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ca3fff640 1 -- 172.21.15.42:0/1103504691 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0021070 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ca3fff640 1 -- 172.21.15.42:0/1103504691 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2cb0017070 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de18f1f7e0 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.931+0000 7f2ca3fff640 1 -- 172.21.15.42:0/1103504691 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cb0004040 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 10 client.17277 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55de1921e6a0 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ca3fff640 1 -- 172.21.15.42:0/1103504691 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55de18f1f7e0 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ca3fff640 1 -- 172.21.15.42:0/1103504691 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2cb0003710 con 0x55de192b8b70 2024-03-12T04:21:33.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 10 client.17277 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 10 client.17277 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 20 client.17277 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de192d18f0 0x55de192d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 4 client.17277 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de18f5c720 con 0x55de192d18f0 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.932+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de192d18f0 0x55de192d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.933+0000 7f2ca3fff640 10 client.17277 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.933+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de192d18f0 0x55de192d3d10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45b63c456786090c server_cookie=8d2c0866571487 in_seq=0 out_seq=0 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.933+0000 7f2ca3fff640 1 -- 172.21.15.42:0/1103504691 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de18f5c720 con 0x55de192d18f0 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.933+0000 7f2ca3fff640 10 client.17277 handle_command_reply: tid=0 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.933+0000 7f2ccdde8740 1 client.17277 shutdown 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 msgr2=0x55de192bec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192bec90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0011010 tx=0x7f2cb0004f20 comp rx=0 tx=0).stop 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 shutdown_connections 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de192d18f0 0x55de192d3d10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ca3fff640 0 client.17277 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8b70 0x55de192bec90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/1103504691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8250 0x55de192be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 >> 172.21.15.42:0/1103504691 conn(0x55de192b2980 msgr2=0x55de192b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 shutdown_connections 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 1 -- 172.21.15.42:0/1103504691 wait complete. 2024-03-12T04:21:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.934+0000 7f2ccdde8740 20 client.17277 trim_cache size 0 max 16384 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.945+0000 7f2ccdde8740 1 Processor -- start 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2ccdde8740 1 -- start start 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8250 0x55de192c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 0x55de192beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de191a5130 con 0x55de192b8250 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de192d8f60 con 0x55de192b8b70 2024-03-12T04:21:33.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de192ae080 con 0x55de192c1750 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8250 0x55de192c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8250 0x55de192c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35722/0 (socket says 172.21.15.42:35722) 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 -- 172.21.15.42:0/542681475 learned_addr learned my addr 172.21.15.42:0/542681475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 -- 172.21.15.42:0/542681475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 msgr2=0x55de192bb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/542681475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 -- 172.21.15.42:0/542681475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 msgr2=0x55de192beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/542681475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 0x55de192beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 -- 172.21.15.42:0/542681475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de191bac20 con 0x55de192b8250 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/542681475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8250 0x55de192c0e80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006010 tx=0x7f2cb8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c47b6ac9f375e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cae7fc640 1 -- 172.21.15.42:0/542681475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80db070 con 0x55de192b8250 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cae7fc640 1 -- 172.21.15.42:0/542681475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb80d4070 con 0x55de192b8250 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.946+0000 7f2cae7fc640 1 -- 172.21.15.42:0/542681475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004dc0 con 0x55de192b8250 2024-03-12T04:21:33.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 -- 172.21.15.42:0/542681475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8250 msgr2=0x55de192c0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/542681475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8250 0x55de192c0e80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006010 tx=0x7f2cb8009260 comp rx=0 tx=0).stop 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 -- 172.21.15.42:0/542681475 shutdown_connections 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/542681475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192c1750 0x55de192bb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/542681475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 0x55de192beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/542681475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192b8250 0x55de192c0e80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 -- 172.21.15.42:0/542681475 >> 172.21.15.42:0/542681475 conn(0x55de192b2980 msgr2=0x55de192b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 -- 172.21.15.42:0/542681475 shutdown_connections 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 -- 172.21.15.42:0/542681475 wait complete. 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 -- start start 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.947+0000 7f2ccdde8740 1 Processor -- start 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192b8250 0x55de192c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 0x55de192c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ccdde8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192c1750 0x55de19327e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de192c8370 con 0x55de192c1750 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19328380 con 0x55de192b8b70 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ccdde8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19328540 con 0x55de192b8250 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ccdde8740 10 client.0 _ll_register_callbacks cb 0x7f2cb000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192c1750 0x55de19327e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192c1750 0x55de19327e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35736/0 (socket says 172.21.15.42:35736) 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 -- 172.21.15.42:0/4222388739 learned_addr learned my addr 172.21.15.42:0/4222388739 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192b8250 0x55de192c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 -- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192b8250 msgr2=0x55de192c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192b8250 0x55de192c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc3fff640 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 0x55de192c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 -- 172.21.15.42:0/4222388739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 msgr2=0x55de192c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 0x55de192c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 -- 172.21.15.42:0/4222388739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc41a4760 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2cc97ab640 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192c1750 0x55de19327e40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc012780 tx=0x7f2cbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985b4ab941a2edf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.948+0000 7f2ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ca3fff640 1 -- 172.21.15.42:0/4222388739 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc028670 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ca3fff640 1 -- 172.21.15.42:0/4222388739 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2cbc019070 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cb0015070 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ca3fff640 1 -- 172.21.15.42:0/4222388739 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cbc012e30 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 10 client.17283 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55de191e08c0 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ca3fff640 1 -- 172.21.15.42:0/4222388739 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb0015070 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ca3fff640 1 -- 172.21.15.42:0/4222388739 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2cbc032af0 con 0x55de192c1750 2024-03-12T04:21:33.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 10 client.17283 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 10 client.17283 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 20 client.17283 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de19329e00 0x55de1932c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 4 client.17283 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de18fab890 con 0x55de19329e00 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.949+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de19329e00 0x55de1932c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2cc8faa640 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de19329e00 0x55de1932c1f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7d403a121288188 server_cookie=7bbe34ca9a00eadb in_seq=0 out_seq=0 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ca3fff640 10 client.17283 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ca3fff640 1 -- 172.21.15.42:0/4222388739 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de18fab890 con 0x55de19329e00 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ca3fff640 10 client.17283 handle_command_reply: tid=0 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 client.17283 shutdown 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192c1750 msgr2=0x55de19327e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192c1750 0x55de19327e40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc012780 tx=0x7f2cbc004a40 comp rx=0 tx=0).stop 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 shutdown_connections 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55de19329e00 0x55de1932c1f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de192c1750 0x55de19327e40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de192b8b70 0x55de192c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ca3fff640 0 client.17283 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 --2- 172.21.15.42:0/4222388739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de192b8250 0x55de192c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 >> 172.21.15.42:0/4222388739 conn(0x55de192b2980 msgr2=0x55de192b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 shutdown_connections 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 1 -- 172.21.15.42:0/4222388739 wait complete. 2024-03-12T04:21:33.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.950+0000 7f2ccdde8740 20 client.17283 trim_cache size 0 max 16384 2024-03-12T04:21:33.962 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:33.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2cb40a1ba0 msgr2=0x7f2cb40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2cb40a1ba0 0x7f2cb40a4060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0092f0 tx=0x7f2cbc010740 comp rx=0 tx=0).stop 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 msgr2=0x7f2cc4198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4198530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005ab0 tx=0x7f2cb80040c0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 shutdown_connections 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2cb40a1ba0 0x7f2cb40a4060 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cc4199050 0x7f2cc41a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc4105370 0x7f2cc4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 --2- 172.21.15.42:0/818503542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc41002a0 0x7f2cc4198530 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 >> 172.21.15.42:0/818503542 conn(0x7f2cc4076400 msgr2=0x7f2cc40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 shutdown_connections 2024-03-12T04:21:33.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:33.951+0000 7f2ccb20e640 1 -- 172.21.15.42:0/818503542 wait complete. 2024-03-12T04:21:33.964 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:33.964 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:21:34.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.067+0000 7fbadecc0640 1 -- 172.21.15.42:0/3243848506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad8100260 msgr2=0x7fbad8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.067+0000 7fbadecc0640 1 --2- 172.21.15.42:0/3243848506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad8100260 0x7fbad8100640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009cd0 tx=0x7fbad002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.067+0000 7fbadecc0640 1 -- 172.21.15.42:0/3243848506 shutdown_connections 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.067+0000 7fbadecc0640 1 --2- 172.21.15.42:0/3243848506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbad8105330 0x7fbad810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.067+0000 7fbadecc0640 1 --2- 172.21.15.42:0/3243848506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad8100c10 0x7fbad8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.067+0000 7fbadecc0640 1 --2- 172.21.15.42:0/3243848506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad8100260 0x7fbad8100640 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.067+0000 7fbadecc0640 1 -- 172.21.15.42:0/3243848506 >> 172.21.15.42:0/3243848506 conn(0x7fbad8077180 msgr2=0x7fbad8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 -- 172.21.15.42:0/3243848506 shutdown_connections 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 -- 172.21.15.42:0/3243848506 wait complete. 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 Processor -- start 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 -- start start 2024-03-12T04:21:34.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad8100260 0x7fbad8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbad8105330 0x7fbad8197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad819afa0 0x7fbad819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad819b990 con 0x7fbad819afa0 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad819bb00 con 0x7fbad8100260 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadecc0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad819bcc0 con 0x7fbad8105330 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadd25d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad819afa0 0x7fbad819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadd25d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad819afa0 0x7fbad819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35756/0 (socket says 172.21.15.42:35756) 2024-03-12T04:21:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadca5c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad8100260 0x7fbad8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.068+0000 7fbadd25d640 1 -- 172.21.15.42:0/4141656313 learned_addr learned my addr 172.21.15.42:0/4141656313 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbad7fff640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbad8105330 0x7fbad8197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadd25d640 1 -- 172.21.15.42:0/4141656313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbad8105330 msgr2=0x7fbad8197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadd25d640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbad8105330 0x7fbad8197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadd25d640 1 -- 172.21.15.42:0/4141656313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad8100260 msgr2=0x7fbad8197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadd25d640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad8100260 0x7fbad8197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadd25d640 1 -- 172.21.15.42:0/4141656313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009900 con 0x7fbad819afa0 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadd25d640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad819afa0 0x7fbad819b450 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009fd0 tx=0x7fbac800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbad5ffb640 1 -- 172.21.15.42:0/4141656313 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011830 con 0x7fbad819afa0 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad81a4330 con 0x7fbad819afa0 2024-03-12T04:21:34.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbad5ffb640 1 -- 172.21.15.42:0/4141656313 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbac8011e40 con 0x7fbad819afa0 2024-03-12T04:21:34.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbad5ffb640 1 -- 172.21.15.42:0/4141656313 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800f2e0 con 0x7fbad819afa0 2024-03-12T04:21:34.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.069+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81a47e0 con 0x7fbad819afa0 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.071+0000 7fbad5ffb640 1 -- 172.21.15.42:0/4141656313 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fbac8011990 con 0x7fbad819afa0 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.071+0000 7fbae196c740 1 Processor -- start 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbae196c740 1 -- start start 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbad5ffb640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbac00a1ba0 0x7fbac00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbae196c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b7435ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbae196c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b7435c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbae196c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbae196c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b743575960 con 0x55b7435c3750 2024-03-12T04:21:34.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbae196c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b743529310 con 0x55b7435ba250 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbae196c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b743526840 con 0x55b7435bab70 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbad7fff640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbac00a1ba0 0x7fbac00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbad5ffb640 1 -- 172.21.15.42:0/4141656313 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbac80cceb0 con 0x7fbad819afa0 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35766/0 (socket says 172.21.15.42:35766) 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 -- 172.21.15.42:0/3999956528 learned_addr learned my addr 172.21.15.42:0/3999956528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbad7fff640 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b7435c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadca5c640 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b7435ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 -- 172.21.15.42:0/3999956528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 msgr2=0x55b7435c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b7435c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 -- 172.21.15.42:0/3999956528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 msgr2=0x55b7435ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b7435ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 -- 172.21.15.42:0/3999956528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7434e4060 con 0x55b7435c3750 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadca5c640 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b7435ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbad7fff640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbac00a1ba0 0x7fbac00a4060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbad80fd900 tx=0x7fbacc0127c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.072+0000 7fbadd25d640 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c5b40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbac80a1e50 tx=0x7fbac8099ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c58710f136d12b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbabeffd640 1 -- 172.21.15.42:0/3999956528 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac809a2e0 con 0x55b7435c3750 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbabeffd640 1 -- 172.21.15.42:0/3999956528 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbac8014120 con 0x55b7435c3750 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbabeffd640 1 -- 172.21.15.42:0/3999956528 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8010380 con 0x55b7435c3750 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 -- 172.21.15.42:0/3999956528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 msgr2=0x55b7435c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c5b40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbac80a1e50 tx=0x7fbac8099ae0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 -- 172.21.15.42:0/3999956528 shutdown_connections 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c5b40 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b7435c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 --2- 172.21.15.42:0/3999956528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b7435ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 -- 172.21.15.42:0/3999956528 >> 172.21.15.42:0/3999956528 conn(0x55b7435b4980 msgr2=0x55b7435b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 -- 172.21.15.42:0/3999956528 shutdown_connections 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.073+0000 7fbae196c740 1 -- 172.21.15.42:0/3999956528 wait complete. 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 -- start start 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 Processor -- start 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 0x55b7435c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b743522020 con 0x55b7435bab70 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7431f5430 con 0x55b7435c3750 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7431b1e00 con 0x55b7435ba250 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbae196c740 10 client.0 _ll_register_callbacks cb 0x55b7434c9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35782/0 (socket says 172.21.15.42:35782) 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 -- 172.21.15.42:0/2276897788 learned_addr learned my addr 172.21.15.42:0/2276897788 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbadca5c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbadd25d640 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 0x55b7435c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 -- 172.21.15.42:0/2276897788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 msgr2=0x55b7435c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 -- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 msgr2=0x55b7435c11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 0x55b7435c11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.074+0000 7fbad7fff640 1 -- 172.21.15.42:0/2276897788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81a47e0 con 0x55b7435bab70 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbad7fff640 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c0cb0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbacc010b40 tx=0x7fbacc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc62dbf58b2503e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbab7fff640 1 -- 172.21.15.42:0/2276897788 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00da50 con 0x55b7435bab70 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbab7fff640 1 -- 172.21.15.42:0/2276897788 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbacc00dbb0 con 0x55b7435bab70 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7432217e0 con 0x55b7435bab70 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbab7fff640 1 -- 172.21.15.42:0/2276897788 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbacc00dd70 con 0x55b7435bab70 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 10 client.17295 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b7435206a0 con 0x55b7435bab70 2024-03-12T04:21:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbab7fff640 1 -- 172.21.15.42:0/2276897788 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b7432217e0 con 0x55b7435bab70 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbab7fff640 1 -- 172.21.15.42:0/2276897788 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbacc01f420 con 0x55b7435bab70 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 10 client.17295 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 10 client.17295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 20 client.17295 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b7435d3ae0 0x55b7435d5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 4 client.17295 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.075+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b74325e720 con 0x55b7435d3ae0 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.076+0000 7fbadca5c640 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b7435d3ae0 0x55b7435d5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.076+0000 7fbab7fff640 10 client.17295 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.076+0000 7fbadca5c640 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b7435d3ae0 0x55b7435d5f00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=293b8411dfca10cc server_cookie=cd8a81504a38f618 in_seq=0 out_seq=0 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbab7fff640 1 -- 172.21.15.42:0/2276897788 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b74325e720 con 0x55b7435d3ae0 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbab7fff640 10 client.17295 handle_command_reply: tid=0 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 client.17295 shutdown 2024-03-12T04:21:34.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 msgr2=0x55b7435c0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c0cb0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbacc010b40 tx=0x7fbacc004730 comp rx=0 tx=0).stop 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 shutdown_connections 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b7435d3ae0 0x55b7435d5f00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 0x55b7435c11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c0cb0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbab7fff640 0 client.17295 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 --2- 172.21.15.42:0/2276897788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 >> 172.21.15.42:0/2276897788 conn(0x55b7435b4980 msgr2=0x55b7435b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 shutdown_connections 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 1 -- 172.21.15.42:0/2276897788 wait complete. 2024-03-12T04:21:34.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.077+0000 7fbae196c740 20 client.17295 trim_cache size 0 max 16384 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 Processor -- start 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 -- start start 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 0x55b7435ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7434a7130 con 0x55b7435bab70 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7435db110 con 0x55b7435c3750 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbae196c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7435bc360 con 0x55b7435ba250 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35794/0 (socket says 172.21.15.42:35794) 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 -- 172.21.15.42:0/323937695 learned_addr learned my addr 172.21.15.42:0/323937695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbadca5c640 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 -- 172.21.15.42:0/323937695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 msgr2=0x55b7435b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 -- 172.21.15.42:0/323937695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 msgr2=0x55b7435ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 0x55b7435ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.089+0000 7fbad7fff640 1 -- 172.21.15.42:0/323937695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7434bcc20 con 0x55b7435bab70 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbad7fff640 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c1010 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbacc010b10 tx=0x7fbacc005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a708a68fb87d33 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbabe7fc640 1 -- 172.21.15.42:0/323937695 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00ad70 con 0x55b7435bab70 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbabe7fc640 1 -- 172.21.15.42:0/323937695 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbacc018070 con 0x55b7435bab70 2024-03-12T04:21:34.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbabe7fc640 1 -- 172.21.15.42:0/323937695 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc031690 con 0x55b7435bab70 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 -- 172.21.15.42:0/323937695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 msgr2=0x55b7435c1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c1010 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbacc010b10 tx=0x7fbacc005fb0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 -- 172.21.15.42:0/323937695 shutdown_connections 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435c3750 0x55b7435ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435bab70 0x55b7435c1010 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 --2- 172.21.15.42:0/323937695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435ba250 0x55b7435b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 -- 172.21.15.42:0/323937695 >> 172.21.15.42:0/323937695 conn(0x55b7435b4980 msgr2=0x55b7435b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 -- 172.21.15.42:0/323937695 shutdown_connections 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.090+0000 7fbae196c740 1 -- 172.21.15.42:0/323937695 wait complete. 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 -- start start 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 Processor -- start 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b743624520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b743624a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7435ca050 con 0x55b7435c3750 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7435ca1c0 con 0x55b7435ba250 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7435ca380 con 0x55b7435bab70 2024-03-12T04:21:34.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbae196c740 10 client.0 _ll_register_callbacks cb 0x7fbac8010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35806/0 (socket says 172.21.15.42:35806) 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 -- 172.21.15.42:0/3915113718 learned_addr learned my addr 172.21.15.42:0/3915113718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadca5c640 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b743624520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbad7fff640 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b743624a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 -- 172.21.15.42:0/3915113718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 msgr2=0x55b743624a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b743624a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 -- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 msgr2=0x55b743624520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b743624520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.091+0000 7fbadd25d640 1 -- 172.21.15.42:0/3915113718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7435206a0 con 0x55b7435c3750 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbadd25d640 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c9980 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbac8099b70 tx=0x7fbac800ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaee977f8080c5e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbab7fff640 1 -- 172.21.15.42:0/3915113718 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac809a2e0 con 0x55b7435c3750 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbab7fff640 1 -- 172.21.15.42:0/3915113718 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbac809dd30 con 0x55b7435c3750 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbac8014120 con 0x55b7435c3750 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbab7fff640 1 -- 172.21.15.42:0/3915113718 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac8099be0 con 0x55b7435c3750 2024-03-12T04:21:34.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 10 client.17301 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b7434e28c0 con 0x55b7435c3750 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbab7fff640 1 -- 172.21.15.42:0/3915113718 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbac8063080 con 0x55b7435c3750 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbab7fff640 1 -- 172.21.15.42:0/3915113718 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbac80ca870 con 0x55b7435c3750 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 10 client.17301 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 20 client.17301 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b74362c050 0x55b74362e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 4 client.17301 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:34.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.092+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7432ad890 con 0x55b74362c050 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbadca5c640 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b74362c050 0x55b74362e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbadca5c640 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b74362c050 0x55b74362e470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a7096fb41d0c129 server_cookie=e8ba6607281ad3e1 in_seq=0 out_seq=0 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbab7fff640 10 client.17301 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbab7fff640 1 -- 172.21.15.42:0/3915113718 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7432ad890 con 0x55b74362c050 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbab7fff640 10 client.17301 handle_command_reply: tid=0 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 client.17301 shutdown 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 msgr2=0x55b7435c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c9980 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbac8099b70 tx=0x7fbac800ffe0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 shutdown_connections 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55b74362c050 0x55b74362e470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7435c3750 0x55b7435c9980 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7435bab70 0x55b743624a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbab7fff640 0 client.17301 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 --2- 172.21.15.42:0/3915113718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7435ba250 0x55b743624520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 >> 172.21.15.42:0/3915113718 conn(0x55b7435b4980 msgr2=0x55b7435b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 shutdown_connections 2024-03-12T04:21:34.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.093+0000 7fbae196c740 1 -- 172.21.15.42:0/3915113718 wait complete. 2024-03-12T04:21:34.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbae196c740 20 client.17301 trim_cache size 0 max 16384 2024-03-12T04:21:34.107 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbac00a1ba0 msgr2=0x7fbac00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbac00a1ba0 0x7fbac00a4060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbad80fd900 tx=0x7fbacc0127c0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad819afa0 msgr2=0x7fbad819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad819afa0 0x7fbad819b450 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009fd0 tx=0x7fbac800b8e0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 shutdown_connections 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbac00a1ba0 0x7fbac00a4060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad819afa0 0x7fbad819b450 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbad8105330 0x7fbad8197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 --2- 172.21.15.42:0/4141656313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad8100260 0x7fbad8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 >> 172.21.15.42:0/4141656313 conn(0x7fbad8077180 msgr2=0x7fbad8068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 shutdown_connections 2024-03-12T04:21:34.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.094+0000 7fbadecc0640 1 -- 172.21.15.42:0/4141656313 wait complete. 2024-03-12T04:21:34.110 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:34.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:21:34.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 -- 172.21.15.42:0/1031505297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac0ffa40 msgr2=0x7f12ac0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 --2- 172.21.15.42:0/1031505297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac0ffa40 0x7f12ac0ffec0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f129c009930 tx=0x7f129c02bcd0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 -- 172.21.15.42:0/1031505297 shutdown_connections 2024-03-12T04:21:34.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 --2- 172.21.15.42:0/1031505297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12ac100400 0x7f12ac10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 --2- 172.21.15.42:0/1031505297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac0ffa40 0x7f12ac0ffec0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 --2- 172.21.15.42:0/1031505297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12ac107380 0x7f12ac107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 -- 172.21.15.42:0/1031505297 >> 172.21.15.42:0/1031505297 conn(0x7f12ac0753c0 msgr2=0x7f12ac0757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 -- 172.21.15.42:0/1031505297 shutdown_connections 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.212+0000 7f12b440e640 1 -- 172.21.15.42:0/1031505297 wait complete. 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 Processor -- start 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 -- start start 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12ac0ffa40 0x7f12ac100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac100400 0x7f12ac101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12ac107380 0x7f12ac1016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1a5b00 con 0x7f12ac100400 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1a5c70 con 0x7f12ac0ffa40 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b440e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1a5e30 con 0x7f12ac107380 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b19a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac100400 0x7f12ac101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b19a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac100400 0x7f12ac101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35824/0 (socket says 172.21.15.42:35824) 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b19a9640 1 -- 172.21.15.42:0/3714105657 learned_addr learned my addr 172.21.15.42:0/3714105657 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.213+0000 7f12b21aa640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12ac0ffa40 0x7f12ac100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b19a9640 1 -- 172.21.15.42:0/3714105657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12ac107380 msgr2=0x7f12ac1016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b19a9640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12ac107380 0x7f12ac1016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b19a9640 1 -- 172.21.15.42:0/3714105657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12ac0ffa40 msgr2=0x7f12ac100c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b19a9640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12ac0ffa40 0x7f12ac100c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b19a9640 1 -- 172.21.15.42:0/3714105657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c009560 con 0x7f12ac100400 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b21aa640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12ac0ffa40 0x7f12ac100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b19a9640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac100400 0x7f12ac101160 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f129c009230 tx=0x7f129c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12a37fe640 1 -- 172.21.15.42:0/3714105657 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c004440 con 0x7f12ac100400 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac1a60b0 con 0x7f12ac100400 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12a37fe640 1 -- 172.21.15.42:0/3714105657 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f129c003aa0 con 0x7f12ac100400 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12a37fe640 1 -- 172.21.15.42:0/3714105657 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0054a0 con 0x7f12ac100400 2024-03-12T04:21:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.214+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac1a6610 con 0x7f12ac100400 2024-03-12T04:21:34.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.216+0000 7f12a37fe640 1 -- 172.21.15.42:0/3714105657 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f129c003c10 con 0x7f12ac100400 2024-03-12T04:21:34.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.216+0000 7f12b6fe6740 1 Processor -- start 2024-03-12T04:21:34.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b6fe6740 1 -- start start 2024-03-12T04:21:34.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12a37fe640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f12980a1ba0 0x7f12980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e122627630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e122630210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b29ab640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f12980a1ba0 0x7f12980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e122632b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12a37fe640 1 -- 172.21.15.42:0/3714105657 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f129c02de40 con 0x7f12ac100400 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1225e2960 con 0x55e122627b70 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122596310 con 0x55e122630750 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122593840 con 0x55e122627250 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e122630210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e122630210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35828/0 (socket says 172.21.15.42:35828) 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 -- 172.21.15.42:0/958399415 learned_addr learned my addr 172.21.15.42:0/958399415 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b21aa640 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e122627630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 -- 172.21.15.42:0/958399415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 msgr2=0x55e122627630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e122627630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 -- 172.21.15.42:0/958399415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 msgr2=0x55e122632b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b29ab640 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e122632b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e122632b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b19a9640 1 -- 172.21.15.42:0/958399415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e122551060 con 0x55e122627b70 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b21aa640 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e122627630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.217+0000 7f12b29ab640 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e122632b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12b29ab640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f12980a1ba0 0x7f12980a4060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005c40 tx=0x7f12a400e720 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12b19a9640 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e122630210 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f129c0bca40 tx=0x7f129c0bd440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc701ea9d81db037 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12a0bfe640 1 -- 172.21.15.42:0/958399415 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0edea0 con 0x55e122627b70 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12a0bfe640 1 -- 172.21.15.42:0/958399415 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f129c0c54e0 con 0x55e122627b70 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12a0bfe640 1 -- 172.21.15.42:0/958399415 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0c5050 con 0x55e122627b70 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12b6fe6740 1 -- 172.21.15.42:0/958399415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 msgr2=0x55e122630210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e122630210 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f129c0bca40 tx=0x7f129c0bd440 comp rx=0 tx=0).stop 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12b6fe6740 1 -- 172.21.15.42:0/958399415 shutdown_connections 2024-03-12T04:21:34.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e122632b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.218+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e122630210 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/958399415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e122627630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 -- 172.21.15.42:0/958399415 >> 172.21.15.42:0/958399415 conn(0x55e122621980 msgr2=0x55e122621d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 -- 172.21.15.42:0/958399415 shutdown_connections 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 -- 172.21.15.42:0/958399415 wait complete. 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 -- start start 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 Processor -- start 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627250 0x55e12262d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e12262dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122630750 0x55e12262e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12258f020 con 0x55e122627b70 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122262430 con 0x55e122627250 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12221ee00 con 0x55e122630750 2024-03-12T04:21:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b6fe6740 10 client.0 _ll_register_callbacks cb 0x55e122536cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b19a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e12262dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b19a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e12262dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35838/0 (socket says 172.21.15.42:35838) 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b19a9640 1 -- 172.21.15.42:0/2591053146 learned_addr learned my addr 172.21.15.42:0/2591053146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.219+0000 7f12b21aa640 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627250 0x55e12262d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b19a9640 1 -- 172.21.15.42:0/2591053146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122630750 msgr2=0x55e12262e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b19a9640 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122630750 0x55e12262e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b19a9640 1 -- 172.21.15.42:0/2591053146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627250 msgr2=0x55e12262d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b19a9640 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627250 0x55e12262d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b19a9640 1 -- 172.21.15.42:0/2591053146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac1a6610 con 0x55e122627b70 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b19a9640 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e12262dcf0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f129c0edd50 tx=0x7f129c0c0f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931c778917680b0b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f1295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f1295ffb640 1 -- 172.21.15.42:0/2591053146 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0edea0 con 0x55e122627b70 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f1295ffb640 1 -- 172.21.15.42:0/2591053146 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f129c0c5050 con 0x55e122627b70 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e12228e7e0 con 0x55e122627b70 2024-03-12T04:21:34.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f1295ffb640 1 -- 172.21.15.42:0/2591053146 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f129c0db2c0 con 0x55e122627b70 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b6fe6740 10 client.17313 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e12258d6a0 con 0x55e122627b70 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f1295ffb640 1 -- 172.21.15.42:0/2591053146 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f129c09f020 con 0x55e122627b70 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f1295ffb640 1 -- 172.21.15.42:0/2591053146 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f129c03c070 con 0x55e122627b70 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b6fe6740 10 client.17313 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b6fe6740 10 client.17313 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b6fe6740 20 client.17313 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.220+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e1226404d0 0x55e1226428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.221+0000 7f12b6fe6740 4 client.17313 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.221+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1222cb720 con 0x55e1226404d0 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.221+0000 7f12b21aa640 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e1226404d0 0x55e1226428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.221+0000 7f12b21aa640 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e1226404d0 0x55e1226428c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6762739e43a0f7f6 server_cookie=ccf53fe83e9b93c9 in_seq=0 out_seq=0 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.221+0000 7f1295ffb640 10 client.17313 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:34.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f1295ffb640 1 -- 172.21.15.42:0/2591053146 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1222cb720 con 0x55e1226404d0 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f1295ffb640 10 client.17313 handle_command_reply: tid=0 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 client.17313 shutdown 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 msgr2=0x55e12262dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e12262dcf0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f129c0edd50 tx=0x7f129c0c0f50 comp rx=0 tx=0).stop 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 shutdown_connections 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e1226404d0 0x55e1226428c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122630750 0x55e12262e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f1295ffb640 0 client.17313 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627b70 0x55e12262dcf0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2591053146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627250 0x55e12262d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 >> 172.21.15.42:0/2591053146 conn(0x55e122621980 msgr2=0x55e1226236d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 shutdown_connections 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2591053146 wait complete. 2024-03-12T04:21:34.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.222+0000 7f12b6fe6740 20 client.17313 trim_cache size 0 max 16384 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 Processor -- start 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 -- start start 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e12262fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627b70 0x55e12262e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122630750 0x55e122628ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122514130 con 0x55e122630750 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1226265d0 con 0x55e122627b70 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122647b10 con 0x55e122627250 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b29ab640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122630750 0x55e122628ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b29ab640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122630750 0x55e122628ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35852/0 (socket says 172.21.15.42:35852) 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b29ab640 1 -- 172.21.15.42:0/1550677212 learned_addr learned my addr 172.21.15.42:0/1550677212 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.234+0000 7f12b21aa640 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e12262fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b29ab640 1 -- 172.21.15.42:0/1550677212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 msgr2=0x55e12262fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b29ab640 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e12262fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b29ab640 1 -- 172.21.15.42:0/1550677212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627b70 msgr2=0x55e12262e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b29ab640 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627b70 0x55e12262e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b29ab640 1 -- 172.21.15.42:0/1550677212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e122529c20 con 0x55e122630750 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b29ab640 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122630750 0x55e122628ad0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f12a4010540 tx=0x7f12a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1dc15ce48f4b0f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f1297fff640 1 -- 172.21.15.42:0/1550677212 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4012710 con 0x55e122630750 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f1297fff640 1 -- 172.21.15.42:0/1550677212 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f12a4012d20 con 0x55e122630750 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f1297fff640 1 -- 172.21.15.42:0/1550677212 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a401ed20 con 0x55e122630750 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b6fe6740 1 -- 172.21.15.42:0/1550677212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122630750 msgr2=0x55e122628ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122630750 0x55e122628ad0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f12a4010540 tx=0x7f12a4013000 comp rx=0 tx=0).stop 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b6fe6740 1 -- 172.21.15.42:0/1550677212 shutdown_connections 2024-03-12T04:21:34.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122630750 0x55e122628ad0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122627b70 0x55e12262e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/1550677212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627250 0x55e12262fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.235+0000 7f12b6fe6740 1 -- 172.21.15.42:0/1550677212 >> 172.21.15.42:0/1550677212 conn(0x55e122621980 msgr2=0x55e12261ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 -- 172.21.15.42:0/1550677212 shutdown_connections 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 -- 172.21.15.42:0/1550677212 wait complete. 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 -- start start 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 Processor -- start 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627250 0x55e122635f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627b70 0x55e122636490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e1226369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122696a40 con 0x55e122627250 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122696bb0 con 0x55e122630750 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e122696d70 con 0x55e122627b70 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b6fe6740 10 client.0 _ll_register_callbacks cb 0x7f129c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:34.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627250 0x55e122635f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627250 0x55e122635f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35854/0 (socket says 172.21.15.42:35854) 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 -- 172.21.15.42:0/2769631543 learned_addr learned my addr 172.21.15.42:0/2769631543 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b19a9640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627b70 0x55e122636490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b29ab640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e1226369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 -- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627b70 msgr2=0x55e122636490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627b70 0x55e122636490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 -- 172.21.15.42:0/2769631543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 msgr2=0x55e1226369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e1226369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b21aa640 1 -- 172.21.15.42:0/2769631543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a400edc0 con 0x55e122627250 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.236+0000 7f12b19a9640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627b70 0x55e122636490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b21aa640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627250 0x55e122635f50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009c00 tx=0x7f12a801af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d492cb3f107367bf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f1295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f1295ffb640 1 -- 172.21.15.42:0/2769631543 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8018040 con 0x55e122627250 2024-03-12T04:21:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f1295ffb640 1 -- 172.21.15.42:0/2769631543 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f12a801a8d0 con 0x55e122627250 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f129c0c54e0 con 0x55e122627250 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f1295ffb640 1 -- 172.21.15.42:0/2769631543 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a8005440 con 0x55e122627250 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 10 client.17319 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e12254f8c0 con 0x55e122627250 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f1295ffb640 1 -- 172.21.15.42:0/2769631543 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f129c0c54e0 con 0x55e122627250 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f1295ffb640 1 -- 172.21.15.42:0/2769631543 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12a80127b0 con 0x55e122627250 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 10 client.17319 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 20 client.17319 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e122698600 0x55e12269aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 4 client.17319 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.237+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e12231a890 con 0x55e122698600 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b19a9640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e122698600 0x55e12269aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b19a9640 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e122698600 0x55e12269aa20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=502ee827ae5caac3 server_cookie=b01f7739031d6353 in_seq=0 out_seq=0 2024-03-12T04:21:34.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f1295ffb640 10 client.17319 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f1295ffb640 1 -- 172.21.15.42:0/2769631543 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e12231a890 con 0x55e122698600 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f1295ffb640 10 client.17319 handle_command_reply: tid=0 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 client.17319 shutdown 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627250 msgr2=0x55e122635f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627250 0x55e122635f50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009c00 tx=0x7f12a801af40 comp rx=0 tx=0).stop 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 shutdown_connections 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e122698600 0x55e12269aa20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e122630750 0x55e1226369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f1295ffb640 0 client.17319 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e122627b70 0x55e122636490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 --2- 172.21.15.42:0/2769631543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e122627250 0x55e122635f50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 >> 172.21.15.42:0/2769631543 conn(0x55e122621980 msgr2=0x55e12262be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 shutdown_connections 2024-03-12T04:21:34.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 1 -- 172.21.15.42:0/2769631543 wait complete. 2024-03-12T04:21:34.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.238+0000 7f12b6fe6740 20 client.17319 trim_cache size 0 max 16384 2024-03-12T04:21:34.248 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f12980a1ba0 msgr2=0x7f12980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f12980a1ba0 0x7f12980a4060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005c40 tx=0x7f12a400e720 comp rx=0 tx=0).stop 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac100400 msgr2=0x7f12ac101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac100400 0x7f12ac101160 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f129c009230 tx=0x7f129c004040 comp rx=0 tx=0).stop 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 shutdown_connections 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f12980a1ba0 0x7f12980a4060 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12ac107380 0x7f12ac1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac100400 0x7f12ac101160 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 --2- 172.21.15.42:0/3714105657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12ac0ffa40 0x7f12ac100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 >> 172.21.15.42:0/3714105657 conn(0x7f12ac0753c0 msgr2=0x7f12ac0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 shutdown_connections 2024-03-12T04:21:34.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.239+0000 7f12b440e640 1 -- 172.21.15.42:0/3714105657 wait complete. 2024-03-12T04:21:34.252 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:21:34.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:21:34.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.353+0000 7f57ecdbc640 1 -- 172.21.15.42:0/4156238655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 msgr2=0x7f57e81027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.353+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/4156238655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e81027f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009ca0 tx=0x7f57d802c180 comp rx=0 tx=0).stop 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 -- 172.21.15.42:0/4156238655 shutdown_connections 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/4156238655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57e810b750 0x7f57e810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/4156238655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e8102d30 0x7f57e810b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/4156238655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e81027f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 -- 172.21.15.42:0/4156238655 >> 172.21.15.42:0/4156238655 conn(0x7f57e80771e0 msgr2=0x7f57e80775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 -- 172.21.15.42:0/4156238655 shutdown_connections 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 -- 172.21.15.42:0/4156238655 wait complete. 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 Processor -- start 2024-03-12T04:21:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 -- start start 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.354+0000 7f57ecdbc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57e8102d30 0x7f57e819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57ecdbc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e810b750 0x7f57e81976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57ecdbc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e8197e40 con 0x7f57e8102410 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57ecdbc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e8197fb0 con 0x7f57e810b750 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57ecdbc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e8198170 con 0x7f57e8102d30 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35870/0 (socket says 172.21.15.42:35870) 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 -- 172.21.15.42:0/3706007213 learned_addr learned my addr 172.21.15.42:0/3706007213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57e8102d30 0x7f57e819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e810b750 0x7f57e81976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 -- 172.21.15.42:0/3706007213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57e8102d30 msgr2=0x7f57e819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57e8102d30 0x7f57e819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 -- 172.21.15.42:0/3706007213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e810b750 msgr2=0x7f57e81976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e810b750 0x7f57e81976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 -- 172.21.15.42:0/3706007213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d8009900 con 0x7f57e8102410 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57e659c640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e819d5f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000f80 tx=0x7f57d80349e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57d77fe640 1 -- 172.21.15.42:0/3706007213 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8032880 con 0x7f57e8102410 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e81983f0 con 0x7f57e8102410 2024-03-12T04:21:34.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57d77fe640 1 -- 172.21.15.42:0/3706007213 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57d8032e90 con 0x7f57e8102410 2024-03-12T04:21:34.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57d77fe640 1 -- 172.21.15.42:0/3706007213 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d803f440 con 0x7f57e8102410 2024-03-12T04:21:34.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.355+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e81a4570 con 0x7f57e8102410 2024-03-12T04:21:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.357+0000 7f57d77fe640 1 -- 172.21.15.42:0/3706007213 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f57d803b040 con 0x7f57e8102410 2024-03-12T04:21:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 Processor -- start 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 -- start start 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414491b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491bb70 0x564144924210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57d77fe640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57d00a1ba0 0x7f57d00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564144924750 0x564144926b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641448d6960 con 0x56414491bb70 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414488a310 con 0x564144924750 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57ef994740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564144887840 con 0x56414491b250 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491bb70 0x564144924210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57e659c640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57d00a1ba0 0x7f57d00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491bb70 0x564144924210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35874/0 (socket says 172.21.15.42:35874) 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57e5d9b640 1 -- 172.21.15.42:0/2854872208 learned_addr learned my addr 172.21.15.42:0/2854872208 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414491b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57d77fe640 1 -- 172.21.15.42:0/3706007213 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f57d80f0430 con 0x7f57e8102410 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.358+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564144924750 0x564144926b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e5d9b640 1 -- 172.21.15.42:0/2854872208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 msgr2=0x56414491b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414491b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e5d9b640 1 -- 172.21.15.42:0/2854872208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564144924750 msgr2=0x564144926b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564144924750 0x564144926b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e5d9b640 1 -- 172.21.15.42:0/2854872208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564144845060 con 0x56414491bb70 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564144924750 0x564144926b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e659c640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57d00a1ba0 0x7f57d00a4060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57d80bd0f0 tx=0x7f57d80af040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491bb70 0x564144924210 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00b7f0 tx=0x7f57dc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6424f8cf5e0387cd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57d4ff9640 1 -- 172.21.15.42:0/2854872208 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00d950 con 0x56414491bb70 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57d4ff9640 1 -- 172.21.15.42:0/2854872208 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57dc00b440 con 0x56414491bb70 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.359+0000 7f57d4ff9640 1 -- 172.21.15.42:0/2854872208 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0123e0 con 0x56414491bb70 2024-03-12T04:21:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- 172.21.15.42:0/2854872208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491bb70 msgr2=0x564144924210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491bb70 0x564144924210 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00b7f0 tx=0x7f57dc00bcb0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- 172.21.15.42:0/2854872208 shutdown_connections 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564144924750 0x564144926b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491bb70 0x564144924210 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 --2- 172.21.15.42:0/2854872208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414491b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- 172.21.15.42:0/2854872208 >> 172.21.15.42:0/2854872208 conn(0x564144915980 msgr2=0x564144915d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- 172.21.15.42:0/2854872208 shutdown_connections 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- 172.21.15.42:0/2854872208 wait complete. 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- start start 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 Processor -- start 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491b250 0x5641449217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x564144921cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564144924750 0x564144922230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564144883020 con 0x56414491b250 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564144556430 con 0x56414491bb70 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.360+0000 7f57ef994740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564144512e00 con 0x564144924750 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491b250 0x5641449217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57ef994740 10 client.0 _ll_register_callbacks cb 0x56414482acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491b250 0x5641449217b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35884/0 (socket says 172.21.15.42:35884) 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 -- 172.21.15.42:0/2089526083 learned_addr learned my addr 172.21.15.42:0/2089526083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x564144921cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564144924750 0x564144922230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 -- 172.21.15.42:0/2089526083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564144924750 msgr2=0x564144922230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564144924750 0x564144922230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 -- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 msgr2=0x564144921cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x564144921cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 -- 172.21.15.42:0/2089526083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc00e040 con 0x56414491b250 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491b250 0x5641449217b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f57e0019780 tx=0x7f57e00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ed1cef3459fc83 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57cdffb640 1 -- 172.21.15.42:0/2089526083 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e00148d0 con 0x56414491b250 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57cdffb640 1 -- 172.21.15.42:0/2089526083 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f57e0014a30 con 0x56414491b250 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641445827e0 con 0x56414491b250 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57cdffb640 1 -- 172.21.15.42:0/2089526083 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e0014bf0 con 0x56414491b250 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57ef994740 10 client.17331 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.361+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5641448816a0 con 0x56414491b250 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57cdffb640 1 -- 172.21.15.42:0/2089526083 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57e0028020 con 0x56414491b250 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57cdffb640 1 -- 172.21.15.42:0/2089526083 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57e00173e0 con 0x56414491b250 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57ef994740 10 client.17331 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57ef994740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57ef994740 20 client.17331 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57ef994740 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564144934ba0 0x564144936fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57ef994740 4 client.17331 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641445bf720 con 0x564144934ba0 2024-03-12T04:21:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.362+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564144934ba0 0x564144936fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564144934ba0 0x564144936fc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f19a4925caacae server_cookie=d7d067db0766842a in_seq=0 out_seq=0 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57cdffb640 10 client.17331 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57cdffb640 1 -- 172.21.15.42:0/2089526083 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641445bf720 con 0x564144934ba0 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57cdffb640 10 client.17331 handle_command_reply: tid=0 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 client.17331 shutdown 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491b250 msgr2=0x5641449217b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491b250 0x5641449217b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f57e0019780 tx=0x7f57e00047d0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 shutdown_connections 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564144934ba0 0x564144936fc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564144924750 0x564144922230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x564144921cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 --2- 172.21.15.42:0/2089526083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56414491b250 0x5641449217b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57cdffb640 0 client.17331 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.363+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 >> 172.21.15.42:0/2089526083 conn(0x564144915980 msgr2=0x5641449176d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.364+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 shutdown_connections 2024-03-12T04:21:34.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.364+0000 7f57ef994740 1 -- 172.21.15.42:0/2089526083 wait complete. 2024-03-12T04:21:34.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.364+0000 7f57ef994740 20 client.17331 trim_cache size 0 max 16384 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 Processor -- start 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 -- start start 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491b250 0x564144923ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491bb70 0x56414491ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414491ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564144808130 con 0x564144924750 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564144931690 con 0x56414491b250 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.375+0000 7f57ef994740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564144922c40 con 0x56414491bb70 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414491ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414491ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35890/0 (socket says 172.21.15.42:35890) 2024-03-12T04:21:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3584000103 learned_addr learned my addr 172.21.15.42:0/3584000103 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491b250 0x564144923ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491bb70 0x56414491ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3584000103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491bb70 msgr2=0x56414491ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491bb70 0x56414491ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3584000103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491b250 msgr2=0x564144923ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491b250 0x564144923ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3584000103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414481dc20 con 0x564144924750 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414491ef50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f57e0017a10 tx=0x7f57e001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2808190db729d2c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57cffff640 1 -- 172.21.15.42:0/3584000103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000a9f0 con 0x564144924750 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57cffff640 1 -- 172.21.15.42:0/3584000103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57e0028070 con 0x564144924750 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.376+0000 7f57cffff640 1 -- 172.21.15.42:0/3584000103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0005c20 con 0x564144924750 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- 172.21.15.42:0/3584000103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 msgr2=0x56414491ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414491ef50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f57e0017a10 tx=0x7f57e001b040 comp rx=0 tx=0).stop 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- 172.21.15.42:0/3584000103 shutdown_connections 2024-03-12T04:21:34.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414491ef50 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491bb70 0x56414491ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 --2- 172.21.15.42:0/3584000103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491b250 0x564144923ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- 172.21.15.42:0/3584000103 >> 172.21.15.42:0/3584000103 conn(0x564144915980 msgr2=0x564144920080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- 172.21.15.42:0/3584000103 shutdown_connections 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- 172.21.15.42:0/3584000103 wait complete. 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- start start 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 Processor -- start 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414492a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x56414492a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414498b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414492af90 con 0x564144924750 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414492b100 con 0x56414491bb70 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414498b5f0 con 0x56414491b250 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57ef994740 10 client.0 _ll_register_callbacks cb 0x7f57dc00b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:34.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414498b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.377+0000 7f57e6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414498b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35896/0 (socket says 172.21.15.42:35896) 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3236436971 learned_addr learned my addr 172.21.15.42:0/3236436971 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x56414492a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414492a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3236436971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 msgr2=0x56414492a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414492a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 msgr2=0x56414492a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x56414492a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 -- 172.21.15.42:0/3236436971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641448816a0 con 0x564144924750 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57e6d9d640 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414498b0b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f57e000aa10 tx=0x7f57e00173e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c4d699b2b84060 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57cdffb640 1 -- 172.21.15.42:0/3236436971 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0017a90 con 0x564144924750 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57cdffb640 1 -- 172.21.15.42:0/3236436971 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f57e0000e40 con 0x564144924750 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57dc00b440 con 0x564144924750 2024-03-12T04:21:34.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57cdffb640 1 -- 172.21.15.42:0/3236436971 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e0019020 con 0x564144924750 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57ef994740 10 client.17337 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5641448438c0 con 0x564144924750 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.378+0000 7f57cdffb640 1 -- 172.21.15.42:0/3236436971 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57dc00b440 con 0x564144924750 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57cdffb640 1 -- 172.21.15.42:0/3236436971 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57e002b030 con 0x564144924750 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57ef994740 10 client.17337 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57ef994740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57ef994740 20 client.17337 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57ef994740 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56414498cb40 0x56414498ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57ef994740 4 client.17337 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56414460e890 con 0x56414498cb40 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56414498cb40 0x56414498ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:34.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57cdffb640 10 client.17337 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57e5d9b640 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56414498cb40 0x56414498ef60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933f2fa765a8df80 server_cookie=59a48a289c444d67 in_seq=0 out_seq=0 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.379+0000 7f57cdffb640 1 -- 172.21.15.42:0/3236436971 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56414460e890 con 0x56414498cb40 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57cdffb640 10 client.17337 handle_command_reply: tid=0 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 client.17337 shutdown 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 msgr2=0x56414498b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414498b0b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f57e000aa10 tx=0x7f57e00173e0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 shutdown_connections 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56414498cb40 0x56414498ef60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564144924750 0x56414498b0b0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57cdffb640 0 client.17337 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414491bb70 0x56414492a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 --2- 172.21.15.42:0/3236436971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56414491b250 0x56414492a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 >> 172.21.15.42:0/3236436971 conn(0x564144915980 msgr2=0x5641449142f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 shutdown_connections 2024-03-12T04:21:34.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 1 -- 172.21.15.42:0/3236436971 wait complete. 2024-03-12T04:21:34.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ef994740 20 client.17337 trim_cache size 0 max 16384 2024-03-12T04:21:34.389 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57d00a1ba0 msgr2=0x7f57d00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57d00a1ba0 0x7f57d00a4060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57d80bd0f0 tx=0x7f57d80af040 comp rx=0 tx=0).stop 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 msgr2=0x7f57e819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e819d5f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000f80 tx=0x7f57d80349e0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 shutdown_connections 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f57d00a1ba0 0x7f57d00a4060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e810b750 0x7f57e81976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57e8102d30 0x7f57e819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 --2- 172.21.15.42:0/3706007213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57e8102410 0x7f57e819d5f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 >> 172.21.15.42:0/3706007213 conn(0x7f57e80771e0 msgr2=0x7f57e80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 shutdown_connections 2024-03-12T04:21:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:34.380+0000 7f57ecdbc640 1 -- 172.21.15.42:0/3706007213 wait complete. 2024-03-12T04:21:34.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:34 smithi042 ceph-mon[32260]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:34.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:34 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/982878799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:21:34.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:34 smithi119 ceph-mon[36119]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:34.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:34 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/982878799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:21:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:34 smithi136 ceph-mon[35571]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:34 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/982878799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:21:35.394 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:21:35.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:21:35.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:21:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 -- 172.21.15.42:0/1701599533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 msgr2=0x7fa27010cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1701599533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa27010cf20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa268009cd0 tx=0x7fa26802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 -- 172.21.15.42:0/1701599533 shutdown_connections 2024-03-12T04:21:35.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1701599533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa27010cf20 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1701599533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa270100c30 0x7fa270104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1701599533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa270100280 0x7fa270100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 -- 172.21.15.42:0/1701599533 >> 172.21.15.42:0/1701599533 conn(0x7fa270077150 msgr2=0x7fa270077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 -- 172.21.15.42:0/1701599533 shutdown_connections 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.511+0000 7fa277a8d640 1 -- 172.21.15.42:0/1701599533 wait complete. 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 Processor -- start 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 -- start start 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa270100280 0x7fa2701975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa270197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27019bea0 0x7fa27019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019c890 con 0x7fa270105350 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019ca00 con 0x7fa27019bea0 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa277a8d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019cb70 con 0x7fa270100280 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa270197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa270197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35902/0 (socket says 172.21.15.42:35902) 2024-03-12T04:21:35.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa275028640 1 -- 172.21.15.42:0/1719938045 learned_addr learned my addr 172.21.15.42:0/1719938045 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa27602a640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27019bea0 0x7fa27019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.512+0000 7fa275829640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa270100280 0x7fa2701975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa275028640 1 -- 172.21.15.42:0/1719938045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa270100280 msgr2=0x7fa2701975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa275028640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa270100280 0x7fa2701975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa275028640 1 -- 172.21.15.42:0/1719938045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27019bea0 msgr2=0x7fa27019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa275028640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27019bea0 0x7fa27019c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa275028640 1 -- 172.21.15.42:0/1719938045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268009900 con 0x7fa270105350 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa275028640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa270197b20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c970 tx=0x7fa26000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa266ffd640 1 -- 172.21.15.42:0/1719938045 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260004440 con 0x7fa270105350 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa266ffd640 1 -- 172.21.15.42:0/1719938045 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa260007d10 con 0x7fa270105350 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa266ffd640 1 -- 172.21.15.42:0/1719938045 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26000f450 con 0x7fa270105350 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2701a4150 con 0x7fa270105350 2024-03-12T04:21:35.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.513+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2701a4650 con 0x7fa270105350 2024-03-12T04:21:35.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.515+0000 7fa266ffd640 1 -- 172.21.15.42:0/1719938045 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa260003680 con 0x7fa270105350 2024-03-12T04:21:35.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 Processor -- start 2024-03-12T04:21:35.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 -- start start 2024-03-12T04:21:35.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b957f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957fb90 0x5606b95886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b9588cf0 0x5606b9589170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b94edf90 con 0x5606b9588cf0 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b94eb4c0 con 0x5606b957fb90 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27a659740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b94e6ca0 con 0x5606b957f1e0 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa266ffd640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa25c0a1c20 0x7fa25c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b9588cf0 0x5606b9589170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b9588cf0 0x5606b9589170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35918/0 (socket says 172.21.15.42:35918) 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27602a640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa25c0a1c20 0x7fa25c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa275028640 1 -- 172.21.15.42:0/2280511602 learned_addr learned my addr 172.21.15.42:0/2280511602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa27602a640 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b957f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa275829640 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957fb90 0x5606b95886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.516+0000 7fa266ffd640 1 -- 172.21.15.42:0/1719938045 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa2600cd910 con 0x7fa270105350 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa275028640 1 -- 172.21.15.42:0/2280511602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 msgr2=0x5606b957f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa275028640 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b957f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa275028640 1 -- 172.21.15.42:0/2280511602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957fb90 msgr2=0x5606b95886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa275028640 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957fb90 0x5606b95886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa275028640 1 -- 172.21.15.42:0/2280511602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606b94a90b0 con 0x5606b9588cf0 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa275829640 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957fb90 0x5606b95886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa27602a640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa25c0a1c20 0x7fa25c0a40e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa268005cc0 tx=0x7fa26802e040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa275028640 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b9588cf0 0x5606b9589170 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa2600a27a0 tx=0x7fa26006d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c8f4b1d5514fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa25bfff640 1 -- 172.21.15.42:0/2280511602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260099e90 con 0x5606b9588cf0 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa25bfff640 1 -- 172.21.15.42:0/2280511602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa260065cd0 con 0x5606b9588cf0 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.517+0000 7fa25bfff640 1 -- 172.21.15.42:0/2280511602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260098830 con 0x5606b9588cf0 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 -- 172.21.15.42:0/2280511602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b9588cf0 msgr2=0x5606b9589170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b9588cf0 0x5606b9589170 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa2600a27a0 tx=0x7fa26006d7e0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 -- 172.21.15.42:0/2280511602 shutdown_connections 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b9588cf0 0x5606b9589170 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957fb90 0x5606b95886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 --2- 172.21.15.42:0/2280511602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b957f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 -- 172.21.15.42:0/2280511602 >> 172.21.15.42:0/2280511602 conn(0x5606b9579840 msgr2=0x5606b9579c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 -- 172.21.15.42:0/2280511602 shutdown_connections 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 -- 172.21.15.42:0/2280511602 wait complete. 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 -- start start 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.518+0000 7fa27a659740 1 Processor -- start 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27a659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957f1e0 0x5606b958c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27a659740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957fb90 0x5606b958cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27a659740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27a659740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b948bf40 con 0x5606b957f1e0 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27a659740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b91ba4b0 con 0x5606b9588cf0 2024-03-12T04:21:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27a659740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b946c670 con 0x5606b957fb90 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27a659740 10 client.0 _ll_register_callbacks cb 0x5606b94d9f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957f1e0 0x5606b958c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957f1e0 0x5606b958c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35920/0 (socket says 172.21.15.42:35920) 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 -- 172.21.15.42:0/2908916289 learned_addr learned my addr 172.21.15.42:0/2908916289 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa27602a640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275028640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957fb90 0x5606b958cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 -- 172.21.15.42:0/2908916289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957fb90 msgr2=0x5606b958cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957fb90 0x5606b958cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 -- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 msgr2=0x5606b958d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 -- 172.21.15.42:0/2908916289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2701a4650 con 0x5606b957f1e0 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275028640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957fb90 0x5606b958cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa275829640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957f1e0 0x5606b958c7d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00bf40 tx=0x7fa26c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1638df2386da7b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.519+0000 7fa2597fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa2597fa640 1 -- 172.21.15.42:0/2908916289 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004230 con 0x5606b957f1e0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa2597fa640 1 -- 172.21.15.42:0/2908916289 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa26c004390 con 0x5606b957f1e0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606b94c0f40 con 0x5606b957f1e0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa2597fa640 1 -- 172.21.15.42:0/2908916289 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa26c0035f0 con 0x5606b957f1e0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 10 client.17349 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5606b94e5320 con 0x5606b957f1e0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa2597fa640 1 -- 172.21.15.42:0/2908916289 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5606b94c0f40 con 0x5606b957f1e0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa2597fa640 1 -- 172.21.15.42:0/2908916289 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa26c012880 con 0x5606b957f1e0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 10 client.17349 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 10 client.17349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 20 client.17349 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b959d2a0 0x5606b959f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 4 client.17349 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606b94dd310 con 0x5606b959d2a0 2024-03-12T04:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.520+0000 7fa275028640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b959d2a0 0x5606b959f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.521+0000 7fa275028640 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b959d2a0 0x5606b959f6c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb6b4f0c16a6e67 server_cookie=f1c70c56bedde04a in_seq=0 out_seq=0 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.521+0000 7fa2597fa640 10 client.17349 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.521+0000 7fa2597fa640 1 -- 172.21.15.42:0/2908916289 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606b94dd310 con 0x5606b959d2a0 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.521+0000 7fa2597fa640 10 client.17349 handle_command_reply: tid=0 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.521+0000 7fa27a659740 1 client.17349 shutdown 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957f1e0 msgr2=0x5606b958c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957f1e0 0x5606b958c7d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00bf40 tx=0x7fa26c011870 comp rx=0 tx=0).stop 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 shutdown_connections 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b959d2a0 0x5606b959f6c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957fb90 0x5606b958cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 --2- 172.21.15.42:0/2908916289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957f1e0 0x5606b958c7d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa2597fa640 0 client.17349 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 >> 172.21.15.42:0/2908916289 conn(0x5606b9579840 msgr2=0x5606b957b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 shutdown_connections 2024-03-12T04:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 1 -- 172.21.15.42:0/2908916289 wait complete. 2024-03-12T04:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.522+0000 7fa27a659740 20 client.17349 trim_cache size 0 max 16384 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 Processor -- start 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 -- start start 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b958ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b958a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b9580110 con 0x5606b957fb90 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b958db20 con 0x5606b9588cf0 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa27a659740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b957e440 con 0x5606b957f1e0 2024-03-12T04:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b958a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b958a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35922/0 (socket says 172.21.15.42:35922) 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.537+0000 7fa275028640 1 -- 172.21.15.42:0/546735289 learned_addr learned my addr 172.21.15.42:0/546735289 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa275829640 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b958ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa27602a640 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa275028640 1 -- 172.21.15.42:0/546735289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 msgr2=0x5606b958ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa275028640 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b958ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa275028640 1 -- 172.21.15.42:0/546735289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 msgr2=0x5606b958d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa275028640 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa275028640 1 -- 172.21.15.42:0/546735289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606b9481c00 con 0x5606b957fb90 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa275028640 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b958a160 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa26009a4c0 tx=0x7fa2600035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fae066555947e0c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa25b7fe640 1 -- 172.21.15.42:0/546735289 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26009e700 con 0x5606b957fb90 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa25b7fe640 1 -- 172.21.15.42:0/546735289 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa26009ac30 con 0x5606b957fb90 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.538+0000 7fa25b7fe640 1 -- 172.21.15.42:0/546735289 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260096ba0 con 0x5606b957fb90 2024-03-12T04:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- 172.21.15.42:0/546735289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 msgr2=0x5606b958a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b958a160 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa26009a4c0 tx=0x7fa2600035f0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- 172.21.15.42:0/546735289 shutdown_connections 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b9588cf0 0x5606b958d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b958a160 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 --2- 172.21.15.42:0/546735289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b957f1e0 0x5606b958ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- 172.21.15.42:0/546735289 >> 172.21.15.42:0/546735289 conn(0x5606b9579840 msgr2=0x5606b958b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- 172.21.15.42:0/546735289 shutdown_connections 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- 172.21.15.42:0/546735289 wait complete. 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- start start 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 Processor -- start 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957f1e0 0x5606b9593240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b9593780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b9588cf0 0x5606b95ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b95eccd0 con 0x5606b957fb90 2024-03-12T04:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b95ece40 con 0x5606b957f1e0 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.539+0000 7fa27a659740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b95ed000 con 0x5606b9588cf0 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa27a659740 10 client.0 _ll_register_callbacks cb 0x7fa260096190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b9593780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b9593780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35928/0 (socket says 172.21.15.42:35928) 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 -- 172.21.15.42:0/906695508 learned_addr learned my addr 172.21.15.42:0/906695508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275829640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957f1e0 0x5606b9593240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa27602a640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b9588cf0 0x5606b95ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 -- 172.21.15.42:0/906695508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b9588cf0 msgr2=0x5606b95ec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b9588cf0 0x5606b95ec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 -- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957f1e0 msgr2=0x5606b9593240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957f1e0 0x5606b9593240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 -- 172.21.15.42:0/906695508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606b94a90b0 con 0x5606b957fb90 2024-03-12T04:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275829640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957f1e0 0x5606b9593240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa275028640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b9593780 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa2600a27a0 tx=0x7fa26009e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83fe81ab642571d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa2597fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa2597fa640 1 -- 172.21.15.42:0/906695508 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26009a530 con 0x5606b957fb90 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa2597fa640 1 -- 172.21.15.42:0/906695508 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa260065c00 con 0x5606b957fb90 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606b950c900 con 0x5606b957fb90 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa2597fa640 1 -- 172.21.15.42:0/906695508 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa260096ba0 con 0x5606b957fb90 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa27a659740 10 client.17355 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.540+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5606b94a7910 con 0x5606b957fb90 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa2597fa640 1 -- 172.21.15.42:0/906695508 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5606b950c900 con 0x5606b957fb90 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa2597fa640 1 -- 172.21.15.42:0/906695508 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa2600a2840 con 0x5606b957fb90 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa27a659740 10 client.17355 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa27a659740 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa27a659740 20 client.17355 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa27a659740 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b95f5620 0x5606b95f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa27a659740 4 client.17355 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5606b91e6860 con 0x5606b95f5620 2024-03-12T04:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.541+0000 7fa275829640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b95f5620 0x5606b95f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.542+0000 7fa275829640 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b95f5620 0x5606b95f7a40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f75360a42891ece2 server_cookie=ead75ab915b0a780 in_seq=0 out_seq=0 2024-03-12T04:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.542+0000 7fa2597fa640 10 client.17355 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa2597fa640 1 -- 172.21.15.42:0/906695508 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5606b91e6860 con 0x5606b95f5620 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa2597fa640 10 client.17355 handle_command_reply: tid=0 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 client.17355 shutdown 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 msgr2=0x5606b9593780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b9593780 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa2600a27a0 tx=0x7fa26009e790 comp rx=0 tx=0).stop 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 shutdown_connections 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5606b95f5620 0x5606b95f7a40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606b9588cf0 0x5606b95ec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606b957fb90 0x5606b9593780 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 --2- 172.21.15.42:0/906695508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606b957f1e0 0x5606b9593240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa2597fa640 0 client.17355 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 >> 172.21.15.42:0/906695508 conn(0x5606b9579840 msgr2=0x5606b9578050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 shutdown_connections 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 1 -- 172.21.15.42:0/906695508 wait complete. 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.543+0000 7fa27a659740 20 client.17355 trim_cache size 0 max 16384 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "bdb125bd-42e6-47e5-a969-42ea8f3ff4d3", 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:21:35.551 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa25c0a1c20 msgr2=0x7fa25c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa25c0a1c20 0x7fa25c0a40e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa268005cc0 tx=0x7fa26802e040 comp rx=0 tx=0).stop 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 msgr2=0x7fa270197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa270197b20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c970 tx=0x7fa26000ce30 comp rx=0 tx=0).stop 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 shutdown_connections 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa25c0a1c20 0x7fa25c0a40e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27019bea0 0x7fa27019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa270105350 0x7fa270197b20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 --2- 172.21.15.42:0/1719938045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa270100280 0x7fa2701975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 >> 172.21.15.42:0/1719938045 conn(0x7fa270077150 msgr2=0x7fa2700fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 shutdown_connections 2024-03-12T04:21:35.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.544+0000 7fa277a8d640 1 -- 172.21.15.42:0/1719938045 wait complete. 2024-03-12T04:21:35.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:21:35.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:21:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 -- 172.21.15.42:0/1350304335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 msgr2=0x7f75ec10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 --2- 172.21.15.42:0/1350304335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec10cf10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009cd0 tx=0x7f75e402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 -- 172.21.15.42:0/1350304335 shutdown_connections 2024-03-12T04:21:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 --2- 172.21.15.42:0/1350304335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec10cf10 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 --2- 172.21.15.42:0/1350304335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ec100c20 0x7f75ec104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 --2- 172.21.15.42:0/1350304335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec100270 0x7f75ec100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 -- 172.21.15.42:0/1350304335 >> 172.21.15.42:0/1350304335 conn(0x7f75ec077230 msgr2=0x7f75ec077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 -- 172.21.15.42:0/1350304335 shutdown_connections 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.660+0000 7f75f158a640 1 -- 172.21.15.42:0/1350304335 wait complete. 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 Processor -- start 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 -- start start 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec100270 0x7f75ec1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ec19be90 0x7f75ec19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19c880 con 0x7f75ec105340 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19c9f0 con 0x7f75ec100270 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75f158a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19cb60 con 0x7f75ec19be90 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35934/0 (socket says 172.21.15.42:35934) 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 -- 172.21.15.42:0/1342874286 learned_addr learned my addr 172.21.15.42:0/1342874286 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75eaffd640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec100270 0x7f75ec1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ec19be90 0x7f75ec19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 -- 172.21.15.42:0/1342874286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ec19be90 msgr2=0x7f75ec19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ec19be90 0x7f75ec19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 -- 172.21.15.42:0/1342874286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec100270 msgr2=0x7f75ec1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec100270 0x7f75ec1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 -- 172.21.15.42:0/1342874286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4009900 con 0x7f75ec105340 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ec19be90 0x7f75ec19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.661+0000 7f75e3fff640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec197b10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f75d400e9f0 tx=0x7f75d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.662+0000 7f75e8ff9640 1 -- 172.21.15.42:0/1342874286 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40096c0 con 0x7f75ec105340 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.662+0000 7f75e8ff9640 1 -- 172.21.15.42:0/1342874286 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75d40054d0 con 0x7f75ec105340 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.662+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec1a4140 con 0x7f75ec105340 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.662+0000 7f75e8ff9640 1 -- 172.21.15.42:0/1342874286 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4010430 con 0x7f75ec105340 2024-03-12T04:21:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.662+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1a45a0 con 0x7f75ec105340 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75e8ff9640 1 -- 172.21.15.42:0/1342874286 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f75d4005000 con 0x7f75ec105340 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 Processor -- start 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 -- start start 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f35b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f364270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f366ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f2ca280 con 0x56115f3647b0 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f2c77b0 con 0x56115f35bbd0 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.664+0000 7f75f423a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f2c2f90 con 0x56115f35b2b0 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e8ff9640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f75d80a1ba0 0x7f75d80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f366ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f75d80a1ba0 0x7f75d80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f366ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35948/0 (socket says 172.21.15.42:35948) 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 -- 172.21.15.42:0/475404080 learned_addr learned my addr 172.21.15.42:0/475404080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e8ff9640 1 -- 172.21.15.42:0/1342874286 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f75d4014070 con 0x7f75ec105340 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75eaffd640 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f364270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f35b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 -- 172.21.15.42:0/475404080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 msgr2=0x56115f35b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f35b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 -- 172.21.15.42:0/475404080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 msgr2=0x56115f364270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f364270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 -- 172.21.15.42:0/475404080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115f281570 con 0x56115f3647b0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e3fff640 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f366ba0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f75d40cb370 tx=0x7f75d406db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7373f78e3735395d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f75d80a1ba0 0x7f75d80a4060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f75e4004af0 tx=0x7f75e4032f60 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e17fa640 1 -- 172.21.15.42:0/475404080 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40046c0 con 0x56115f3647b0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e17fa640 1 -- 172.21.15.42:0/475404080 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75d409aef0 con 0x56115f3647b0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.665+0000 7f75e17fa640 1 -- 172.21.15.42:0/475404080 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4098890 con 0x56115f3647b0 2024-03-12T04:21:35.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 -- 172.21.15.42:0/475404080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 msgr2=0x56115f366ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f366ba0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f75d40cb370 tx=0x7f75d406db60 comp rx=0 tx=0).stop 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 -- 172.21.15.42:0/475404080 shutdown_connections 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f366ba0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f364270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 --2- 172.21.15.42:0/475404080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f35b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 -- 172.21.15.42:0/475404080 >> 172.21.15.42:0/475404080 conn(0x56115f355990 msgr2=0x56115f355d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 -- 172.21.15.42:0/475404080 shutdown_connections 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 -- 172.21.15.42:0/475404080 wait complete. 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.666+0000 7f75f423a740 1 -- start start 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 1 Processor -- start 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35b2b0 0x56115f361610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35bbd0 0x56115f361b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f362090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115ef96430 con 0x56115f3647b0 2024-03-12T04:21:35.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115ef52e00 con 0x56115f35b2b0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f247fd0 con 0x56115f35bbd0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75f423a740 10 client.0 _ll_register_callbacks cb 0x56115f29dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f362090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f362090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35964/0 (socket says 172.21.15.42:35964) 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 -- 172.21.15.42:0/1090065761 learned_addr learned my addr 172.21.15.42:0/1090065761 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75e3fff640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35bbd0 0x56115f361b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eaffd640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35b2b0 0x56115f361610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 -- 172.21.15.42:0/1090065761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35bbd0 msgr2=0x56115f361b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35bbd0 0x56115f361b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 -- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35b2b0 msgr2=0x56115f361610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35b2b0 0x56115f361610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 -- 172.21.15.42:0/1090065761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1a45a0 con 0x56115f3647b0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eaffd640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35b2b0 0x56115f361610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f362090 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f75e40348c0 tx=0x7f75e4044070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9e0fda73d0c4fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.667+0000 7f75d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75d27fc640 1 -- 172.21.15.42:0/1090065761 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4003b70 con 0x56115f3647b0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75d27fc640 1 -- 172.21.15.42:0/1090065761 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f75e4034e40 con 0x56115f3647b0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56115efc27e0 con 0x56115f3647b0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75d27fc640 1 -- 172.21.15.42:0/1090065761 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e4003cd0 con 0x56115f3647b0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 10 client.17367 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56115f2c1610 con 0x56115f3647b0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75d27fc640 1 -- 172.21.15.42:0/1090065761 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56115efc27e0 con 0x56115f3647b0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75d27fc640 1 -- 172.21.15.42:0/1090065761 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f75e4040550 con 0x56115f3647b0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 10 client.17367 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 20 client.17367 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3749e0 0x56115f376e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 4 client.17367 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.668+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56115efff720 con 0x56115f3749e0 2024-03-12T04:21:35.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.669+0000 7f75eaffd640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3749e0 0x56115f376e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.669+0000 7f75d27fc640 10 client.17367 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.669+0000 7f75eaffd640 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3749e0 0x56115f376e00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f54a69bffc7836 server_cookie=123ea8db5687839 in_seq=0 out_seq=0 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.669+0000 7f75d27fc640 1 -- 172.21.15.42:0/1090065761 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56115efff720 con 0x56115f3749e0 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.669+0000 7f75d27fc640 10 client.17367 handle_command_reply: tid=0 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 client.17367 shutdown 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 msgr2=0x56115f362090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f362090 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f75e40348c0 tx=0x7f75e4044070 comp rx=0 tx=0).stop 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 shutdown_connections 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3749e0 0x56115f376e00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f362090 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35bbd0 0x56115f361b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 --2- 172.21.15.42:0/1090065761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35b2b0 0x56115f361610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75d27fc640 0 client.17367 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 >> 172.21.15.42:0/1090065761 conn(0x56115f355990 msgr2=0x56115f357730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 shutdown_connections 2024-03-12T04:21:35.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 1 -- 172.21.15.42:0/1090065761 wait complete. 2024-03-12T04:21:35.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.670+0000 7f75f423a740 20 client.17367 trim_cache size 0 max 16384 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 Processor -- start 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 -- start start 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f360790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f361eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f3548d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f366fa0 con 0x56115f3647b0 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f3569e0 con 0x56115f35bbd0 2024-03-12T04:21:35.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75f423a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f35a3a0 con 0x56115f35b2b0 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75eb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f3548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75eb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f3548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35976/0 (socket says 172.21.15.42:35976) 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75eb7fe640 1 -- 172.21.15.42:0/472659559 learned_addr learned my addr 172.21.15.42:0/472659559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.685+0000 7f75eaffd640 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f360790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75e3fff640 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f361eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75eb7fe640 1 -- 172.21.15.42:0/472659559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 msgr2=0x56115f360790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f360790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75eb7fe640 1 -- 172.21.15.42:0/472659559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 msgr2=0x56115f361eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f361eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75eb7fe640 1 -- 172.21.15.42:0/472659559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115f25dac0 con 0x56115f3647b0 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75e3fff640 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f361eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:21:35.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f3548d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f75e4034890 tx=0x7f75e4032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab3d0011b6cfd1b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75e0ff9640 1 -- 172.21.15.42:0/472659559 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40329a0 con 0x56115f3647b0 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75e0ff9640 1 -- 172.21.15.42:0/472659559 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75e402cca0 con 0x56115f3647b0 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75e0ff9640 1 -- 172.21.15.42:0/472659559 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4003c60 con 0x56115f3647b0 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75f423a740 1 -- 172.21.15.42:0/472659559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 msgr2=0x56115f3548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.686+0000 7f75f423a740 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f3548d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f75e4034890 tx=0x7f75e4032e60 comp rx=0 tx=0).stop 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- 172.21.15.42:0/472659559 shutdown_connections 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f3647b0 0x56115f3548d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f361eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 --2- 172.21.15.42:0/472659559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f35b2b0 0x56115f360790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- 172.21.15.42:0/472659559 >> 172.21.15.42:0/472659559 conn(0x56115f355990 msgr2=0x56115f3541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- 172.21.15.42:0/472659559 shutdown_connections 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- 172.21.15.42:0/472659559 wait complete. 2024-03-12T04:21:35.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- start start 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 Processor -- start 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f35b2b0 0x56115f36a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f36a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f3647b0 0x56115f3caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f36b0d0 con 0x56115f35b2b0 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f3cb450 con 0x56115f35bbd0 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115f3cb590 con 0x56115f3647b0 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75f423a740 10 client.0 _ll_register_callbacks cb 0x7f75d400cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75eaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f35b2b0 0x56115f36a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75eaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f35b2b0 0x56115f36a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35988/0 (socket says 172.21.15.42:35988) 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75eaffd640 1 -- 172.21.15.42:0/3840308095 learned_addr learned my addr 172.21.15.42:0/3840308095 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.687+0000 7f75e3fff640 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f36a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75eb7fe640 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f3647b0 0x56115f3caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75eaffd640 1 -- 172.21.15.42:0/3840308095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f3647b0 msgr2=0x56115f3caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75eaffd640 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f3647b0 0x56115f3caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75eaffd640 1 -- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 msgr2=0x56115f36a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75eaffd640 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f36a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75eaffd640 1 -- 172.21.15.42:0/3840308095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56115f2c1610 con 0x56115f35b2b0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75eaffd640 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f35b2b0 0x56115f36a350 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x56115f3776d0 tx=0x7f75dc01b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154c95fad8e3230e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75d27fc640 1 -- 172.21.15.42:0/3840308095 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc016040 con 0x56115f35b2b0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75d27fc640 1 -- 172.21.15.42:0/3840308095 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f75dc005020 con 0x56115f35b2b0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75d409aef0 con 0x56115f35b2b0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75d27fc640 1 -- 172.21.15.42:0/3840308095 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75dc0051e0 con 0x56115f35b2b0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 10 client.17373 fetch_fsmap learned FSMap version 27 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56115f2836a0 con 0x56115f35b2b0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75d27fc640 1 -- 172.21.15.42:0/3840308095 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75d409aef0 con 0x56115f35b2b0 2024-03-12T04:21:35.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75d27fc640 1 -- 172.21.15.42:0/3840308095 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f75dc02c430 con 0x56115f35b2b0 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 10 client.17373 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 20 client.17373 populate_metadata read hostname 'smithi042' 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3ccc20 0x56115f3cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 4 client.17373 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.688+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56115f04e890 con 0x56115f3ccc20 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75e3fff640 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3ccc20 0x56115f3cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75e3fff640 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3ccc20 0x56115f3cf040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ddad0dd7a4423b server_cookie=c567cd25adfed851 in_seq=0 out_seq=0 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75d27fc640 10 client.17373 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75d27fc640 1 -- 172.21.15.42:0/3840308095 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56115f04e890 con 0x56115f3ccc20 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75d27fc640 10 client.17373 handle_command_reply: tid=0 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 client.17373 shutdown 2024-03-12T04:21:35.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f35b2b0 msgr2=0x56115f36a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f35b2b0 0x56115f36a350 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x56115f3776d0 tx=0x7f75dc01b9d0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 shutdown_connections 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56115f3ccc20 0x56115f3cf040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56115f3647b0 0x56115f3caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56115f35bbd0 0x56115f36a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75d27fc640 0 client.17373 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 --2- 172.21.15.42:0/3840308095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115f35b2b0 0x56115f36a350 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.689+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 >> 172.21.15.42:0/3840308095 conn(0x56115f355990 msgr2=0x56115f361f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 shutdown_connections 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f423a740 1 -- 172.21.15.42:0/3840308095 wait complete. 2024-03-12T04:21:35.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f423a740 20 client.17373 trim_cache size 0 max 16384 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bdb125bd-42e6-47e5-a969-42ea8f3ff4d3": { 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "bdb125bd-42e6-47e5-a969-42ea8f3ff4d3", 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:21:35.700 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:21:35.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f75d80a1ba0 msgr2=0x7f75d80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f75d80a1ba0 0x7f75d80a4060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f75e4004af0 tx=0x7f75e4032f60 comp rx=0 tx=0).stop 2024-03-12T04:21:35.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 msgr2=0x7f75ec197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:21:35.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec197b10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f75d400e9f0 tx=0x7f75d400edf0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 shutdown_connections 2024-03-12T04:21:35.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f75d80a1ba0 0x7f75d80a4060 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ec19be90 0x7f75ec19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec105340 0x7f75ec197b10 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 --2- 172.21.15.42:0/1342874286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec100270 0x7f75ec1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:21:35.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 >> 172.21.15.42:0/1342874286 conn(0x7f75ec077230 msgr2=0x7f75ec0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:21:35.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 shutdown_connections 2024-03-12T04:21:35.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:21:35.690+0000 7f75f158a640 1 -- 172.21.15.42:0/1342874286 wait complete. 2024-03-12T04:21:35.703 INFO:tasks.cephfs.filesystem:scrub status for tag:bdb125bd-42e6-47e5-a969-42ea8f3ff4d3 - {'path': '/', 'tag': 'bdb125bd-42e6-47e5-a969-42ea8f3ff4d3', 'options': 'recursive,force'} 2024-03-12T04:21:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:36 smithi119 ceph-mon[36119]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:36 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:21:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:36 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:36 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:21:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:36 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:36 smithi136 ceph-mon[35571]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:36 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:21:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:36 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:36 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:21:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:36 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:36.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:36 smithi042 ceph-mon[32260]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:36.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:36 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:21:36.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:36 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:36.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:36 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:21:36.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:36 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:21:38.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:38 smithi119 ceph-mon[36119]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:38.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:38 smithi136 ceph-mon[35571]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:38.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:38 smithi042 ceph-mon[32260]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:39.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:39 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:21:39.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:39 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:21:39.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:39 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:21:40.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:40 smithi119 ceph-mon[36119]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:40.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:40 smithi136 ceph-mon[35571]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:40.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:40 smithi042 ceph-mon[32260]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:42.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:42 smithi042 ceph-mon[32260]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:42.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:42 smithi119 ceph-mon[36119]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:42 smithi136 ceph-mon[35571]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:44 smithi119 ceph-mon[36119]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:44 smithi136 ceph-mon[35571]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:44 smithi042 ceph-mon[32260]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:46 smithi119 ceph-mon[36119]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:46 smithi136 ceph-mon[35571]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:46 smithi042 ceph-mon[32260]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:21:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:48 smithi119 ceph-mon[36119]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:48 smithi136 ceph-mon[35571]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:48.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:48 smithi042 ceph-mon[32260]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:21:50.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:50 smithi119 ceph-mon[36119]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:50.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:50 smithi136 ceph-mon[35571]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:50.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:50 smithi042 ceph-mon[32260]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:52.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:52 smithi119 ceph-mon[36119]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:52 smithi136 ceph-mon[35571]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:52.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:52 smithi042 ceph-mon[32260]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:54.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:54 smithi119 ceph-mon[36119]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:54 smithi136 ceph-mon[35571]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:54 smithi042 ceph-mon[32260]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:56.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:56 smithi119 ceph-mon[36119]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:56.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:56 smithi136 ceph-mon[35571]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:56.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:56 smithi042 ceph-mon[32260]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:21:58.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:21:58 smithi119 ceph-mon[36119]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:58.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:21:58 smithi136 ceph-mon[35571]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:21:58.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:21:58 smithi042 ceph-mon[32260]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:00.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:00 smithi119 ceph-mon[36119]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:00.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:00 smithi136 ceph-mon[35571]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:00.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:00 smithi042 ceph-mon[32260]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:02 smithi119 ceph-mon[36119]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:02.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:02 smithi136 ceph-mon[35571]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:02.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:02 smithi042 ceph-mon[32260]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:04.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:04 smithi119 ceph-mon[36119]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:22:04.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:04 smithi136 ceph-mon[35571]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:22:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:04 smithi042 ceph-mon[32260]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:22:05.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:22:05.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:22:05.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 -- 172.21.15.42:0/2949837970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8101d10 msgr2=0x7fe2f8105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/2949837970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8101d10 0x7fe2f8105d50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009930 tx=0x7fe2e802bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:05.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 -- 172.21.15.42:0/2949837970 shutdown_connections 2024-03-12T04:22:05.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/2949837970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f8106430 0x7fe2f810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/2949837970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8101d10 0x7fe2f8105d50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/2949837970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f8101360 0x7fe2f8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 -- 172.21.15.42:0/2949837970 >> 172.21.15.42:0/2949837970 conn(0x7fe2f80fcc20 msgr2=0x7fe2f80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 -- 172.21.15.42:0/2949837970 shutdown_connections 2024-03-12T04:22:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.822+0000 7fe2ff98c640 1 -- 172.21.15.42:0/2949837970 wait complete. 2024-03-12T04:22:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 Processor -- start 2024-03-12T04:22:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 -- start start 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f8101360 0x7fe2f8197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8106430 0x7fe2f8197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f819bfe0 0x7fe2f819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f819c9d0 con 0x7fe2f8106430 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f819cb40 con 0x7fe2f8101360 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2ff98c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f819ccb0 con 0x7fe2f819bfe0 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8106430 0x7fe2f8197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8106430 0x7fe2f8197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42060/0 (socket says 172.21.15.42:42060) 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 -- 172.21.15.42:0/930578150 learned_addr learned my addr 172.21.15.42:0/930578150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fd728640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f8101360 0x7fe2f8197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f819bfe0 0x7fe2f819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 -- 172.21.15.42:0/930578150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f819bfe0 msgr2=0x7fe2f819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f819bfe0 0x7fe2f819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 -- 172.21.15.42:0/930578150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f8101360 msgr2=0x7fe2f8197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f8101360 0x7fe2f8197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.823+0000 7fe2fcf27640 1 -- 172.21.15.42:0/930578150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009560 con 0x7fe2f8106430 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.824+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8106430 0x7fe2f8197c60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8000c00 tx=0x7fe2e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.824+0000 7fe2ee7fc640 1 -- 172.21.15.42:0/930578150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e802cb70 con 0x7fe2f8106430 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.824+0000 7fe2ee7fc640 1 -- 172.21.15.42:0/930578150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2e802de90 con 0x7fe2f8106430 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.824+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f81a4290 con 0x7fe2f8106430 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.824+0000 7fe2ee7fc640 1 -- 172.21.15.42:0/930578150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80053c0 con 0x7fe2f8106430 2024-03-12T04:22:05.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.824+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f81a4710 con 0x7fe2f8106430 2024-03-12T04:22:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe2ee7fc640 1 -- 172.21.15.42:0/930578150 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe2e80035f0 con 0x7fe2f8106430 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 Processor -- start 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 -- start start 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c42340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c51fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291bb1280 con 0x55f291c4fbc0 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291bae7b0 con 0x55f291c41f60 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe302563740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291ba9f90 con 0x55f291c42880 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe2ee7fc640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2e40a1ba0 0x7fe2e40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.827+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c51fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2e40a1ba0 0x7fe2e40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c51fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42064/0 (socket says 172.21.15.42:42064) 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 -- 172.21.15.42:0/2475712207 learned_addr learned my addr 172.21.15.42:0/2475712207 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c42340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2ee7fc640 1 -- 172.21.15.42:0/930578150 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe2e80efb60 con 0x7fe2f8106430 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fd728640 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 -- 172.21.15.42:0/2475712207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 msgr2=0x55f291c4f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 -- 172.21.15.42:0/2475712207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 msgr2=0x55f291c42340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c42340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 -- 172.21.15.42:0/2475712207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f291b68570 con 0x55f291c4fbc0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2e40a1ba0 0x7fe2e40a4060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005c40 tx=0x7fe2f0009c90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c51fb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80bc880 tx=0x7fe2e802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a933004681e4e2a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2e37fe640 1 -- 172.21.15.42:0/2475712207 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80c52e0 con 0x55f291c4fbc0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2e37fe640 1 -- 172.21.15.42:0/2475712207 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2e80bcff0 con 0x55f291c4fbc0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.828+0000 7fe2e37fe640 1 -- 172.21.15.42:0/2475712207 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80bacc0 con 0x55f291c4fbc0 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 -- 172.21.15.42:0/2475712207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 msgr2=0x55f291c51fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c51fb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80bc880 tx=0x7fe2e802bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 -- 172.21.15.42:0/2475712207 shutdown_connections 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c51fb0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 --2- 172.21.15.42:0/2475712207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c42340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 -- 172.21.15.42:0/2475712207 >> 172.21.15.42:0/2475712207 conn(0x55f291c3c990 msgr2=0x55f291c3cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 -- 172.21.15.42:0/2475712207 shutdown_connections 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 -- 172.21.15.42:0/2475712207 wait complete. 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 -- start start 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.829+0000 7fe302563740 1 Processor -- start 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe302563740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe302563740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291c4c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe302563740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c4fbc0 0x55f291c4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe302563740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29187d430 con 0x55f291c42880 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe302563740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291839e00 con 0x55f291c41f60 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe302563740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291b2efd0 con 0x55f291c4fbc0 2024-03-12T04:22:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe302563740 10 client.0 _ll_register_callbacks cb 0x55f291b84fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291c4c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291c4c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42074/0 (socket says 172.21.15.42:42074) 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 -- 172.21.15.42:0/3313942103 learned_addr learned my addr 172.21.15.42:0/3313942103 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c4fbc0 0x55f291c4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fd728640 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 -- 172.21.15.42:0/3313942103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c4fbc0 msgr2=0x55f291c4ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c4fbc0 0x55f291c4ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 -- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 msgr2=0x55f291c4f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.830+0000 7fe2fcf27640 1 -- 172.21.15.42:0/3313942103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f81a4710 con 0x55f291c42880 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291c4c8f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80b8d50 tx=0x7fe2e80c5090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae2e2d87fac11f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe2e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/3313942103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80c0a40 con 0x55f291c42880 2024-03-12T04:22:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/3313942103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe2e80bc8f0 con 0x55f291c42880 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2918a97e0 con 0x55f291c42880 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/3313942103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2e80b7110 con 0x55f291c42880 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 10 client.17385 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f291ba8610 con 0x55f291c42880 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/3313942103 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f2918a97e0 con 0x55f291c42880 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/3313942103 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2e803c070 con 0x55f291c42880 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 10 client.17385 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 20 client.17385 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291c5f750 0x55f291c61b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 4 client.17385 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.831+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2918e6720 con 0x55f291c5f750 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.832+0000 7fe2fd728640 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291c5f750 0x55f291c61b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.832+0000 7fe2e0ff9640 10 client.17385 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:05.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.832+0000 7fe2fd728640 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291c5f750 0x55f291c61b40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed228a7f38f206c server_cookie=816015dc90f8777d in_seq=0 out_seq=0 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.832+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/3313942103 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2918e6720 con 0x55f291c5f750 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.832+0000 7fe2e0ff9640 10 client.17385 handle_command_reply: tid=0 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 client.17385 shutdown 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 msgr2=0x55f291c4c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291c4c8f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80b8d50 tx=0x7fe2e80c5090 comp rx=0 tx=0).stop 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 shutdown_connections 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291c5f750 0x55f291c61b40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c4fbc0 0x55f291c4ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291c4c8f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe2e0ff9640 0 client.17385 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 --2- 172.21.15.42:0/3313942103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 >> 172.21.15.42:0/3313942103 conn(0x55f291c3c990 msgr2=0x55f291c3e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 shutdown_connections 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 1 -- 172.21.15.42:0/3313942103 wait complete. 2024-03-12T04:22:05.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.833+0000 7fe302563740 20 client.17385 trim_cache size 0 max 16384 2024-03-12T04:22:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 Processor -- start 2024-03-12T04:22:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 -- start start 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c4f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291c3db10 con 0x55f291c4fbc0 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291c42fa0 con 0x55f291c41f60 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe302563740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291c52370 con 0x55f291c42880 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe2fdf29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c4f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe2fdf29640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c4f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42090/0 (socket says 172.21.15.42:42090) 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe2fdf29640 1 -- 172.21.15.42:0/1920980994 learned_addr learned my addr 172.21.15.42:0/1920980994 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.848+0000 7fe2fd728640 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fdf29640 1 -- 172.21.15.42:0/1920980994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 msgr2=0x55f291c4f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fdf29640 1 -- 172.21.15.42:0/1920980994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 msgr2=0x55f291c4c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fdf29640 1 -- 172.21.15.42:0/1920980994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f291b44ac0 con 0x55f291c4fbc0 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fd728640 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c4f560 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0012620 tx=0x7fe2f001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef63055cde8590c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2e2ffd640 1 -- 172.21.15.42:0/1920980994 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f002c480 con 0x55f291c4fbc0 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2e2ffd640 1 -- 172.21.15.42:0/1920980994 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2f0025070 con 0x55f291c4fbc0 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.849+0000 7fe2e2ffd640 1 -- 172.21.15.42:0/1920980994 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00218a0 con 0x55f291c4fbc0 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- 172.21.15.42:0/1920980994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 msgr2=0x55f291c4f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c4f560 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0012620 tx=0x7fe2f001f000 comp rx=0 tx=0).stop 2024-03-12T04:22:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- 172.21.15.42:0/1920980994 shutdown_connections 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c4fbc0 0x55f291c4f560 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c42880 0x55f291c4f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 --2- 172.21.15.42:0/1920980994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c41f60 0x55f291c4c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- 172.21.15.42:0/1920980994 >> 172.21.15.42:0/1920980994 conn(0x55f291c3c990 msgr2=0x55f291c4c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- 172.21.15.42:0/1920980994 shutdown_connections 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- 172.21.15.42:0/1920980994 wait complete. 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- start start 2024-03-12T04:22:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 Processor -- start 2024-03-12T04:22:05.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c41f60 0x55f291cb0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291cb0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c4fbc0 0x55f291c55cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291c56290 con 0x55f291c42880 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291c56400 con 0x55f291c4fbc0 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f291c565c0 con 0x55f291c41f60 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.850+0000 7fe302563740 10 client.0 _ll_register_callbacks cb 0x7fe2e8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291cb0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291cb0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42104/0 (socket says 172.21.15.42:42104) 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 -- 172.21.15.42:0/742711460 learned_addr learned my addr 172.21.15.42:0/742711460 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fdf29640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c4fbc0 0x55f291c55cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fd728640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c41f60 0x55f291cb0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 -- 172.21.15.42:0/742711460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c41f60 msgr2=0x55f291cb0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c41f60 0x55f291cb0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 -- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c4fbc0 msgr2=0x55f291c55cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c4fbc0 0x55f291c55cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 -- 172.21.15.42:0/742711460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f001a040 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fd728640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c41f60 0x55f291cb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2fcf27640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291cb0640 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80ed900 tx=0x7fe2e80bc250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f9140b714202f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/742711460 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80c5100 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/742711460 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe2e80ed2c0 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2e80bcff0 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/742711460 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2e80bc6c0 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe302563740 10 client.17391 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.851+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f291b6a6a0 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/742711460 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2e80bcff0 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/742711460 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2e803c070 con 0x55f291c42880 2024-03-12T04:22:05.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe302563740 10 client.17391 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe302563740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe302563740 20 client.17391 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe302563740 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291cb7b10 0x55f291cb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe302563740 4 client.17391 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f291935890 con 0x55f291cb7b10 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe2fd728640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291cb7b10 0x55f291cb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe2fd728640 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291cb7b10 0x55f291cb9f00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca25f19fd9a7c99 server_cookie=fe06c154e2e63846 in_seq=0 out_seq=0 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.852+0000 7fe2e0ff9640 10 client.17391 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2e0ff9640 1 -- 172.21.15.42:0/742711460 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f291935890 con 0x55f291cb7b10 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2e0ff9640 10 client.17391 handle_command_reply: tid=0 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 client.17391 shutdown 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 msgr2=0x55f291cb0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291cb0640 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80ed900 tx=0x7fe2e80bc250 comp rx=0 tx=0).stop 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 shutdown_connections 2024-03-12T04:22:05.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55f291cb7b10 0x55f291cb9f00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f291c4fbc0 0x55f291c55cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2e0ff9640 0 client.17391 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f291c42880 0x55f291cb0640 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 --2- 172.21.15.42:0/742711460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f291c41f60 0x55f291cb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 >> 172.21.15.42:0/742711460 conn(0x55f291c3c990 msgr2=0x55f291c4b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 shutdown_connections 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 1 -- 172.21.15.42:0/742711460 wait complete. 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe302563740 20 client.17391 trim_cache size 0 max 16384 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:22:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:22:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:22:05.864 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:22:05.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2e40a1ba0 msgr2=0x7fe2e40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2e40a1ba0 0x7fe2e40a4060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005c40 tx=0x7fe2f0009c90 comp rx=0 tx=0).stop 2024-03-12T04:22:05.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8106430 msgr2=0x7fe2f8197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8106430 0x7fe2f8197c60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8000c00 tx=0x7fe2e8004270 comp rx=0 tx=0).stop 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 shutdown_connections 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2e40a1ba0 0x7fe2e40a4060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f819bfe0 0x7fe2f819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2f8106430 0x7fe2f8197c60 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 --2- 172.21.15.42:0/930578150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f8101360 0x7fe2f8197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.853+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 >> 172.21.15.42:0/930578150 conn(0x7fe2f80fcc20 msgr2=0x7fe2f80fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.854+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 shutdown_connections 2024-03-12T04:22:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.854+0000 7fe2ff98c640 1 -- 172.21.15.42:0/930578150 wait complete. 2024-03-12T04:22:05.867 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:22:05.867 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:22:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 -- 172.21.15.42:0/3305746849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 msgr2=0x7f30b8100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/3305746849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8100690 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009930 tx=0x7f30a802bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 -- 172.21.15.42:0/3305746849 shutdown_connections 2024-03-12T04:22:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/3305746849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30b8105380 0x7f30b810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/3305746849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b8100c60 0x7f30b8104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/3305746849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8100690 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 -- 172.21.15.42:0/3305746849 >> 172.21.15.42:0/3305746849 conn(0x7f30b8077120 msgr2=0x7f30b8077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 -- 172.21.15.42:0/3305746849 shutdown_connections 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 -- 172.21.15.42:0/3305746849 wait complete. 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 Processor -- start 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.968+0000 7f30bd1bc640 1 -- start start 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b8105380 0x7f30b8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30b819bfd0 0x7f30b819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b819c9c0 con 0x7f30b81002b0 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b819cb30 con 0x7f30b819bfd0 2024-03-12T04:22:05.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b819cca0 con 0x7f30b8105380 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42120/0 (socket says 172.21.15.42:42120) 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 -- 172.21.15.42:0/1336169293 learned_addr learned my addr 172.21.15.42:0/1336169293 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30affff640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b8105380 0x7f30b8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b759e640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30b819bfd0 0x7f30b819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 -- 172.21.15.42:0/1336169293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b8105380 msgr2=0x7f30b8197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b8105380 0x7f30b8197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 -- 172.21.15.42:0/1336169293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30b819bfd0 msgr2=0x7f30b819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30b819bfd0 0x7f30b819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 -- 172.21.15.42:0/1336169293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a8009560 con 0x7f30b81002b0 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30affff640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b8105380 0x7f30b8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8197710 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009230 tx=0x7f30a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b4d99640 1 -- 172.21.15.42:0/1336169293 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8004440 con 0x7f30b81002b0 2024-03-12T04:22:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b81a4280 con 0x7f30b81002b0 2024-03-12T04:22:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b4d99640 1 -- 172.21.15.42:0/1336169293 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30a802de90 con 0x7f30b81002b0 2024-03-12T04:22:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30b4d99640 1 -- 172.21.15.42:0/1336169293 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80052e0 con 0x7f30b81002b0 2024-03-12T04:22:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.969+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b81a4730 con 0x7f30b81002b0 2024-03-12T04:22:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.970+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b80683c0 con 0x7f30b81002b0 2024-03-12T04:22:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.971+0000 7f30b4d99640 1 -- 172.21.15.42:0/1336169293 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f30a8003cf0 con 0x7f30b81002b0 2024-03-12T04:22:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.972+0000 7f30b4d99640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f30a40a1ba0 0x7f30a40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.972+0000 7f30affff640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f30a40a1ba0 0x7f30a40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.972+0000 7f30b4d99640 1 -- 172.21.15.42:0/1336169293 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f30a80ef7b0 con 0x7f30b81002b0 2024-03-12T04:22:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.972+0000 7f30affff640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f30a40a1ba0 0x7f30a40a4060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30a00058c0 tx=0x7f30a0005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:05.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:05.976+0000 7f30b4d99640 1 -- 172.21.15.42:0/1336169293 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30a80b87e0 con 0x7f30b81002b0 2024-03-12T04:22:06.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.197+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30b81085b0 con 0x7f30b81002b0 2024-03-12T04:22:06.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.197+0000 7f30b4d99640 1 -- 172.21.15.42:0/1336169293 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f30a80bbea0 con 0x7f30b81002b0 2024-03-12T04:22:06.200 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:22:06.200 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:22:06.200 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:22:06.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f30a40a1ba0 msgr2=0x7f30a40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f30a40a1ba0 0x7f30a40a4060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30a00058c0 tx=0x7f30a0005850 comp rx=0 tx=0).stop 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 msgr2=0x7f30b8197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8197710 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009230 tx=0x7f30a8004040 comp rx=0 tx=0).stop 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 shutdown_connections 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f30a40a1ba0 0x7f30a40a4060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30b819bfd0 0x7f30b819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b8105380 0x7f30b8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 --2- 172.21.15.42:0/1336169293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b81002b0 0x7f30b8197710 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.199+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 >> 172.21.15.42:0/1336169293 conn(0x7f30b8077120 msgr2=0x7f30b80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.200+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 shutdown_connections 2024-03-12T04:22:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.200+0000 7f30bd1bc640 1 -- 172.21.15.42:0/1336169293 wait complete. 2024-03-12T04:22:06.213 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:06.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:22:06.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 -- 172.21.15.42:0/261165049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0068f80 msgr2=0x7fd2b0105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 --2- 172.21.15.42:0/261165049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0068f80 0x7fd2b0105d50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009930 tx=0x7fd2a002bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 -- 172.21.15.42:0/261165049 shutdown_connections 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 --2- 172.21.15.42:0/261165049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2b0106430 0x7fd2b010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 --2- 172.21.15.42:0/261165049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0068f80 0x7fd2b0105d50 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 --2- 172.21.15.42:0/261165049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2b00685d0 0x7fd2b00689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 -- 172.21.15.42:0/261165049 >> 172.21.15.42:0/261165049 conn(0x7fd2b0077180 msgr2=0x7fd2b0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 -- 172.21.15.42:0/261165049 shutdown_connections 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 -- 172.21.15.42:0/261165049 wait complete. 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 Processor -- start 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.316+0000 7fd2b824e640 1 -- start start 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2b00685d0 0x7fd2b019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0106430 0x7fd2b019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2b01975e0 0x7fd2b0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0197fd0 con 0x7fd2b0106430 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0198140 con 0x7fd2b00685d0 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0198300 con 0x7fd2b01975e0 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0106430 0x7fd2b019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0106430 0x7fd2b019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42130/0 (socket says 172.21.15.42:42130) 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 -- 172.21.15.42:0/992340841 learned_addr learned my addr 172.21.15.42:0/992340841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2b00685d0 0x7fd2b019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 -- 172.21.15.42:0/992340841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2b01975e0 msgr2=0x7fd2b0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2b01975e0 0x7fd2b0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 -- 172.21.15.42:0/992340841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2b00685d0 msgr2=0x7fd2b019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2b00685d0 0x7fd2b019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 -- 172.21.15.42:0/992340841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009560 con 0x7fd2b0106430 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0106430 0x7fd2b019d920 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000c00 tx=0x7fd2a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2a6ffd640 1 -- 172.21.15.42:0/992340841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a002cb70 con 0x7fd2b0106430 2024-03-12T04:22:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2a6ffd640 1 -- 172.21.15.42:0/992340841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2a002de90 con 0x7fd2b0106430 2024-03-12T04:22:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b01a42a0 con 0x7fd2b0106430 2024-03-12T04:22:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2a6ffd640 1 -- 172.21.15.42:0/992340841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00053c0 con 0x7fd2b0106430 2024-03-12T04:22:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.317+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b01a4750 con 0x7fd2b0106430 2024-03-12T04:22:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.319+0000 7fd2a6ffd640 1 -- 172.21.15.42:0/992340841 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd2a0005520 con 0x7fd2b0106430 2024-03-12T04:22:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 Processor -- start 2024-03-12T04:22:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 -- start start 2024-03-12T04:22:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2a6ffd640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd29c0a1ba0 0x7fd29c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41c4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4eb70 0x55dc41c57210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41c09960 con 0x55dc41c57750 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41bbd310 con 0x55dc41c4eb70 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2bae27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41bba840 con 0x55dc41c4e250 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd29c0a1ba0 0x7fd29c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2a6ffd640 1 -- 172.21.15.42:0/992340841 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd2a00ef910 con 0x7fd2b0106430 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.320+0000 7fd2b67eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c59b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42134/0 (socket says 172.21.15.42:42134) 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 -- 172.21.15.42:0/1656375207 learned_addr learned my addr 172.21.15.42:0/1656375207 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4eb70 0x55dc41c57210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41c4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 -- 172.21.15.42:0/1656375207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 msgr2=0x55dc41c4e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41c4e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 -- 172.21.15.42:0/1656375207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4eb70 msgr2=0x55dc41c57210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4eb70 0x55dc41c57210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 -- 172.21.15.42:0/1656375207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc41b78060 con 0x55dc41c57750 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd29c0a1ba0 0x7fd29c0a4060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80139b0 tx=0x7fd2a8009b50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c59b40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8011b40 tx=0x7fd2a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f50eb8356171a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd29bfff640 1 -- 172.21.15.42:0/1656375207 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8018070 con 0x55dc41c57750 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd29bfff640 1 -- 172.21.15.42:0/1656375207 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2a800eba0 con 0x55dc41c57750 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.321+0000 7fd29bfff640 1 -- 172.21.15.42:0/1656375207 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801e9e0 con 0x55dc41c57750 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 -- 172.21.15.42:0/1656375207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 msgr2=0x55dc41c59b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c59b40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8011b40 tx=0x7fd2a8018000 comp rx=0 tx=0).stop 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 -- 172.21.15.42:0/1656375207 shutdown_connections 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c59b40 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4eb70 0x55dc41c57210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1656375207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41c4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 -- 172.21.15.42:0/1656375207 >> 172.21.15.42:0/1656375207 conn(0x55dc41c48980 msgr2=0x55dc41c48d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 -- 172.21.15.42:0/1656375207 shutdown_connections 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 -- 172.21.15.42:0/1656375207 wait complete. 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 -- start start 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.322+0000 7fd2bae27740 1 Processor -- start 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4e250 0x55dc41c54730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2bae27740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41bb6020 con 0x55dc41c57750 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2bae27740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41889430 con 0x55dc41c4e250 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2bae27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41845e00 con 0x55dc41c4eb70 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2bae27740 10 client.0 _ll_register_callbacks cb 0x55dc41b5dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42148/0 (socket says 172.21.15.42:42148) 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 -- 172.21.15.42:0/3065464024 learned_addr learned my addr 172.21.15.42:0/3065464024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4e250 0x55dc41c54730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 -- 172.21.15.42:0/3065464024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 msgr2=0x55dc41c54c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 -- 172.21.15.42:0/3065464024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4e250 msgr2=0x55dc41c54730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4e250 0x55dc41c54730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 -- 172.21.15.42:0/3065464024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a8011900 con 0x55dc41c57750 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4e250 0x55dc41c54730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c551b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009970 tx=0x7fd2a80099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0a5443041c5b6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.323+0000 7fd2997fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2997fa640 1 -- 172.21.15.42:0/3065464024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8003970 con 0x55dc41c57750 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2997fa640 1 -- 172.21.15.42:0/3065464024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd2a8003ad0 con 0x55dc41c57750 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc418b57e0 con 0x55dc41c57750 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2997fa640 1 -- 172.21.15.42:0/3065464024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a8003c90 con 0x55dc41c57750 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 10 client.17409 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc41bb46a0 con 0x55dc41c57750 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2997fa640 1 -- 172.21.15.42:0/3065464024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55dc418b57e0 con 0x55dc41c57750 2024-03-12T04:22:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2997fa640 1 -- 172.21.15.42:0/3065464024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2a8004160 con 0x55dc41c57750 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 10 client.17409 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 10 client.17409 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 20 client.17409 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41c67a40 0x55dc41c69e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 4 client.17409 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc418f2720 con 0x55dc41c67a40 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.324+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41c67a40 0x55dc41c69e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41c67a40 0x55dc41c69e30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6749c8a91d042e0b server_cookie=10f0df1e27cd2bb3 in_seq=0 out_seq=0 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2997fa640 10 client.17409 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2997fa640 1 -- 172.21.15.42:0/3065464024 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc418f2720 con 0x55dc41c67a40 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2997fa640 10 client.17409 handle_command_reply: tid=0 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 client.17409 shutdown 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 msgr2=0x55dc41c551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c551b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009970 tx=0x7fd2a80099a0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 shutdown_connections 2024-03-12T04:22:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41c67a40 0x55dc41c69e30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c57750 0x55dc41c551b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2997fa640 0 client.17409 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 --2- 172.21.15.42:0/3065464024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c4e250 0x55dc41c54730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 >> 172.21.15.42:0/3065464024 conn(0x55dc41c48980 msgr2=0x55dc41c4a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 shutdown_connections 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.325+0000 7fd2bae27740 1 -- 172.21.15.42:0/3065464024 wait complete. 2024-03-12T04:22:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.326+0000 7fd2bae27740 20 client.17409 trim_cache size 0 max 16384 2024-03-12T04:22:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 Processor -- start 2024-03-12T04:22:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 -- start start 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4e250 0x55dc41c56e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41b3b130 con 0x55dc41c4e250 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41c67840 con 0x55dc41c57750 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2bae27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41c4deb0 con 0x55dc41c4eb70 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2b5fea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4e250 0x55dc41c56e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2b5fea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4e250 0x55dc41c56e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42150/0 (socket says 172.21.15.42:42150) 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.337+0000 7fd2b5fea640 1 -- 172.21.15.42:0/569055473 learned_addr learned my addr 172.21.15.42:0/569055473 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b5fea640 1 -- 172.21.15.42:0/569055473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 msgr2=0x55dc41c54fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b5fea640 1 -- 172.21.15.42:0/569055473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 msgr2=0x55dc41c519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b5fea640 1 -- 172.21.15.42:0/569055473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc41b50c20 con 0x55dc41c4e250 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4e250 0x55dc41c56e60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x55dc41c6a730 tx=0x7fd2ac00b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cb068c850a86a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd29b7fe640 1 -- 172.21.15.42:0/569055473 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac018040 con 0x55dc41c4e250 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd29b7fe640 1 -- 172.21.15.42:0/569055473 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2ac0142d0 con 0x55dc41c4e250 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.338+0000 7fd29b7fe640 1 -- 172.21.15.42:0/569055473 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0126b0 con 0x55dc41c4e250 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- 172.21.15.42:0/569055473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4e250 msgr2=0x55dc41c56e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4e250 0x55dc41c56e60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x55dc41c6a730 tx=0x7fd2ac00b4d0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- 172.21.15.42:0/569055473 shutdown_connections 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4eb70 0x55dc41c54fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 --2- 172.21.15.42:0/569055473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4e250 0x55dc41c56e60 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- 172.21.15.42:0/569055473 >> 172.21.15.42:0/569055473 conn(0x55dc41c48980 msgr2=0x55dc41c4fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- 172.21.15.42:0/569055473 shutdown_connections 2024-03-12T04:22:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- 172.21.15.42:0/569055473 wait complete. 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- start start 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 Processor -- start 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41cb8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4eb70 0x55dc41cb8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41c5ddb0 con 0x55dc41c4eb70 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41c5df20 con 0x55dc41c57750 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41c5e0e0 con 0x55dc41c4e250 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.339+0000 7fd2bae27740 10 client.0 _ll_register_callbacks cb 0x7fd2a80134c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4eb70 0x55dc41cb8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4eb70 0x55dc41cb8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42154/0 (socket says 172.21.15.42:42154) 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 -- 172.21.15.42:0/1623918685 learned_addr learned my addr 172.21.15.42:0/1623918685 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41cb8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 -- 172.21.15.42:0/1623918685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 msgr2=0x55dc41cb8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41cb8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 -- 172.21.15.42:0/1623918685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 msgr2=0x55dc41c5d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c5d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 -- 172.21.15.42:0/1623918685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc41b768c0 con 0x55dc41c4eb70 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41cb8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b67eb640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c5d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2b57e9640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4eb70 0x55dc41cb8760 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00bc3a0 tx=0x7fd2a00bc0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55828b838521fcd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2997fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2997fa640 1 -- 172.21.15.42:0/1623918685 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00c07f0 con 0x55dc41c4eb70 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2997fa640 1 -- 172.21.15.42:0/1623918685 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd2a00c4e40 con 0x55dc41c4eb70 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a800eba0 con 0x55dc41c4eb70 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2997fa640 1 -- 172.21.15.42:0/1623918685 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a00a75a0 con 0x55dc41c4eb70 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2bae27740 10 client.17415 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc41b5d050 con 0x55dc41c4eb70 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.340+0000 7fd2997fa640 1 -- 172.21.15.42:0/1623918685 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2a800eba0 con 0x55dc41c4eb70 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2997fa640 1 -- 172.21.15.42:0/1623918685 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2a003c070 con 0x55dc41c4eb70 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 10 client.17415 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 10 client.17415 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 20 client.17415 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41cbffd0 0x55dc41cc23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 4 client.17415 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc41941890 con 0x55dc41cbffd0 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41cbffd0 0x55dc41cc23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2997fa640 10 client.17415 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2b5fea640 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41cbffd0 0x55dc41cc23f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9cce7471f9466b57 server_cookie=edb015694fdef2f1 in_seq=0 out_seq=0 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2997fa640 1 -- 172.21.15.42:0/1623918685 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc41941890 con 0x55dc41cbffd0 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2997fa640 10 client.17415 handle_command_reply: tid=0 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 1 client.17415 shutdown 2024-03-12T04:22:06.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4eb70 msgr2=0x55dc41cb8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.341+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4eb70 0x55dc41cb8760 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00bc3a0 tx=0x7fd2a00bc0c0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 shutdown_connections 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc41c57750 0x55dc41c5d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55dc41cbffd0 0x55dc41cc23f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc41c4eb70 0x55dc41cb8760 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 --2- 172.21.15.42:0/1623918685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc41c4e250 0x55dc41cb8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2997fa640 0 client.17415 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 >> 172.21.15.42:0/1623918685 conn(0x55dc41c48980 msgr2=0x55dc41c49d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 shutdown_connections 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 1 -- 172.21.15.42:0/1623918685 wait complete. 2024-03-12T04:22:06.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2bae27740 20 client.17415 trim_cache size 0 max 16384 2024-03-12T04:22:06.352 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:06.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd29c0a1ba0 msgr2=0x7fd29c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd29c0a1ba0 0x7fd29c0a4060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80139b0 tx=0x7fd2a8009b50 comp rx=0 tx=0).stop 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0106430 msgr2=0x7fd2b019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0106430 0x7fd2b019d920 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000c00 tx=0x7fd2a0004270 comp rx=0 tx=0).stop 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 shutdown_connections 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd29c0a1ba0 0x7fd29c0a4060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2b01975e0 0x7fd2b0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2b0106430 0x7fd2b019d920 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 --2- 172.21.15.42:0/992340841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2b00685d0 0x7fd2b019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 >> 172.21.15.42:0/992340841 conn(0x7fd2b0077180 msgr2=0x7fd2b00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 shutdown_connections 2024-03-12T04:22:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.342+0000 7fd2b824e640 1 -- 172.21.15.42:0/992340841 wait complete. 2024-03-12T04:22:06.355 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:06.356 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 -- 172.21.15.42:0/1849871985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 msgr2=0x7f048c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 --2- 172.21.15.42:0/1849871985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c105ba0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f047c009930 tx=0x7f047c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 -- 172.21.15.42:0/1849871985 shutdown_connections 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 --2- 172.21.15.42:0/1849871985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f048c100170 0x7f048c10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 --2- 172.21.15.42:0/1849871985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f048c0ff7b0 0x7f048c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 --2- 172.21.15.42:0/1849871985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c105ba0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 -- 172.21.15.42:0/1849871985 >> 172.21.15.42:0/1849871985 conn(0x7f048c076640 msgr2=0x7f048c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 -- 172.21.15.42:0/1849871985 shutdown_connections 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 -- 172.21.15.42:0/1849871985 wait complete. 2024-03-12T04:22:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 Processor -- start 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.459+0000 7f0492688640 1 -- start start 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0492688640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f048c0ff7b0 0x7f048c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0492688640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f048c100170 0x7f048c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0492688640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c1a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0492688640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1992b0 con 0x7f048c1057c0 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0492688640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1a3ad0 con 0x7f048c100170 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0492688640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1a3c10 con 0x7f048c0ff7b0 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c1a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c1a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42178/0 (socket says 172.21.15.42:42178) 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 -- 172.21.15.42:0/1432406303 learned_addr learned my addr 172.21.15.42:0/1432406303 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f048b7fe640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f048c100170 0x7f048c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f048bfff640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f048c0ff7b0 0x7f048c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 -- 172.21.15.42:0/1432406303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f048c0ff7b0 msgr2=0x7f048c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f048c0ff7b0 0x7f048c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 -- 172.21.15.42:0/1432406303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f048c100170 msgr2=0x7f048c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f048c100170 0x7f048c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 -- 172.21.15.42:0/1432406303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c009560 con 0x7f048c1057c0 2024-03-12T04:22:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.460+0000 7f0490c25640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c1a3590 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f048400eaa0 tx=0x7f048400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.461+0000 7f04897fa640 1 -- 172.21.15.42:0/1432406303 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400cdb0 con 0x7f048c1057c0 2024-03-12T04:22:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.461+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c1a3e70 con 0x7f048c1057c0 2024-03-12T04:22:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.461+0000 7f04897fa640 1 -- 172.21.15.42:0/1432406303 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0484005020 con 0x7f048c1057c0 2024-03-12T04:22:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.461+0000 7f04897fa640 1 -- 172.21.15.42:0/1432406303 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840116b0 con 0x7f048c1057c0 2024-03-12T04:22:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.461+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c1a4350 con 0x7f048c1057c0 2024-03-12T04:22:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.463+0000 7f04897fa640 1 -- 172.21.15.42:0/1432406303 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0484005190 con 0x7f048c1057c0 2024-03-12T04:22:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.463+0000 7f0495268740 1 Processor -- start 2024-03-12T04:22:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0495268740 1 -- start start 2024-03-12T04:22:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0495268740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f30630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0495268740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30b70 0x5614e3f39210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0495268740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f3bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0495268740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3eeb960 con 0x5614e3f30250 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0495268740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3e9f310 con 0x5614e3f30b70 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0495268740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3e9c840 con 0x5614e3f39750 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f30630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f30630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42188/0 (socket says 172.21.15.42:42188) 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 -- 172.21.15.42:0/112651419 learned_addr learned my addr 172.21.15.42:0/112651419 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f04897fa640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f04740a1ba0 0x7f04740a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f0490c25640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f04740a1ba0 0x7f04740a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048bfff640 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30b70 0x5614e3f39210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f3bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 -- 172.21.15.42:0/112651419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 msgr2=0x5614e3f3bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f3bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 -- 172.21.15.42:0/112651419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30b70 msgr2=0x5614e3f39210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30b70 0x5614e3f39210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 -- 172.21.15.42:0/112651419 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e3e5a060 con 0x5614e3f30250 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f04897fa640 1 -- 172.21.15.42:0/1432406303 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f04840cd9a0 con 0x7f048c1057c0 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.464+0000 7f048b7fe640 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f30630 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f04800145b0 tx=0x7f048000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2432483cb1807f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f04727fc640 1 -- 172.21.15.42:0/112651419 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480004030 con 0x5614e3f30250 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0490c25640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f04740a1ba0 0x7f04740a4060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04840a2830 tx=0x7f0484085040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f04727fc640 1 -- 172.21.15.42:0/112651419 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0480005020 con 0x5614e3f30250 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f04727fc640 1 -- 172.21.15.42:0/112651419 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480004a70 con 0x5614e3f30250 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 -- 172.21.15.42:0/112651419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 msgr2=0x5614e3f30630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f30630 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f04800145b0 tx=0x7f048000b960 comp rx=0 tx=0).stop 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 -- 172.21.15.42:0/112651419 shutdown_connections 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f3bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30b70 0x5614e3f39210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 --2- 172.21.15.42:0/112651419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f30630 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 -- 172.21.15.42:0/112651419 >> 172.21.15.42:0/112651419 conn(0x5614e3f2a980 msgr2=0x5614e3f2ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 -- 172.21.15.42:0/112651419 shutdown_connections 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.465+0000 7f0495268740 1 -- 172.21.15.42:0/112651419 wait complete. 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 -- start start 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 Processor -- start 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f36780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f36cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f37200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3e98020 con 0x5614e3f30b70 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3b6b430 con 0x5614e3f30250 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3b27e00 con 0x5614e3f39750 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f0495268740 10 client.0 _ll_register_callbacks cb 0x5614e3e3fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f36cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f36cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42200/0 (socket says 172.21.15.42:42200) 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 -- 172.21.15.42:0/2514162571 learned_addr learned my addr 172.21.15.42:0/2514162571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f36780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f37200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 -- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 msgr2=0x5614e3f37200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f37200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 -- 172.21.15.42:0/2514162571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 msgr2=0x5614e3f36780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f36780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048bfff640 1 -- 172.21.15.42:0/2514162571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048000be00 con 0x5614e3f30b70 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.466+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f36780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f048bfff640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f36cc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f047c000c00 tx=0x7f047c02ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d673e30fcab43f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f046b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f046b7fe640 1 -- 172.21.15.42:0/2514162571 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c02cbf0 con 0x5614e3f30b70 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614e3b977e0 con 0x5614e3f30b70 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f046b7fe640 1 -- 172.21.15.42:0/2514162571 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f047c004460 con 0x5614e3f30b70 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f046b7fe640 1 -- 172.21.15.42:0/2514162571 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f047c005d50 con 0x5614e3f30b70 2024-03-12T04:22:06.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 10 client.17427 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614e3e966a0 con 0x5614e3f30b70 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f046b7fe640 1 -- 172.21.15.42:0/2514162571 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5614e3b977e0 con 0x5614e3f30b70 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f046b7fe640 1 -- 172.21.15.42:0/2514162571 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f047c005130 con 0x5614e3f30b70 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 10 client.17427 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 10 client.17427 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 20 client.17427 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3f49aa0 0x5614e3f4bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 4 client.17427 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.467+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614e3bd4720 con 0x5614e3f49aa0 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.468+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3f49aa0 0x5614e3f4bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.468+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3f49aa0 0x5614e3f4bec0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=903aeb3b5769d130 server_cookie=9c1b8cd03907d171 in_seq=0 out_seq=0 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.468+0000 7f046b7fe640 10 client.17427 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f046b7fe640 1 -- 172.21.15.42:0/2514162571 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614e3bd4720 con 0x5614e3f49aa0 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f046b7fe640 10 client.17427 handle_command_reply: tid=0 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 client.17427 shutdown 2024-03-12T04:22:06.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 msgr2=0x5614e3f36cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f36cc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f047c000c00 tx=0x7f047c02ca00 comp rx=0 tx=0).stop 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 shutdown_connections 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3f49aa0 0x5614e3f4bec0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f37200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f36cc0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f046b7fe640 0 client.17427 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 --2- 172.21.15.42:0/2514162571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f36780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 >> 172.21.15.42:0/2514162571 conn(0x5614e3f2a980 msgr2=0x5614e3f2c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 shutdown_connections 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 1 -- 172.21.15.42:0/2514162571 wait complete. 2024-03-12T04:22:06.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.469+0000 7f0495268740 20 client.17427 trim_cache size 0 max 16384 2024-03-12T04:22:06.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 Processor -- start 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 -- start start 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f38eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f30b70 0x5614e3f37020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f39750 0x5614e3f35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3e1d130 con 0x5614e3f30250 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3f262b0 con 0x5614e3f39750 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f0495268740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3f2feb0 con 0x5614e3f30b70 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f38eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f38eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42208/0 (socket says 172.21.15.42:42208) 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 -- 172.21.15.42:0/2099199996 learned_addr learned my addr 172.21.15.42:0/2099199996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048bfff640 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f30b70 0x5614e3f37020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 -- 172.21.15.42:0/2099199996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f30b70 msgr2=0x5614e3f37020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f30b70 0x5614e3f37020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 -- 172.21.15.42:0/2099199996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f39750 msgr2=0x5614e3f35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f39750 0x5614e3f35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.481+0000 7f048b7fe640 1 -- 172.21.15.42:0/2099199996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e3e32c20 con 0x5614e3f30250 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f048b7fe640 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f38eb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x5614e3f4c7f0 tx=0x7f048000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16a0befecc6bb15 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0471ffb640 1 -- 172.21.15.42:0/2099199996 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048002f070 con 0x5614e3f30250 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0471ffb640 1 -- 172.21.15.42:0/2099199996 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0480014170 con 0x5614e3f30250 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0471ffb640 1 -- 172.21.15.42:0/2099199996 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048002b5c0 con 0x5614e3f30250 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 -- 172.21.15.42:0/2099199996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 msgr2=0x5614e3f38eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f38eb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x5614e3f4c7f0 tx=0x7f048000b820 comp rx=0 tx=0).stop 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 -- 172.21.15.42:0/2099199996 shutdown_connections 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f39750 0x5614e3f35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f30b70 0x5614e3f37020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 --2- 172.21.15.42:0/2099199996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30250 0x5614e3f38eb0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 -- 172.21.15.42:0/2099199996 >> 172.21.15.42:0/2099199996 conn(0x5614e3f2a980 msgr2=0x5614e3f50840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 -- 172.21.15.42:0/2099199996 shutdown_connections 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.482+0000 7f0495268740 1 -- 172.21.15.42:0/2099199996 wait complete. 2024-03-12T04:22:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 -- start start 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 Processor -- start 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f40ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f3eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f3f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3f3fc40 con 0x5614e3f30b70 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3f9ffd0 con 0x5614e3f30250 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e3fa0110 con 0x5614e3f39750 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f0495268740 10 client.0 _ll_register_callbacks cb 0x7f048000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f3eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f3eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42218/0 (socket says 172.21.15.42:42218) 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 -- 172.21.15.42:0/1135899799 learned_addr learned my addr 172.21.15.42:0/1135899799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048b7fe640 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f40ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 -- 172.21.15.42:0/1135899799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 msgr2=0x5614e3f3f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f3f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 -- 172.21.15.42:0/1135899799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 msgr2=0x5614e3f40ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f40ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.483+0000 7f048bfff640 1 -- 172.21.15.42:0/1135899799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614e3e5a060 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f048bfff640 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f3eec0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x5614e3f35fe0 tx=0x7f047c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db55a47439ec98a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f046b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f046b7fe640 1 -- 172.21.15.42:0/1135899799 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0051d0 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f046b7fe640 1 -- 172.21.15.42:0/1135899799 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f047c056070 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0480005020 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f046b7fe640 1 -- 172.21.15.42:0/1135899799 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f047c02c5d0 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 10 client.17433 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614e3e588c0 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f046b7fe640 1 -- 172.21.15.42:0/1135899799 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0480005020 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f046b7fe640 1 -- 172.21.15.42:0/1135899799 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f047c0035f0 con 0x5614e3f30b70 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 10 client.17433 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 10 client.17433 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 20 client.17433 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3fa1700 0x5614e3fa3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 4 client.17433 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.484+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614e3c23890 con 0x5614e3fa1700 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.485+0000 7f048b7fe640 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3fa1700 0x5614e3fa3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.485+0000 7f046b7fe640 10 client.17433 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.485+0000 7f048b7fe640 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3fa1700 0x5614e3fa3b20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e962bda07d97b36d server_cookie=37797b833352bab7 in_seq=0 out_seq=0 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.485+0000 7f046b7fe640 1 -- 172.21.15.42:0/1135899799 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614e3c23890 con 0x5614e3fa1700 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.485+0000 7f046b7fe640 10 client.17433 handle_command_reply: tid=0 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.485+0000 7f0495268740 1 client.17433 shutdown 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 msgr2=0x5614e3f3eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f3eec0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x5614e3f35fe0 tx=0x7f047c005b50 comp rx=0 tx=0).stop 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 shutdown_connections 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x5614e3fa1700 0x5614e3fa3b20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5614e3f39750 0x5614e3f3f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614e3f30b70 0x5614e3f3eec0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f046b7fe640 0 client.17433 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 --2- 172.21.15.42:0/1135899799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614e3f30250 0x5614e3f40ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 >> 172.21.15.42:0/1135899799 conn(0x5614e3f2a980 msgr2=0x5614e3f2bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 shutdown_connections 2024-03-12T04:22:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 1 -- 172.21.15.42:0/1135899799 wait complete. 2024-03-12T04:22:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0495268740 20 client.17433 trim_cache size 0 max 16384 2024-03-12T04:22:06.494 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f04740a1ba0 msgr2=0x7f04740a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f04740a1ba0 0x7f04740a4060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04840a2830 tx=0x7f0484085040 comp rx=0 tx=0).stop 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 msgr2=0x7f048c1a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c1a3590 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f048400eaa0 tx=0x7f048400ef60 comp rx=0 tx=0).stop 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 shutdown_connections 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f04740a1ba0 0x7f04740a4060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f048c1057c0 0x7f048c1a3590 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f048c100170 0x7f048c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 --2- 172.21.15.42:0/1432406303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f048c0ff7b0 0x7f048c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 >> 172.21.15.42:0/1432406303 conn(0x7f048c076640 msgr2=0x7f048c102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 shutdown_connections 2024-03-12T04:22:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.486+0000 7f0492688640 1 -- 172.21.15.42:0/1432406303 wait complete. 2024-03-12T04:22:06.498 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:06 smithi042 ceph-mon[32260]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:22:06.498 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:06 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1336169293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:22:06.500 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:06.500 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 -- 172.21.15.42:0/1817544727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 msgr2=0x7fdefc100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 --2- 172.21.15.42:0/1817544727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc100680 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009930 tx=0x7fdeec02bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 -- 172.21.15.42:0/1817544727 shutdown_connections 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 --2- 172.21.15.42:0/1817544727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdefc105370 0x7fdefc10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 --2- 172.21.15.42:0/1817544727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdefc100c50 0x7fdefc104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 --2- 172.21.15.42:0/1817544727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc100680 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 -- 172.21.15.42:0/1817544727 >> 172.21.15.42:0/1817544727 conn(0x7fdefc076400 msgr2=0x7fdefc076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 -- 172.21.15.42:0/1817544727 shutdown_connections 2024-03-12T04:22:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.601+0000 7fdf012fb640 1 -- 172.21.15.42:0/1817544727 wait complete. 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 Processor -- start 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 -- start start 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdefc105370 0x7fdefc197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdefc19bf40 0x7fdefc19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19c930 con 0x7fdefc1002a0 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19caa0 con 0x7fdefc19bf40 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdf012fb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19cc10 con 0x7fdefc105370 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc1976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42246/0 (socket says 172.21.15.42:42246) 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 -- 172.21.15.42:0/3682158947 learned_addr learned my addr 172.21.15.42:0/3682158947 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdef27fc640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdefc105370 0x7fdefc197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefb7fe640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdefc19bf40 0x7fdefc19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 -- 172.21.15.42:0/3682158947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdefc105370 msgr2=0x7fdefc197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdefc105370 0x7fdefc197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 -- 172.21.15.42:0/3682158947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdefc19bf40 msgr2=0x7fdefc19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdefc19bf40 0x7fdefc19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.602+0000 7fdefaffd640 1 -- 172.21.15.42:0/3682158947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec009560 con 0x7fdefc1002a0 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.603+0000 7fdef27fc640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdefc105370 0x7fdefc197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.603+0000 7fdefaffd640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc1976d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009230 tx=0x7fdeec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.603+0000 7fdef8ff9640 1 -- 172.21.15.42:0/3682158947 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0044e0 con 0x7fdefc1002a0 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.603+0000 7fdef8ff9640 1 -- 172.21.15.42:0/3682158947 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdeec02de90 con 0x7fdefc1002a0 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.603+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc1a41f0 con 0x7fdefc1002a0 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.603+0000 7fdef8ff9640 1 -- 172.21.15.42:0/3682158947 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec005400 con 0x7fdefc1002a0 2024-03-12T04:22:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.603+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc1a4670 con 0x7fdefc1002a0 2024-03-12T04:22:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.605+0000 7fdef8ff9640 1 -- 172.21.15.42:0/3682158947 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fdeec005560 con 0x7fdefc1002a0 2024-03-12T04:22:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 Processor -- start 2024-03-12T04:22:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 -- start start 2024-03-12T04:22:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374b250 0x558fa374b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdef8ff9640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdee80a1ba0 0x7fdee80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa3754210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa3754750 0x558fa3756b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa3706960 con 0x558fa3754750 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa36ba310 con 0x558fa374b250 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdf03faf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa36b7840 con 0x558fa374bb70 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdefb7fe640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdee80a1ba0 0x7fdee80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdef8ff9640 1 -- 172.21.15.42:0/3682158947 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdeec0efa00 con 0x7fdefc1002a0 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdefb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa3754750 0x558fa3756b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdefb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa3754750 0x558fa3756b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42260/0 (socket says 172.21.15.42:42260) 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdefb7fe640 1 -- 172.21.15.42:0/2234776068 learned_addr learned my addr 172.21.15.42:0/2234776068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374b250 0x558fa374b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.606+0000 7fdef27fc640 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa3754210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefaffd640 1 -- 172.21.15.42:0/2234776068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 msgr2=0x558fa3754210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa3754210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefaffd640 1 -- 172.21.15.42:0/2234776068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa3754750 msgr2=0x558fa3756b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa3754750 0x558fa3756b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefaffd640 1 -- 172.21.15.42:0/2234776068 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa3675060 con 0x558fa374b250 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefb7fe640 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa3754750 0x558fa3756b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefb7fe640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdee80a1ba0 0x7fdee80a4060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdef4013b30 tx=0x7fdef400bd10 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374b250 0x558fa374b630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0ed2b0 tx=0x7fdeec0b8a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33d2b0a7cb7852b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdef17fa640 1 -- 172.21.15.42:0/2234776068 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0bce90 con 0x558fa374b250 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdef17fa640 1 -- 172.21.15.42:0/2234776068 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdeec0bc790 con 0x558fa374b250 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.607+0000 7fdef17fa640 1 -- 172.21.15.42:0/2234776068 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0bb860 con 0x558fa374b250 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 -- 172.21.15.42:0/2234776068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374b250 msgr2=0x558fa374b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374b250 0x558fa374b630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0ed2b0 tx=0x7fdeec0b8a70 comp rx=0 tx=0).stop 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 -- 172.21.15.42:0/2234776068 shutdown_connections 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa3754750 0x558fa3756b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa3754210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2234776068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374b250 0x558fa374b630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 -- 172.21.15.42:0/2234776068 >> 172.21.15.42:0/2234776068 conn(0x558fa3745980 msgr2=0x558fa3745d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 -- 172.21.15.42:0/2234776068 shutdown_connections 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 -- 172.21.15.42:0/2234776068 wait complete. 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 -- start start 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.608+0000 7fdf03faf740 1 Processor -- start 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374b250 0x558fa37517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374bb70 0x558fa3751ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 0x558fa3752220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdf03faf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa36b3020 con 0x558fa374bb70 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdf03faf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa3386430 con 0x558fa3754750 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdf03faf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa3342e00 con 0x558fa374b250 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdf03faf740 10 client.0 _ll_register_callbacks cb 0x558fa365acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374bb70 0x558fa3751ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374bb70 0x558fa3751ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42270/0 (socket says 172.21.15.42:42270) 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 -- 172.21.15.42:0/2923278333 learned_addr learned my addr 172.21.15.42:0/2923278333 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdefb7fe640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 0x558fa3752220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374b250 0x558fa37517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 -- 172.21.15.42:0/2923278333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374b250 msgr2=0x558fa37517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374b250 0x558fa37517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 -- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 msgr2=0x558fa3752220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 0x558fa3752220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 -- 172.21.15.42:0/2923278333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc1a4670 con 0x558fa374bb70 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374b250 0x558fa37517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdef27fc640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374bb70 0x558fa3751ce0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdee40117f0 tx=0x7fdee4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830034ac7905d148 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdee27fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.609+0000 7fdee27fc640 1 -- 172.21.15.42:0/2923278333 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4004520 con 0x558fa374bb70 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdee27fc640 1 -- 172.21.15.42:0/2923278333 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdee4011e40 con 0x558fa374bb70 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fa33b27e0 con 0x558fa374bb70 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdee27fc640 1 -- 172.21.15.42:0/2923278333 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee400f9a0 con 0x558fa374bb70 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 10 client.17445 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558fa36b16a0 con 0x558fa374bb70 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdee27fc640 1 -- 172.21.15.42:0/2923278333 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558fa33b27e0 con 0x558fa374bb70 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdee27fc640 1 -- 172.21.15.42:0/2923278333 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdee4022430 con 0x558fa374bb70 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 10 client.17445 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 10 client.17445 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 20 client.17445 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37648e0 0x558fa3766d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 4 client.17445 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fa33ef720 con 0x558fa37648e0 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.610+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37648e0 0x558fa3766d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.611+0000 7fdefaffd640 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37648e0 0x558fa3766d20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a7cdb77ac6645ec server_cookie=155a27e54e7a5065 in_seq=0 out_seq=0 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.611+0000 7fdee27fc640 10 client.17445 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdee27fc640 1 -- 172.21.15.42:0/2923278333 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558fa33ef720 con 0x558fa37648e0 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdee27fc640 10 client.17445 handle_command_reply: tid=0 2024-03-12T04:22:06.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 client.17445 shutdown 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374bb70 msgr2=0x558fa3751ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374bb70 0x558fa3751ce0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdee40117f0 tx=0x7fdee4011bf0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 shutdown_connections 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37648e0 0x558fa3766d20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 0x558fa3752220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374bb70 0x558fa3751ce0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 --2- 172.21.15.42:0/2923278333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374b250 0x558fa37517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdee27fc640 0 client.17445 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 >> 172.21.15.42:0/2923278333 conn(0x558fa3745980 msgr2=0x558fa37476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 shutdown_connections 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 1 -- 172.21.15.42:0/2923278333 wait complete. 2024-03-12T04:22:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.612+0000 7fdf03faf740 20 client.17445 trim_cache size 0 max 16384 2024-03-12T04:22:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.623+0000 7fdf03faf740 1 Processor -- start 2024-03-12T04:22:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.623+0000 7fdf03faf740 1 -- start start 2024-03-12T04:22:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa3753ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374bb70 0x558fa375b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa3754750 0x558fa3752040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdf03faf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa3638130 con 0x558fa374b250 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdf03faf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa37603d0 con 0x558fa374bb70 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdf03faf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa375bce0 con 0x558fa3754750 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa3753ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa3753ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42272/0 (socket says 172.21.15.42:42272) 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 -- 172.21.15.42:0/345774909 learned_addr learned my addr 172.21.15.42:0/345774909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdef27fc640 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374bb70 0x558fa375b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 -- 172.21.15.42:0/345774909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa3754750 msgr2=0x558fa3752040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa3754750 0x558fa3752040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 -- 172.21.15.42:0/345774909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374bb70 msgr2=0x558fa375b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374bb70 0x558fa375b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 -- 172.21.15.42:0/345774909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa364dc20 con 0x558fa374b250 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdefaffd640 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa3753ed0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0bb650 tx=0x7fdeec0a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e090a998f09dc810 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdef0ff9640 1 -- 172.21.15.42:0/345774909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0bce90 con 0x558fa374b250 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdef0ff9640 1 -- 172.21.15.42:0/345774909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdeec0c08e0 con 0x558fa374b250 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.624+0000 7fdef0ff9640 1 -- 172.21.15.42:0/345774909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0d5270 con 0x558fa374b250 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 -- 172.21.15.42:0/345774909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 msgr2=0x558fa3753ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa3753ed0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0bb650 tx=0x7fdeec0a7040 comp rx=0 tx=0).stop 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 -- 172.21.15.42:0/345774909 shutdown_connections 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa3754750 0x558fa3752040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa374bb70 0x558fa375b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 --2- 172.21.15.42:0/345774909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa3753ed0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 -- 172.21.15.42:0/345774909 >> 172.21.15.42:0/345774909 conn(0x558fa3745980 msgr2=0x558fa374ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 -- 172.21.15.42:0/345774909 shutdown_connections 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 -- 172.21.15.42:0/345774909 wait complete. 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.625+0000 7fdf03faf740 1 -- start start 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 Processor -- start 2024-03-12T04:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa37b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa37b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 0x558fa37b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa37b3470 con 0x558fa374b250 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa37b35e0 con 0x558fa3754750 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa37b37a0 con 0x558fa374bb70 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 10 client.0 _ll_register_callbacks cb 0x7fdeec0b8370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa37b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa37b8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42276/0 (socket says 172.21.15.42:42276) 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 -- 172.21.15.42:0/3224676416 learned_addr learned my addr 172.21.15.42:0/3224676416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdef27fc640 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa37b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 -- 172.21.15.42:0/3224676416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 msgr2=0x558fa37b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa37b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 -- 172.21.15.42:0/3224676416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 msgr2=0x558fa37b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 0x558fa37b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 -- 172.21.15.42:0/3224676416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fa3675060 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdefaffd640 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa37b8aa0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0c4a00 tx=0x7fdeec0bbfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b47bb1bd77c2a33 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdee27fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdee27fc640 1 -- 172.21.15.42:0/3224676416 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0d9070 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdee27fc640 1 -- 172.21.15.42:0/3224676416 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdeec0bce90 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.626+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdeec0bc790 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdee27fc640 1 -- 172.21.15.42:0/3224676416 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeec0d5270 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 10 client.17451 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558fa36738c0 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdee27fc640 1 -- 172.21.15.42:0/3224676416 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdeec0bc790 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdee27fc640 1 -- 172.21.15.42:0/3224676416 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdeec03c070 con 0x558fa374b250 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 10 client.17451 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 10 client.17451 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 20 client.17451 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37c04f0 0x558fa37c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 4 client.17451 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558fa343e890 con 0x558fa37c04f0 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdef27fc640 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37c04f0 0x558fa37c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdef27fc640 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37c04f0 0x558fa37c28e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=873f8f3d1eadf4e8 server_cookie=1c69b9ca50a5436 in_seq=0 out_seq=0 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.627+0000 7fdee27fc640 10 client.17451 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdee27fc640 1 -- 172.21.15.42:0/3224676416 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558fa343e890 con 0x558fa37c04f0 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdee27fc640 10 client.17451 handle_command_reply: tid=0 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 client.17451 shutdown 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 msgr2=0x558fa37b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa37b8aa0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0c4a00 tx=0x7fdeec0bbfe0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 shutdown_connections 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x558fa37c04f0 0x558fa37c28e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fa3754750 0x558fa37b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fa374bb70 0x558fa37b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdee27fc640 0 client.17451 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 --2- 172.21.15.42:0/3224676416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fa374b250 0x558fa37b8aa0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 >> 172.21.15.42:0/3224676416 conn(0x558fa3745980 msgr2=0x558fa37442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 shutdown_connections 2024-03-12T04:22:06.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 1 -- 172.21.15.42:0/3224676416 wait complete. 2024-03-12T04:22:06.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf03faf740 20 client.17451 trim_cache size 0 max 16384 2024-03-12T04:22:06.637 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdee80a1ba0 msgr2=0x7fdee80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf012fb640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdee80a1ba0 0x7fdee80a4060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdef4013b30 tx=0x7fdef400bd10 comp rx=0 tx=0).stop 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 msgr2=0x7fdefc1976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.628+0000 7fdf012fb640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc1976d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009230 tx=0x7fdeec0042f0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 shutdown_connections 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdee80a1ba0 0x7fdee80a4060 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdefc19bf40 0x7fdefc19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdefc105370 0x7fdefc197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 --2- 172.21.15.42:0/3682158947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdefc1002a0 0x7fdefc1976d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 >> 172.21.15.42:0/3682158947 conn(0x7fdefc076400 msgr2=0x7fdefc104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 shutdown_connections 2024-03-12T04:22:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.629+0000 7fdf012fb640 1 -- 172.21.15.42:0/3682158947 wait complete. 2024-03-12T04:22:06.642 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:06.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:22:06.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:06 smithi119 ceph-mon[36119]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:22:06.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:06 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1336169293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:22:06.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:06 smithi136 ceph-mon[35571]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:22:06.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:06 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1336169293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:22:06.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 -- 172.21.15.42:0/1310921338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184102d10 msgr2=0x7fc18410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/1310921338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184102d10 0x7fc18410b1f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc178009900 tx=0x7fc17802bcd0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 -- 172.21.15.42:0/1310921338 shutdown_connections 2024-03-12T04:22:06.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/1310921338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc18410b7c0 0x7fc18410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/1310921338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184102d10 0x7fc18410b1f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/1310921338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1841023f0 0x7fc1841027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 -- 172.21.15.42:0/1310921338 >> 172.21.15.42:0/1310921338 conn(0x7fc1840fbb50 msgr2=0x7fc1840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 -- 172.21.15.42:0/1310921338 shutdown_connections 2024-03-12T04:22:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.744+0000 7fc18aa1c640 1 -- 172.21.15.42:0/1310921338 wait complete. 2024-03-12T04:22:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 Processor -- start 2024-03-12T04:22:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 -- start start 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1841023f0 0x7fc18419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc184102d10 0x7fc18419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18410b7c0 0x7fc1841975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc184197bf0 con 0x7fc18410b7c0 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc184197d60 con 0x7fc184102d10 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc18aa1c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc184197f20 con 0x7fc1841023f0 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc188fb9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18410b7c0 0x7fc1841975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc188fb9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18410b7c0 0x7fc1841975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42300/0 (socket says 172.21.15.42:42300) 2024-03-12T04:22:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.745+0000 7fc188fb9640 1 -- 172.21.15.42:0/3973886122 learned_addr learned my addr 172.21.15.42:0/3973886122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc1837fe640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc184102d10 0x7fc18419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc183fff640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1841023f0 0x7fc18419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc188fb9640 1 -- 172.21.15.42:0/3973886122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1841023f0 msgr2=0x7fc18419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc188fb9640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1841023f0 0x7fc18419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc188fb9640 1 -- 172.21.15.42:0/3973886122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc184102d10 msgr2=0x7fc18419d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc188fb9640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc184102d10 0x7fc18419d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc188fb9640 1 -- 172.21.15.42:0/3973886122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178009560 con 0x7fc18410b7c0 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc183fff640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1841023f0 0x7fc18419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc1837fe640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc184102d10 0x7fc18419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc188fb9640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18410b7c0 0x7fc1841975a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00eaa0 tx=0x7fc17c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc1817fa640 1 -- 172.21.15.42:0/3973886122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00cdb0 con 0x7fc18410b7c0 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc184198200 con 0x7fc18410b7c0 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc1817fa640 1 -- 172.21.15.42:0/3973886122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc17c005020 con 0x7fc18410b7c0 2024-03-12T04:22:06.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc1817fa640 1 -- 172.21.15.42:0/3973886122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0116d0 con 0x7fc18410b7c0 2024-03-12T04:22:06.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.746+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a4490 con 0x7fc18410b7c0 2024-03-12T04:22:06.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.748+0000 7fc1817fa640 1 -- 172.21.15.42:0/3973886122 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fc17c005190 con 0x7fc18410b7c0 2024-03-12T04:22:06.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 Processor -- start 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 -- start start 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc1817fa640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc16c0a1ba0 0x7fc16c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589badfa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badfa8a0 0x5589bae07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589bae07b60 0x5589bae09f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589badb5960 con 0x5589badf9f80 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bad69310 con 0x5589bae07b60 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bad66840 con 0x5589badfa8a0 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589badfa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc1837fe640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc16c0a1ba0 0x7fc16c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc1817fa640 1 -- 172.21.15.42:0/3973886122 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc17c0cd920 con 0x7fc18410b7c0 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589badfa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42314/0 (socket says 172.21.15.42:42314) 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 -- 172.21.15.42:0/49611533 learned_addr learned my addr 172.21.15.42:0/49611533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc1837fe640 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badfa8a0 0x5589bae07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc188fb9640 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589bae07b60 0x5589bae09f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 -- 172.21.15.42:0/49611533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badfa8a0 msgr2=0x5589bae07620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badfa8a0 0x5589bae07620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 -- 172.21.15.42:0/49611533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589bae07b60 msgr2=0x5589bae09f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589bae07b60 0x5589bae09f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.749+0000 7fc183fff640 1 -- 172.21.15.42:0/49611533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589bad24060 con 0x5589badf9f80 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc1837fe640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc16c0a1ba0 0x7fc16c0a4060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1841986e0 tx=0x7fc17802c810 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc183fff640 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589badfa360 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc174005fd0 tx=0x7fc17400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215574b45d6c7d85 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc16a7fc640 1 -- 172.21.15.42:0/49611533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174005020 con 0x5589badf9f80 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc16a7fc640 1 -- 172.21.15.42:0/49611533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc174005630 con 0x5589badf9f80 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc16a7fc640 1 -- 172.21.15.42:0/49611533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1740116d0 con 0x5589badf9f80 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc18d5f5740 1 -- 172.21.15.42:0/49611533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 msgr2=0x5589badfa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589badfa360 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc174005fd0 tx=0x7fc17400e6c0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc18d5f5740 1 -- 172.21.15.42:0/49611533 shutdown_connections 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589bae07b60 0x5589bae09f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badfa8a0 0x5589bae07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/49611533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589badfa360 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.750+0000 7fc18d5f5740 1 -- 172.21.15.42:0/49611533 >> 172.21.15.42:0/49611533 conn(0x5589badf4980 msgr2=0x5589badf4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 -- 172.21.15.42:0/49611533 shutdown_connections 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 -- 172.21.15.42:0/49611533 wait complete. 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 -- start start 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 Processor -- start 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589bae04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589bae055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bad62020 con 0x5589bae07b60 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589baa35430 con 0x5589badfa8a0 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ba9f1e00 con 0x5589badf9f80 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc18d5f5740 10 client.0 _ll_register_callbacks cb 0x5589bad09cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc188fb9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589bae055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc188fb9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589bae055f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42320/0 (socket says 172.21.15.42:42320) 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.751+0000 7fc188fb9640 1 -- 172.21.15.42:0/1538363834 learned_addr learned my addr 172.21.15.42:0/1538363834 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1837fe640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc183fff640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589bae04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc188fb9640 1 -- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 msgr2=0x5589bae04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc188fb9640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589bae04b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc188fb9640 1 -- 172.21.15.42:0/1538363834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 msgr2=0x5589bae050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc188fb9640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc188fb9640 1 -- 172.21.15.42:0/1538363834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc174009900 con 0x5589bae07b60 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc183fff640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589bae04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1837fe640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc188fb9640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589bae055f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0cb120 tx=0x7fc17c09a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c1f01393ec154d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1637fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1637fe640 1 -- 172.21.15.42:0/1538363834 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c09e6f0 con 0x5589bae07b60 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1637fe640 1 -- 172.21.15.42:0/1538363834 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc17c0a2d40 con 0x5589bae07b60 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589baa617e0 con 0x5589bae07b60 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1637fe640 1 -- 172.21.15.42:0/1538363834 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc17c085070 con 0x5589bae07b60 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc18d5f5740 10 client.17463 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5589bad606a0 con 0x5589bae07b60 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1637fe640 1 -- 172.21.15.42:0/1538363834 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5589baa617e0 con 0x5589bae07b60 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.752+0000 7fc1637fe640 1 -- 172.21.15.42:0/1538363834 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc17c083d20 con 0x5589bae07b60 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc18d5f5740 10 client.17463 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc18d5f5740 10 client.17463 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc18d5f5740 20 client.17463 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae17cf0 0x5589bae1a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc18d5f5740 4 client.17463 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589baa9e720 con 0x5589bae17cf0 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc183fff640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae17cf0 0x5589bae1a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc1637fe640 10 client.17463 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.753+0000 7fc183fff640 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae17cf0 0x5589bae1a0e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=926c762053cf28a2 server_cookie=107862f45d700ab9 in_seq=0 out_seq=0 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc1637fe640 1 -- 172.21.15.42:0/1538363834 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589baa9e720 con 0x5589bae17cf0 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc1637fe640 10 client.17463 handle_command_reply: tid=0 2024-03-12T04:22:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 client.17463 shutdown 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 msgr2=0x5589bae055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589bae055f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0cb120 tx=0x7fc17c09a1d0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 shutdown_connections 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae17cf0 0x5589bae1a0e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589bae055f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc1637fe640 0 client.17463 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1538363834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589bae04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 >> 172.21.15.42:0/1538363834 conn(0x5589badf4980 msgr2=0x5589badf66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 shutdown_connections 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1538363834 wait complete. 2024-03-12T04:22:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.754+0000 7fc18d5f5740 20 client.17463 trim_cache size 0 max 16384 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 Processor -- start 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 -- start start 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589badefc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae05410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589badf3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bace7130 con 0x5589bae07b60 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bae1f330 con 0x5589badfa8a0 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589badf9340 con 0x5589badf9f80 2024-03-12T04:22:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc188fb9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589badf3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc188fb9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589badf3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42328/0 (socket says 172.21.15.42:42328) 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc188fb9640 1 -- 172.21.15.42:0/1952330389 learned_addr learned my addr 172.21.15.42:0/1952330389 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.766+0000 7fc183fff640 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589badefc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc1837fe640 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae05410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc188fb9640 1 -- 172.21.15.42:0/1952330389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 msgr2=0x5589badefc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc188fb9640 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589badefc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc188fb9640 1 -- 172.21.15.42:0/1952330389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 msgr2=0x5589bae05410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc188fb9640 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae05410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc188fb9640 1 -- 172.21.15.42:0/1952330389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589bacfcc20 con 0x5589bae07b60 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc183fff640 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589badefc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc188fb9640 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589badf3a00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0a2810 tx=0x7fc17c09a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5fc6db5673855c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc169ffb640 1 -- 172.21.15.42:0/1952330389 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c09aca0 con 0x5589bae07b60 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc169ffb640 1 -- 172.21.15.42:0/1952330389 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc17c0a2d40 con 0x5589bae07b60 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc169ffb640 1 -- 172.21.15.42:0/1952330389 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c096550 con 0x5589bae07b60 2024-03-12T04:22:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1952330389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 msgr2=0x5589badf3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589badf3a00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0a2810 tx=0x7fc17c09a720 comp rx=0 tx=0).stop 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1952330389 shutdown_connections 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589bae07b60 0x5589badf3a00 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae05410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/1952330389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589badf9f80 0x5589badefc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.767+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1952330389 >> 172.21.15.42:0/1952330389 conn(0x5589badf4980 msgr2=0x5589badf32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1952330389 shutdown_connections 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 -- 172.21.15.42:0/1952330389 wait complete. 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 -- start start 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 Processor -- start 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589bae0dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae0e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589bae07b60 0x5589bae68250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bae68790 con 0x5589badf9f80 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bae688d0 con 0x5589badfa8a0 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589bae68a90 con 0x5589bae07b60 2024-03-12T04:22:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc18d5f5740 10 client.0 _ll_register_callbacks cb 0x7fc174009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc183fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589bae0dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc183fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589bae0dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42330/0 (socket says 172.21.15.42:42330) 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc183fff640 1 -- 172.21.15.42:0/119070094 learned_addr learned my addr 172.21.15.42:0/119070094 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc1837fe640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae0e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.768+0000 7fc188fb9640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589bae07b60 0x5589bae68250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc183fff640 1 -- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589bae07b60 msgr2=0x5589bae68250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc183fff640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589bae07b60 0x5589bae68250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc183fff640 1 -- 172.21.15.42:0/119070094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 msgr2=0x5589bae0e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc183fff640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae0e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc183fff640 1 -- 172.21.15.42:0/119070094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589bad606a0 con 0x5589badf9f80 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc188fb9640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589bae07b60 0x5589bae68250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc183fff640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589bae0dd30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc174004c50 tx=0x7fc17400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e854e6c4c19d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc1637fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc1637fe640 1 -- 172.21.15.42:0/119070094 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174004880 con 0x5589badf9f80 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc1637fe640 1 -- 172.21.15.42:0/119070094 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc174004db0 con 0x5589badf9f80 2024-03-12T04:22:06.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc174005630 con 0x5589badf9f80 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc1637fe640 1 -- 172.21.15.42:0/119070094 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1740049e0 con 0x5589badf9f80 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 10 client.17469 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5589bad228c0 con 0x5589badf9f80 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc1637fe640 1 -- 172.21.15.42:0/119070094 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc174005630 con 0x5589badf9f80 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc1637fe640 1 -- 172.21.15.42:0/119070094 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc17400c070 con 0x5589badf9f80 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 10 client.17469 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 10 client.17469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 20 client.17469 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae70130 0x5589bae72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 4 client.17469 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.769+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5589baaed890 con 0x5589bae70130 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc1837fe640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae70130 0x5589bae72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc1837fe640 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae70130 0x5589bae72550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4a08f661fce1013 server_cookie=421d6a3da9df78b1 in_seq=0 out_seq=0 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc1637fe640 10 client.17469 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc1637fe640 1 -- 172.21.15.42:0/119070094 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5589baaed890 con 0x5589bae70130 2024-03-12T04:22:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc1637fe640 10 client.17469 handle_command_reply: tid=0 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc18d5f5740 1 client.17469 shutdown 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 msgr2=0x5589bae0dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.770+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589bae0dd30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc174004c50 tx=0x7fc17400ef80 comp rx=0 tx=0).stop 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 shutdown_connections 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5589bae70130 0x5589bae72550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589bae07b60 0x5589bae68250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5589badfa8a0 0x5589bae0e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc1637fe640 0 client.17469 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 --2- 172.21.15.42:0/119070094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589badf9f80 0x5589bae0dd30 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 >> 172.21.15.42:0/119070094 conn(0x5589badf4980 msgr2=0x5589badf3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 shutdown_connections 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 1 -- 172.21.15.42:0/119070094 wait complete. 2024-03-12T04:22:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18d5f5740 20 client.17469 trim_cache size 0 max 16384 2024-03-12T04:22:06.780 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:06.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc16c0a1ba0 msgr2=0x7fc16c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc16c0a1ba0 0x7fc16c0a4060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1841986e0 tx=0x7fc17802c810 comp rx=0 tx=0).stop 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18410b7c0 msgr2=0x7fc1841975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18410b7c0 0x7fc1841975a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00eaa0 tx=0x7fc17c00ef60 comp rx=0 tx=0).stop 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 shutdown_connections 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fc16c0a1ba0 0x7fc16c0a4060 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18410b7c0 0x7fc1841975a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc184102d10 0x7fc18419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 --2- 172.21.15.42:0/3973886122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1841023f0 0x7fc18419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 >> 172.21.15.42:0/3973886122 conn(0x7fc1840fbb50 msgr2=0x7fc1840fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 shutdown_connections 2024-03-12T04:22:06.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.771+0000 7fc18aa1c640 1 -- 172.21.15.42:0/3973886122 wait complete. 2024-03-12T04:22:06.784 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:06.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:22:06.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.888+0000 7fe7135c2640 1 -- 172.21.15.42:0/3711148055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c0685f0 msgr2=0x7fe70c0689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.888+0000 7fe7135c2640 1 --2- 172.21.15.42:0/3711148055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c0685f0 0x7fe70c0689d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe704009cd0 tx=0x7fe70402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.888+0000 7fe7135c2640 1 -- 172.21.15.42:0/3711148055 shutdown_connections 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.888+0000 7fe7135c2640 1 --2- 172.21.15.42:0/3711148055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70c1064c0 0x7fe70c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.888+0000 7fe7135c2640 1 --2- 172.21.15.42:0/3711148055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c068fa0 0x7fe70c105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.888+0000 7fe7135c2640 1 --2- 172.21.15.42:0/3711148055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c0685f0 0x7fe70c0689d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 -- 172.21.15.42:0/3711148055 >> 172.21.15.42:0/3711148055 conn(0x7fe70c077180 msgr2=0x7fe70c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 -- 172.21.15.42:0/3711148055 shutdown_connections 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 -- 172.21.15.42:0/3711148055 wait complete. 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 Processor -- start 2024-03-12T04:22:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 -- start start 2024-03-12T04:22:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c0685f0 0x7fe70c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70c1064c0 0x7fe70c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c19bfb0 0x7fe70c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c19c9a0 con 0x7fe70c19bfb0 2024-03-12T04:22:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c19cb10 con 0x7fe70c1064c0 2024-03-12T04:22:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.889+0000 7fe7135c2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c19cc80 con 0x7fe70c0685f0 2024-03-12T04:22:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c19bfb0 0x7fe70c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c19bfb0 0x7fe70c19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42350/0 (socket says 172.21.15.42:42350) 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 -- 172.21.15.42:0/1695061691 learned_addr learned my addr 172.21.15.42:0/1695061691 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe710b5d640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70c1064c0 0x7fe70c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe71135e640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c0685f0 0x7fe70c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 -- 172.21.15.42:0/1695061691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c0685f0 msgr2=0x7fe70c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c0685f0 0x7fe70c1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 -- 172.21.15.42:0/1695061691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70c1064c0 msgr2=0x7fe70c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70c1064c0 0x7fe70c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 -- 172.21.15.42:0/1695061691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704009900 con 0x7fe70c19bfb0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe711b5f640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c19bfb0 0x7fe70c19c460 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d750 tx=0x7fe6fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe70a7fc640 1 -- 172.21.15.42:0/1695061691 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc009bd0 con 0x7fe70c19bfb0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c1a4260 con 0x7fe70c19bfb0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe70a7fc640 1 -- 172.21.15.42:0/1695061691 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc00de60 con 0x7fe70c19bfb0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe70a7fc640 1 -- 172.21.15.42:0/1695061691 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc004e00 con 0x7fe70c19bfb0 2024-03-12T04:22:06.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.890+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c1a46c0 con 0x7fe70c19bfb0 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.892+0000 7fe70a7fc640 1 -- 172.21.15.42:0/1695061691 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe6fc004f60 con 0x7fe70c19bfb0 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 Processor -- start 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 -- start start 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb2a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb33210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe70a7fc640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6f40a1ba0 0x7fe6f40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb35b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52eae5960 con 0x55d52eb33750 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52ea99310 con 0x55d52eb2ab70 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71618b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52ea96840 con 0x55d52eb2a250 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe71135e640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6f40a1ba0 0x7fe6f40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb35b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb35b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42352/0 (socket says 172.21.15.42:42352) 2024-03-12T04:22:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe711b5f640 1 -- 172.21.15.42:0/2727249412 learned_addr learned my addr 172.21.15.42:0/2727249412 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.893+0000 7fe710b5d640 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb33210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe70a7fc640 1 -- 172.21.15.42:0/1695061691 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe6fc0cd0d0 con 0x7fe70c19bfb0 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe710b5d640 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb2a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe711b5f640 1 -- 172.21.15.42:0/2727249412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 msgr2=0x55d52eb2a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe711b5f640 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb2a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe711b5f640 1 -- 172.21.15.42:0/2727249412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 msgr2=0x55d52eb33210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe711b5f640 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb33210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe711b5f640 1 -- 172.21.15.42:0/2727249412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d52ea54060 con 0x55d52eb33750 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe71135e640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6f40a1ba0 0x7fe6f40a4060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe704009430 tx=0x7fe704009360 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe711b5f640 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb35b40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0ca910 tx=0x7fe6fc0998d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b0eacf994d5a83 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe6f37fe640 1 -- 172.21.15.42:0/2727249412 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc09a490 con 0x55d52eb33750 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe6f37fe640 1 -- 172.21.15.42:0/2727249412 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc0a33f0 con 0x55d52eb33750 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.894+0000 7fe6f37fe640 1 -- 172.21.15.42:0/2727249412 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc09dee0 con 0x55d52eb33750 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 -- 172.21.15.42:0/2727249412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 msgr2=0x55d52eb35b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb35b40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0ca910 tx=0x7fe6fc0998d0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 -- 172.21.15.42:0/2727249412 shutdown_connections 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb35b40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb33210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 --2- 172.21.15.42:0/2727249412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb2a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 -- 172.21.15.42:0/2727249412 >> 172.21.15.42:0/2727249412 conn(0x55d52eb24980 msgr2=0x55d52eb24d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 -- 172.21.15.42:0/2727249412 shutdown_connections 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 -- 172.21.15.42:0/2727249412 wait complete. 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 -- start start 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.895+0000 7fe71618b740 1 Processor -- start 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb30640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb30b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52ea92020 con 0x55d52eb33750 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52e765430 con 0x55d52eb2ab70 2024-03-12T04:22:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52e721e00 con 0x55d52eb2a250 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 10 client.0 _ll_register_callbacks cb 0x55d52ea39cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb310c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42366/0 (socket says 172.21.15.42:42366) 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 -- 172.21.15.42:0/176094541 learned_addr learned my addr 172.21.15.42:0/176094541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71135e640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb30640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 -- 172.21.15.42:0/176094541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 msgr2=0x55d52eb30640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe710b5d640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb30b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb30640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 -- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 msgr2=0x55d52eb30b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb30b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 -- 172.21.15.42:0/176094541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc016e70 con 0x55d52eb33750 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71135e640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb30640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe711b5f640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb310c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc099a90 tx=0x7fe6fc099ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e25d6ed53ec6c33 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe6f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/176094541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc014070 con 0x55d52eb33750 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/176094541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe6fc09a490 con 0x55d52eb33750 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.896+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d52e7917e0 con 0x55d52eb33750 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/176094541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6fc0a2530 con 0x55d52eb33750 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 10 client.17481 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d52ea906a0 con 0x55d52eb33750 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/176094541 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc0b6020 con 0x55d52eb33750 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/176094541 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe6fc005560 con 0x55d52eb33750 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 10 client.17481 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 20 client.17481 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb43700 0x55d52eb45af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 4 client.17481 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d52e7ce720 con 0x55d52eb43700 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.897+0000 7fe71135e640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb43700 0x55d52eb45af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.898+0000 7fe6f0ff9640 10 client.17481 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.898+0000 7fe71135e640 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb43700 0x55d52eb45af0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79fd2abdd00f5a4c server_cookie=4545c969bc3e82a6 in_seq=0 out_seq=0 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.898+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/176094541 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d52e7ce720 con 0x55d52eb43700 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.898+0000 7fe6f0ff9640 10 client.17481 handle_command_reply: tid=0 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.898+0000 7fe71618b740 1 client.17481 shutdown 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.898+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 msgr2=0x55d52eb310c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.898+0000 7fe71618b740 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb310c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc099a90 tx=0x7fe6fc099ed0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 shutdown_connections 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb43700 0x55d52eb45af0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb310c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe6f0ff9640 0 client.17481 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2ab70 0x55d52eb30b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 --2- 172.21.15.42:0/176094541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2a250 0x55d52eb30640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 >> 172.21.15.42:0/176094541 conn(0x55d52eb24980 msgr2=0x55d52eb266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 shutdown_connections 2024-03-12T04:22:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 1 -- 172.21.15.42:0/176094541 wait complete. 2024-03-12T04:22:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.899+0000 7fe71618b740 20 client.17481 trim_cache size 0 max 16384 2024-03-12T04:22:06.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.910+0000 7fe71618b740 1 Processor -- start 2024-03-12T04:22:06.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.910+0000 7fe71618b740 1 -- start start 2024-03-12T04:22:06.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71618b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71618b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2ab70 0x55d52eb30ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71618b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb2d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71618b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52ea17130 con 0x55d52eb33750 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71618b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52eb3f4b0 con 0x55d52eb2a250 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71618b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52eb205f0 con 0x55d52eb2ab70 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb2d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb2d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42374/0 (socket says 172.21.15.42:42374) 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 -- 172.21.15.42:0/2361918983 learned_addr learned my addr 172.21.15.42:0/2361918983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 -- 172.21.15.42:0/2361918983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2ab70 msgr2=0x55d52eb30ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2ab70 0x55d52eb30ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 -- 172.21.15.42:0/2361918983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 msgr2=0x55d52eb32d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71135e640 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb32d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 -- 172.21.15.42:0/2361918983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d52ea2cc20 con 0x55d52eb33750 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe71135e640 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe711b5f640 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb2d8a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc096760 tx=0x7fe6fc004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111e301fe55a9c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe6f2ffd640 1 -- 172.21.15.42:0/2361918983 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc014070 con 0x55d52eb33750 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe6f2ffd640 1 -- 172.21.15.42:0/2361918983 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc0b6070 con 0x55d52eb33750 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.911+0000 7fe6f2ffd640 1 -- 172.21.15.42:0/2361918983 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc09dee0 con 0x55d52eb33750 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 -- 172.21.15.42:0/2361918983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 msgr2=0x55d52eb2d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb2d8a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc096760 tx=0x7fe6fc004840 comp rx=0 tx=0).stop 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 -- 172.21.15.42:0/2361918983 shutdown_connections 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb33750 0x55d52eb2d8a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb2ab70 0x55d52eb30ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 --2- 172.21.15.42:0/2361918983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 -- 172.21.15.42:0/2361918983 >> 172.21.15.42:0/2361918983 conn(0x55d52eb24980 msgr2=0x55d52eb1fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 -- 172.21.15.42:0/2361918983 shutdown_connections 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 -- 172.21.15.42:0/2361918983 wait complete. 2024-03-12T04:22:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 -- start start 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.912+0000 7fe71618b740 1 Processor -- start 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71618b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71618b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb2ab70 0x55d52eb38d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71618b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb33750 0x55d52eb92b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71618b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52eb93060 con 0x55d52eb2ab70 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71618b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52eb931a0 con 0x55d52eb2a250 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71618b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52eb93360 con 0x55d52eb33750 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71618b740 10 client.0 _ll_register_callbacks cb 0x7fe70000b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb2ab70 0x55d52eb38d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb2ab70 0x55d52eb38d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42386/0 (socket says 172.21.15.42:42386) 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 -- 172.21.15.42:0/4221885359 learned_addr learned my addr 172.21.15.42:0/4221885359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe71135e640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe711b5f640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb33750 0x55d52eb92b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 -- 172.21.15.42:0/4221885359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb33750 msgr2=0x55d52eb92b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb33750 0x55d52eb92b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 -- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 msgr2=0x55d52eb387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 -- 172.21.15.42:0/4221885359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d52ea906a0 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe711b5f640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb33750 0x55d52eb92b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe710b5d640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb2ab70 0x55d52eb38d30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe700005fd0 tx=0x7fe7000119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea85e2d04dda7b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe6f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/4221885359 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700017070 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.913+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/4221885359 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe700004220 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6fc0a33f0 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/4221885359 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7000043e0 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 10 client.17487 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d52ea528c0 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/4221885359 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc0a33f0 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/4221885359 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe700004c20 con 0x55d52eb2ab70 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 10 client.17487 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 20 client.17487 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb9ac70 0x55d52eb9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 4 client.17487 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d52e81d890 con 0x55d52eb9ac70 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.914+0000 7fe71135e640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb9ac70 0x55d52eb9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe6f0ff9640 10 client.17487 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71135e640 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb9ac70 0x55d52eb9d090 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948013ecf73a6d60 server_cookie=dbac347a9ea58e60 in_seq=0 out_seq=0 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe6f0ff9640 1 -- 172.21.15.42:0/4221885359 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d52e81d890 con 0x55d52eb9ac70 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe6f0ff9640 10 client.17487 handle_command_reply: tid=0 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 client.17487 shutdown 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb2ab70 msgr2=0x55d52eb38d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb2ab70 0x55d52eb38d30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe700005fd0 tx=0x7fe7000119e0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 shutdown_connections 2024-03-12T04:22:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55d52eb9ac70 0x55d52eb9d090 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d52eb33750 0x55d52eb92b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe6f0ff9640 0 client.17487 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d52eb2ab70 0x55d52eb38d30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 --2- 172.21.15.42:0/4221885359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d52eb2a250 0x55d52eb387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 >> 172.21.15.42:0/4221885359 conn(0x55d52eb24980 msgr2=0x55d52eb232f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 shutdown_connections 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 1 -- 172.21.15.42:0/4221885359 wait complete. 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.915+0000 7fe71618b740 20 client.17487 trim_cache size 0 max 16384 2024-03-12T04:22:06.927 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6f40a1ba0 msgr2=0x7fe6f40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6f40a1ba0 0x7fe6f40a4060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe704009430 tx=0x7fe704009360 comp rx=0 tx=0).stop 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c19bfb0 msgr2=0x7fe70c19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c19bfb0 0x7fe70c19c460 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d750 tx=0x7fe6fc00dc10 comp rx=0 tx=0).stop 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 shutdown_connections 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe6f40a1ba0 0x7fe6f40a4060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70c19bfb0 0x7fe70c19c460 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70c1064c0 0x7fe70c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 --2- 172.21.15.42:0/1695061691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c0685f0 0x7fe70c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 >> 172.21.15.42:0/1695061691 conn(0x7fe70c077180 msgr2=0x7fe70c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 shutdown_connections 2024-03-12T04:22:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:06.916+0000 7fe7135c2640 1 -- 172.21.15.42:0/1695061691 wait complete. 2024-03-12T04:22:07.930 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:22:07.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:22:07.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:22:08.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 -- 172.21.15.42:0/1834920033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc101d40 msgr2=0x7fafcc105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1834920033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc101d40 0x7fafcc105d80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009930 tx=0x7fafb402bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 -- 172.21.15.42:0/1834920033 shutdown_connections 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1834920033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc106460 0x7fafcc10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1834920033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc101d40 0x7fafcc105d80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1834920033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafcc101390 0x7fafcc101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 -- 172.21.15.42:0/1834920033 >> 172.21.15.42:0/1834920033 conn(0x7fafcc0fcbb0 msgr2=0x7fafcc0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 -- 172.21.15.42:0/1834920033 shutdown_connections 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.047+0000 7fafd13a0640 1 -- 172.21.15.42:0/1834920033 wait complete. 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 Processor -- start 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 -- start start 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafcc101390 0x7fafcc073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc106460 0x7fafcc0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc073f00 0x7fafcc1a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1a5e70 con 0x7fafcc106460 2024-03-12T04:22:08.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1a5fe0 con 0x7fafcc073f00 2024-03-12T04:22:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafd13a0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1a61a0 con 0x7fafcc101390 2024-03-12T04:22:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafc3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc106460 0x7fafcc0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafc3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc106460 0x7fafcc0739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42412/0 (socket says 172.21.15.42:42412) 2024-03-12T04:22:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.048+0000 7fafc3fff640 1 -- 172.21.15.42:0/1692478907 learned_addr learned my addr 172.21.15.42:0/1692478907 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc073f00 0x7fafcc1a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafcaffd640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafcc101390 0x7fafcc073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc3fff640 1 -- 172.21.15.42:0/1692478907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafcc101390 msgr2=0x7fafcc073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc3fff640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafcc101390 0x7fafcc073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc3fff640 1 -- 172.21.15.42:0/1692478907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc073f00 msgr2=0x7fafcc1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc3fff640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc073f00 0x7fafcc1a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc3fff640 1 -- 172.21.15.42:0/1692478907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4009560 con 0x7fafcc106460 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc3fff640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc106460 0x7fafcc0739c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005ab0 tx=0x7fafb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc8ff9640 1 -- 172.21.15.42:0/1692478907 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb40035f0 con 0x7fafcc106460 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc8ff9640 1 -- 172.21.15.42:0/1692478907 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fafb402de90 con 0x7fafcc106460 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc1a6420 con 0x7fafcc106460 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafc8ff9640 1 -- 172.21.15.42:0/1692478907 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4005470 con 0x7fafcc106460 2024-03-12T04:22:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.049+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc1a6930 con 0x7fafcc106460 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.051+0000 7fafc8ff9640 1 -- 172.21.15.42:0/1692478907 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fafb4003750 con 0x7fafcc106460 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 Processor -- start 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 -- start start 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbb2380 0x55617bbb2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbbb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafc8ff9640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fafb80a1bd0 0x7fafb80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbbb8b0 0x55617bbbdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bb20f90 con 0x55617bbb2380 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bb1e4c0 con 0x55617bbb2d30 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafd3f70740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bb19ca0 con 0x55617bbbb8b0 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbb2380 0x55617bbb2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbb2380 0x55617bbb2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42418/0 (socket says 172.21.15.42:42418) 2024-03-12T04:22:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1572796393 learned_addr learned my addr 172.21.15.42:0/1572796393 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafc3fff640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fafb80a1bd0 0x7fafb80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafc8ff9640 1 -- 172.21.15.42:0/1692478907 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fafb40ef9f0 con 0x7fafcc106460 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.052+0000 7fafcaffd640 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbbb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1572796393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbbb8b0 msgr2=0x55617bbbdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbbb8b0 0x55617bbbdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1572796393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 msgr2=0x55617bbbb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbbb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1572796393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55617badc0b0 con 0x55617bbb2380 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafcaffd640 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbbb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafc3fff640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fafb80a1bd0 0x7fafb80a4090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafb40ed350 tx=0x7fafb40a7040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbb2380 0x55617bbb2760 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fafc400aec0 tx=0x7fafc40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d068caa7df9cc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafc17fa640 1 -- 172.21.15.42:0/1572796393 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003b70 con 0x55617bbb2380 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafc17fa640 1 -- 172.21.15.42:0/1572796393 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fafc4003cd0 con 0x55617bbb2380 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.053+0000 7fafc17fa640 1 -- 172.21.15.42:0/1572796393 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40126b0 con 0x55617bbb2380 2024-03-12T04:22:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- 172.21.15.42:0/1572796393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbb2380 msgr2=0x55617bbb2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbb2380 0x55617bbb2760 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fafc400aec0 tx=0x7fafc40117b0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- 172.21.15.42:0/1572796393 shutdown_connections 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbbb8b0 0x55617bbbdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbbb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1572796393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbb2380 0x55617bbb2760 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- 172.21.15.42:0/1572796393 >> 172.21.15.42:0/1572796393 conn(0x55617bbac840 msgr2=0x55617bbacc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- 172.21.15.42:0/1572796393 shutdown_connections 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- 172.21.15.42:0/1572796393 wait complete. 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- start start 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 Processor -- start 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bbb8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbb8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617babef40 con 0x55617bbbb8b0 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617b7ed4b0 con 0x55617bbb2d30 2024-03-12T04:22:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.054+0000 7fafd3f70740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617ba9f670 con 0x55617bbb2380 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafd3f70740 10 client.0 _ll_register_callbacks cb 0x55617bb0cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42430/0 (socket says 172.21.15.42:42430) 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 -- 172.21.15.42:0/3750191862 learned_addr learned my addr 172.21.15.42:0/3750191862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bbb8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcaffd640 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbb8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 -- 172.21.15.42:0/3750191862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 msgr2=0x55617bbb8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bbb8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 -- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 msgr2=0x55617bbb8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbb8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 -- 172.21.15.42:0/3750191862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc400ab70 con 0x55617bbbb8b0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb92c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fafc4004770 tx=0x7fafc400a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8660d6b265dec32 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafb27fc640 1 -- 172.21.15.42:0/3750191862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4004a40 con 0x55617bbbb8b0 2024-03-12T04:22:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafb27fc640 1 -- 172.21.15.42:0/3750191862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fafc4004ba0 con 0x55617bbbb8b0 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55617baf3f40 con 0x55617bbbb8b0 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafb27fc640 1 -- 172.21.15.42:0/3750191862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc4004d60 con 0x55617bbbb8b0 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafd3f70740 10 client.17499 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.055+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55617bb18320 con 0x55617bbbb8b0 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafb27fc640 1 -- 172.21.15.42:0/3750191862 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55617baf3f40 con 0x55617bbbb8b0 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafb27fc640 1 -- 172.21.15.42:0/3750191862 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fafc4018030 con 0x55617bbbb8b0 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafd3f70740 10 client.17499 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafd3f70740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafd3f70740 20 client.17499 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafd3f70740 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bbcbe50 0x55617bbce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafd3f70740 4 client.17499 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55617bb10310 con 0x55617bbcbe50 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.056+0000 7fafcaffd640 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bbcbe50 0x55617bbce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafcaffd640 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bbcbe50 0x55617bbce270 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a6fa1ddc45166b server_cookie=55bae0b569033380 in_seq=0 out_seq=0 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafb27fc640 10 client.17499 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafb27fc640 1 -- 172.21.15.42:0/3750191862 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55617bb10310 con 0x55617bbcbe50 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafb27fc640 10 client.17499 handle_command_reply: tid=0 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 client.17499 shutdown 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 msgr2=0x55617bbb92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb92c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fafc4004770 tx=0x7fafc400a7f0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 shutdown_connections 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bbcbe50 0x55617bbce270 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb92c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bbb8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafb27fc640 0 client.17499 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 --2- 172.21.15.42:0/3750191862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bbb8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 >> 172.21.15.42:0/3750191862 conn(0x55617bbac840 msgr2=0x55617bbae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 shutdown_connections 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.057+0000 7fafd3f70740 1 -- 172.21.15.42:0/3750191862 wait complete. 2024-03-12T04:22:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.058+0000 7fafd3f70740 20 client.17499 trim_cache size 0 max 16384 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 Processor -- start 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 -- start start 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2380 0x55617bbbaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2d30 0x55617bbb79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bbb1cd0 con 0x55617bbbb8b0 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bbb4470 con 0x55617bbb2380 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafd3f70740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bbb7ee0 con 0x55617bbb2d30 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.073+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42442/0 (socket says 172.21.15.42:42442) 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 -- 172.21.15.42:0/2485240527 learned_addr learned my addr 172.21.15.42:0/2485240527 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2380 0x55617bbbaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcaffd640 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2d30 0x55617bbb79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 -- 172.21.15.42:0/2485240527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2d30 msgr2=0x55617bbb79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2d30 0x55617bbb79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 -- 172.21.15.42:0/2485240527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2380 msgr2=0x55617bbbaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2380 0x55617bbbaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 -- 172.21.15.42:0/2485240527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55617bab4c00 con 0x55617bbbb8b0 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb8f10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fafc4002230 tx=0x7fafc400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec9043ce34a37b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafc0ff9640 1 -- 172.21.15.42:0/2485240527 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc400ad90 con 0x55617bbbb8b0 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafc0ff9640 1 -- 172.21.15.42:0/2485240527 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fafc40045c0 con 0x55617bbbb8b0 2024-03-12T04:22:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.074+0000 7fafc0ff9640 1 -- 172.21.15.42:0/2485240527 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003da0 con 0x55617bbbb8b0 2024-03-12T04:22:08.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 -- 172.21.15.42:0/2485240527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 msgr2=0x55617bbb8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb8f10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fafc4002230 tx=0x7fafc400a5c0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 -- 172.21.15.42:0/2485240527 shutdown_connections 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbb8f10 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2d30 0x55617bbb79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 --2- 172.21.15.42:0/2485240527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2380 0x55617bbbaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 -- 172.21.15.42:0/2485240527 >> 172.21.15.42:0/2485240527 conn(0x55617bbac840 msgr2=0x55617bbb8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 -- 172.21.15.42:0/2485240527 shutdown_connections 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 -- 172.21.15.42:0/2485240527 wait complete. 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 -- start start 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.075+0000 7fafd3f70740 1 Processor -- start 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bc1c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bc1cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbc1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bbc2320 con 0x55617bbbb8b0 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bbc2490 con 0x55617bbb2d30 2024-03-12T04:22:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617bbc2650 con 0x55617bbb2380 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 10 client.0 _ll_register_callbacks cb 0x7fafc400a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbc1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbc1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42456/0 (socket says 172.21.15.42:42456) 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1722087199 learned_addr learned my addr 172.21.15.42:0/1722087199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcaffd640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bc1cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bc1c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1722087199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 msgr2=0x55617bc1c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bc1c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 msgr2=0x55617bc1cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bc1cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 -- 172.21.15.42:0/1722087199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55617bb18320 con 0x55617bbbb8b0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcaffd640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bc1cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafcb7fe640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbc1d60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fafc4004360 tx=0x7fafc4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce83b76c18f2ad8a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafb27fc640 1 -- 172.21.15.42:0/1722087199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40041b0 con 0x55617bbbb8b0 2024-03-12T04:22:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafb27fc640 1 -- 172.21.15.42:0/1722087199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fafc4011cb0 con 0x55617bbbb8b0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55617bb3f900 con 0x55617bbbb8b0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafb27fc640 1 -- 172.21.15.42:0/1722087199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc4005850 con 0x55617bbbb8b0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 10 client.17505 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.076+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55617bada910 con 0x55617bbbb8b0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafb27fc640 1 -- 172.21.15.42:0/1722087199 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55617bb3f900 con 0x55617bbbb8b0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafb27fc640 1 -- 172.21.15.42:0/1722087199 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fafc4018070 con 0x55617bbbb8b0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafd3f70740 10 client.17505 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafd3f70740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafd3f70740 20 client.17505 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bc243f0 0x55617bc26810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafd3f70740 4 client.17505 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55617b819860 con 0x55617bc243f0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafcaffd640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bc243f0 0x55617bc26810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafb27fc640 10 client.17505 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.077+0000 7fafcaffd640 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bc243f0 0x55617bc26810 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821419e56311fc2a server_cookie=ee5773002524049d in_seq=0 out_seq=0 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafb27fc640 1 -- 172.21.15.42:0/1722087199 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55617b819860 con 0x55617bc243f0 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafb27fc640 10 client.17505 handle_command_reply: tid=0 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 client.17505 shutdown 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 msgr2=0x55617bbc1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbc1d60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fafc4004360 tx=0x7fafc4005770 comp rx=0 tx=0).stop 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 shutdown_connections 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55617bc243f0 0x55617bc26810 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55617bbbb8b0 0x55617bbc1d60 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55617bbb2d30 0x55617bc1cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 --2- 172.21.15.42:0/1722087199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55617bbb2380 0x55617bc1c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 >> 172.21.15.42:0/1722087199 conn(0x55617bbac840 msgr2=0x55617bbb7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafb27fc640 0 client.17505 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 shutdown_connections 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 1 -- 172.21.15.42:0/1722087199 wait complete. 2024-03-12T04:22:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.079+0000 7fafd3f70740 20 client.17505 trim_cache size 0 max 16384 2024-03-12T04:22:08.087 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:22:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:22:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "d882ce4b-803f-4e3f-9ca6-90692ea8239e", 2024-03-12T04:22:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:22:08.087 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fafb80a1bd0 msgr2=0x7fafb80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fafb80a1bd0 0x7fafb80a4090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafb40ed350 tx=0x7fafb40a7040 comp rx=0 tx=0).stop 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc106460 msgr2=0x7fafcc0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc106460 0x7fafcc0739c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005ab0 tx=0x7fafb4004270 comp rx=0 tx=0).stop 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 shutdown_connections 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fafb80a1bd0 0x7fafb80a4090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc073f00 0x7fafcc1a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafcc106460 0x7fafcc0739c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 --2- 172.21.15.42:0/1692478907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafcc101390 0x7fafcc073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 >> 172.21.15.42:0/1692478907 conn(0x7fafcc0fcbb0 msgr2=0x7fafcc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 shutdown_connections 2024-03-12T04:22:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.080+0000 7fafd13a0640 1 -- 172.21.15.42:0/1692478907 wait complete. 2024-03-12T04:22:08.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:22:08.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 -- 172.21.15.42:0/4018971179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100c20 msgr2=0x7f5978104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 --2- 172.21.15.42:0/4018971179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100c20 0x7f5978104c60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5960009930 tx=0x7f596002bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 -- 172.21.15.42:0/4018971179 shutdown_connections 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 --2- 172.21.15.42:0/4018971179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978105340 0x7f597810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 --2- 172.21.15.42:0/4018971179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978100c20 0x7f5978104c60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 --2- 172.21.15.42:0/4018971179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5978100270 0x7f5978100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 -- 172.21.15.42:0/4018971179 >> 172.21.15.42:0/4018971179 conn(0x7f5978077230 msgr2=0x7f5978077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 -- 172.21.15.42:0/4018971179 shutdown_connections 2024-03-12T04:22:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.195+0000 7f597d4d4640 1 -- 172.21.15.42:0/4018971179 wait complete. 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 Processor -- start 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 -- start start 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5978100270 0x7f5978198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105340 0x7f5978198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978199010 0x7f59781a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59781a3c60 con 0x7f5978105340 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59781a3dd0 con 0x7f5978199010 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f597d4d4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59781a3f90 con 0x7f5978100270 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f59767fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105340 0x7f5978198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f59767fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105340 0x7f5978198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42470/0 (socket says 172.21.15.42:42470) 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f59767fc640 1 -- 172.21.15.42:0/2720362988 learned_addr learned my addr 172.21.15.42:0/2720362988 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.196+0000 7f59777fe640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978199010 0x7f59781a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5978100270 0x7f5978198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f59767fc640 1 -- 172.21.15.42:0/2720362988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5978100270 msgr2=0x7f5978198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f59767fc640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5978100270 0x7f5978198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f59767fc640 1 -- 172.21.15.42:0/2720362988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978199010 msgr2=0x7f59781a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f59767fc640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978199010 0x7f59781a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f59767fc640 1 -- 172.21.15.42:0/2720362988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960009560 con 0x7f5978105340 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f59767fc640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105340 0x7f5978198ad0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5960005ab0 tx=0x7f5960004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f596ffff640 1 -- 172.21.15.42:0/2720362988 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59600035f0 con 0x7f5978105340 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59781a4210 con 0x7f5978105340 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f596ffff640 1 -- 172.21.15.42:0/2720362988 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f596002de90 con 0x7f5978105340 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f596ffff640 1 -- 172.21.15.42:0/2720362988 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960005470 con 0x7f5978105340 2024-03-12T04:22:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.197+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59781a4770 con 0x7f5978105340 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.199+0000 7f596ffff640 1 -- 172.21.15.42:0/2720362988 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f5960003750 con 0x7f5978105340 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 Processor -- start 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 -- start start 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d287690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d290270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d292ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f596ffff640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1ba0 0x7f59640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d1f6280 con 0x56144d287bd0 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d1f37b0 con 0x56144d2907b0 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f598017f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d1eef90 con 0x56144d2872b0 2024-03-12T04:22:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d290270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f59777fe640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1ba0 0x7f59640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d290270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42486/0 (socket says 172.21.15.42:42486) 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f5976ffd640 1 -- 172.21.15.42:0/2869255958 learned_addr learned my addr 172.21.15.42:0/2869255958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.200+0000 7f59767fc640 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d292ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f596ffff640 1 -- 172.21.15.42:0/2720362988 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59600efa30 con 0x7f5978105340 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f59777fe640 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d287690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f5976ffd640 1 -- 172.21.15.42:0/2869255958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 msgr2=0x56144d287690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d287690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f5976ffd640 1 -- 172.21.15.42:0/2869255958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 msgr2=0x56144d292ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d292ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f5976ffd640 1 -- 172.21.15.42:0/2869255958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144d1ad570 con 0x56144d287bd0 2024-03-12T04:22:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f59777fe640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1ba0 0x7f59640a4060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5968005cc0 tx=0x7f5968013ac0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d290270 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f597000cbf0 tx=0x7f597000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67afefb20085120e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f596d7fa640 1 -- 172.21.15.42:0/2869255958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970010430 con 0x56144d287bd0 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f596d7fa640 1 -- 172.21.15.42:0/2869255958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5970010a40 con 0x56144d287bd0 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.201+0000 7f596d7fa640 1 -- 172.21.15.42:0/2869255958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970019960 con 0x56144d287bd0 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 -- 172.21.15.42:0/2869255958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 msgr2=0x56144d290270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d290270 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f597000cbf0 tx=0x7f597000e350 comp rx=0 tx=0).stop 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 -- 172.21.15.42:0/2869255958 shutdown_connections 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d292ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d290270 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 --2- 172.21.15.42:0/2869255958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d287690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 -- 172.21.15.42:0/2869255958 >> 172.21.15.42:0/2869255958 conn(0x56144d281990 msgr2=0x56144d281d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 -- 172.21.15.42:0/2869255958 shutdown_connections 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 -- 172.21.15.42:0/2869255958 wait complete. 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 -- start start 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.202+0000 7f598017f740 1 Processor -- start 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d290210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d28d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2907b0 0x56144d28dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144cec2430 con 0x56144d2907b0 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ce7ee00 con 0x56144d287bd0 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d173fd0 con 0x56144d2872b0 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 10 client.0 _ll_register_callbacks cb 0x56144d1c9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2907b0 0x56144d28dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2907b0 0x56144d28dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42498/0 (socket says 172.21.15.42:42498) 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 -- 172.21.15.42:0/2728440022 learned_addr learned my addr 172.21.15.42:0/2728440022 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59767fc640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d28d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d290210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 -- 172.21.15.42:0/2728440022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 msgr2=0x56144d290210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d290210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 -- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 msgr2=0x56144d28d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d28d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 -- 172.21.15.42:0/2728440022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597000c850 con 0x56144d2907b0 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59767fc640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d28d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d290210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f59777fe640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2907b0 0x56144d28dcc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f5968011ce0 tx=0x7f596800a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cb45b5256fca17 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f595e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f595e7fc640 1 -- 172.21.15.42:0/2728440022 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968012ea0 con 0x56144d2907b0 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f595e7fc640 1 -- 172.21.15.42:0/2728440022 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5968003970 con 0x56144d2907b0 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56144ceee7e0 con 0x56144d2907b0 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f595e7fc640 1 -- 172.21.15.42:0/2728440022 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5968003b30 con 0x56144d2907b0 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 10 client.17517 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.203+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56144d1ed610 con 0x56144d2907b0 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f595e7fc640 1 -- 172.21.15.42:0/2728440022 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56144ceee7e0 con 0x56144d2907b0 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f595e7fc640 1 -- 172.21.15.42:0/2728440022 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f596801f3f0 con 0x56144d2907b0 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f598017f740 10 client.17517 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f598017f740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f598017f740 20 client.17517 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f598017f740 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2a0970 0x56144d2a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f598017f740 4 client.17517 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:08.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56144cf2b720 con 0x56144d2a0970 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.204+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2a0970 0x56144d2a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.205+0000 7f5976ffd640 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2a0970 0x56144d2a2d90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18fb34ddc4e8877 server_cookie=466e4acccc299374 in_seq=0 out_seq=0 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.205+0000 7f595e7fc640 10 client.17517 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.205+0000 7f595e7fc640 1 -- 172.21.15.42:0/2728440022 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56144cf2b720 con 0x56144d2a0970 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.205+0000 7f595e7fc640 10 client.17517 handle_command_reply: tid=0 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.205+0000 7f598017f740 1 client.17517 shutdown 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2907b0 msgr2=0x56144d28dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2907b0 0x56144d28dcc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f5968011ce0 tx=0x7f596800a4d0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 shutdown_connections 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2a0970 0x56144d2a2d90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2907b0 0x56144d28dcc0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d28d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f595e7fc640 0 client.17517 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 --2- 172.21.15.42:0/2728440022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d290210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 >> 172.21.15.42:0/2728440022 conn(0x56144d281990 msgr2=0x56144d283730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 shutdown_connections 2024-03-12T04:22:08.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 1 -- 172.21.15.42:0/2728440022 wait complete. 2024-03-12T04:22:08.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.206+0000 7f598017f740 20 client.17517 trim_cache size 0 max 16384 2024-03-12T04:22:08.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 Processor -- start 2024-03-12T04:22:08.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 -- start start 2024-03-12T04:22:08.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2872b0 0x56144d28d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d297660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2907b0 0x56144d297ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d2817f0 con 0x56144d2872b0 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d29c570 con 0x56144d287bd0 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f598017f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d28e2c0 con 0x56144d2907b0 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2872b0 0x56144d28d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2872b0 0x56144d28d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42514/0 (socket says 172.21.15.42:42514) 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 -- 172.21.15.42:0/3535440096 learned_addr learned my addr 172.21.15.42:0/3535440096 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 -- 172.21.15.42:0/3535440096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2907b0 msgr2=0x56144d297ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 --2- 172.21.15.42:0/3535440096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2907b0 0x56144d297ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 -- 172.21.15.42:0/3535440096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 msgr2=0x56144d297660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 --2- 172.21.15.42:0/3535440096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d297660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 -- 172.21.15.42:0/3535440096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144d189ac0 con 0x56144d2872b0 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.221+0000 7f5976ffd640 1 --2- 172.21.15.42:0/3535440096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2872b0 0x56144d28d540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f59700195b0 tx=0x7f59700195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149026c7b583e48f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f596cff9640 1 -- 172.21.15.42:0/3535440096 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970021070 con 0x56144d2872b0 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f596cff9640 1 -- 172.21.15.42:0/3535440096 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59700044d0 con 0x56144d2872b0 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f596cff9640 1 -- 172.21.15.42:0/3535440096 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970025a90 con 0x56144d2872b0 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 -- 172.21.15.42:0/3535440096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2872b0 msgr2=0x56144d28d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 --2- 172.21.15.42:0/3535440096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2872b0 0x56144d28d540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f59700195b0 tx=0x7f59700195e0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 -- 172.21.15.42:0/3535440096 shutdown_connections 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 --2- 172.21.15.42:0/3535440096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2907b0 0x56144d297ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 --2- 172.21.15.42:0/3535440096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d287bd0 0x56144d297660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 --2- 172.21.15.42:0/3535440096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d2872b0 0x56144d28d540 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 -- 172.21.15.42:0/3535440096 >> 172.21.15.42:0/3535440096 conn(0x56144d281990 msgr2=0x56144d28ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 -- 172.21.15.42:0/3535440096 shutdown_connections 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.222+0000 7f598017f740 1 -- 172.21.15.42:0/3535440096 wait complete. 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 -- start start 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 Processor -- start 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d296390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d2968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d296e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d2f7060 con 0x56144d287bd0 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d2f71d0 con 0x56144d2907b0 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144d2f7390 con 0x56144d2872b0 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f598017f740 10 client.0 _ll_register_callbacks cb 0x7f597000c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d2968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d2968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42530/0 (socket says 172.21.15.42:42530) 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 -- 172.21.15.42:0/1084993296 learned_addr learned my addr 172.21.15.42:0/1084993296 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f5976ffd640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d296390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59777fe640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d296e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 -- 172.21.15.42:0/1084993296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 msgr2=0x56144d296390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d296390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 -- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 msgr2=0x56144d296e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d296e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59767fc640 1 -- 172.21.15.42:0/1084993296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56144d1ad570 con 0x56144d287bd0 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.223+0000 7f59777fe640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d296e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f59767fc640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d2968d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f59600ed340 tx=0x7f5960009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73edaa46a0d70b09 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f595e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f595e7fc640 1 -- 172.21.15.42:0/1084993296 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59600bc120 con 0x56144d287bd0 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f595e7fc640 1 -- 172.21.15.42:0/1084993296 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59600bcec0 con 0x56144d287bd0 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5970010a40 con 0x56144d287bd0 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f595e7fc640 1 -- 172.21.15.42:0/1084993296 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59600bc7c0 con 0x56144d287bd0 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 10 client.17523 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56144d1af6a0 con 0x56144d287bd0 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f595e7fc640 1 -- 172.21.15.42:0/1084993296 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59600d5020 con 0x56144d287bd0 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f595e7fc640 1 -- 172.21.15.42:0/1084993296 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f596003c070 con 0x56144d287bd0 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 10 client.17523 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 20 client.17523 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2f8a80 0x56144d2faea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 4 client.17523 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56144cf7a890 con 0x56144d2f8a80 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.224+0000 7f5976ffd640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2f8a80 0x56144d2faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f5976ffd640 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2f8a80 0x56144d2faea0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b3acfd59760e9d server_cookie=2c956feedb7a591e in_seq=0 out_seq=0 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f595e7fc640 10 client.17523 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f595e7fc640 1 -- 172.21.15.42:0/1084993296 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56144cf7a890 con 0x56144d2f8a80 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f595e7fc640 10 client.17523 handle_command_reply: tid=0 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 client.17523 shutdown 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 msgr2=0x56144d2968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d2968d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f59600ed340 tx=0x7f5960009260 comp rx=0 tx=0).stop 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 shutdown_connections 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56144d2f8a80 0x56144d2faea0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56144d2907b0 0x56144d296e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144d287bd0 0x56144d2968d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f595e7fc640 0 client.17523 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 --2- 172.21.15.42:0/1084993296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56144d2872b0 0x56144d296390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 >> 172.21.15.42:0/1084993296 conn(0x56144d281990 msgr2=0x56144d283680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 shutdown_connections 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 1 -- 172.21.15.42:0/1084993296 wait complete. 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.225+0000 7f598017f740 20 client.17523 trim_cache size 0 max 16384 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:22:08.236 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:22:08.237 INFO:teuthology.orchestra.run.smithi042.stdout: "d882ce4b-803f-4e3f-9ca6-90692ea8239e": { 2024-03-12T04:22:08.237 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:22:08.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "d882ce4b-803f-4e3f-9ca6-90692ea8239e", 2024-03-12T04:22:08.237 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:22:08.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:22:08.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:22:08.237 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1ba0 msgr2=0x7f59640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1ba0 0x7f59640a4060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5968005cc0 tx=0x7f5968013ac0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105340 msgr2=0x7f5978198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105340 0x7f5978198ad0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5960005ab0 tx=0x7f5960004270 comp rx=0 tx=0).stop 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 shutdown_connections 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59640a1ba0 0x7f59640a4060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5978199010 0x7f59781a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5978105340 0x7f5978198ad0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 --2- 172.21.15.42:0/2720362988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5978100270 0x7f5978198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:08.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 >> 172.21.15.42:0/2720362988 conn(0x7f5978077230 msgr2=0x7f59780fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:08.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 shutdown_connections 2024-03-12T04:22:08.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:08.226+0000 7f597d4d4640 1 -- 172.21.15.42:0/2720362988 wait complete. 2024-03-12T04:22:08.240 INFO:tasks.cephfs.filesystem:scrub status for tag:d882ce4b-803f-4e3f-9ca6-90692ea8239e - {'path': '/', 'tag': 'd882ce4b-803f-4e3f-9ca6-90692ea8239e', 'options': 'recursive,force'} 2024-03-12T04:22:08.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:08 smithi042 ceph-mon[32260]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:22:08.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:08 smithi119 ceph-mon[36119]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:22:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:08 smithi136 ceph-mon[35571]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:22:09.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:09 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:22:09.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:09 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:09.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:09 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:22:09.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:09 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:09.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:09 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:22:09.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:09 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:22:09.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:09 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:09.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:09 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:22:09.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:09 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:09.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:09 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:22:09.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:09 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:22:09.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:09 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:09.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:09 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:22:09.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:09 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:09.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:09 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:22:10.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:10 smithi119 ceph-mon[36119]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:22:10.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:10 smithi119 ceph-mon[36119]: osdmap e91: 12 total, 12 up, 12 in 2024-03-12T04:22:10.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:10 smithi136 ceph-mon[35571]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:22:10.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:10 smithi136 ceph-mon[35571]: osdmap e91: 12 total, 12 up, 12 in 2024-03-12T04:22:10.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:10 smithi042 ceph-mon[32260]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:22:10.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:10 smithi042 ceph-mon[32260]: osdmap e91: 12 total, 12 up, 12 in 2024-03-12T04:22:12.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:12 smithi119 ceph-mon[36119]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:22:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:12 smithi136 ceph-mon[35571]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:22:12.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:12 smithi042 ceph-mon[32260]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:22:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:14 smithi119 ceph-mon[36119]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:22:14.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:14 smithi136 ceph-mon[35571]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:22:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:14 smithi042 ceph-mon[32260]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:22:16.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:16 smithi119 ceph-mon[36119]: osdmap e92: 12 total, 12 up, 12 in 2024-03-12T04:22:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:16 smithi136 ceph-mon[35571]: osdmap e92: 12 total, 12 up, 12 in 2024-03-12T04:22:16.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:16 smithi042 ceph-mon[32260]: osdmap e92: 12 total, 12 up, 12 in 2024-03-12T04:22:17.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:17 smithi119 ceph-mon[36119]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:22:17.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:17 smithi136 ceph-mon[35571]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:22:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:17 smithi042 ceph-mon[32260]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:22:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:19 smithi119 ceph-mon[36119]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-12T04:22:19.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:19 smithi136 ceph-mon[35571]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-12T04:22:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:19 smithi042 ceph-mon[32260]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-12T04:22:21.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:21 smithi119 ceph-mon[36119]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 207 B/s wr, 10 op/s 2024-03-12T04:22:21.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:21 smithi136 ceph-mon[35571]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 207 B/s wr, 10 op/s 2024-03-12T04:22:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:21 smithi042 ceph-mon[32260]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 207 B/s wr, 10 op/s 2024-03-12T04:22:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:23 smithi119 ceph-mon[36119]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:22:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:23 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:22:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:23 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:22:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:23 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:22:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:23 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:22:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:23 smithi136 ceph-mon[35571]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:22:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:23 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:22:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:23 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:22:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:23 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:22:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:23 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:22:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:23 smithi042 ceph-mon[32260]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:22:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:23 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:22:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:23 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:22:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:23 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:22:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:23 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:22:25.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:25 smithi119 ceph-mon[36119]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:22:25.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:25 smithi136 ceph-mon[35571]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:22:25.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:25 smithi042 ceph-mon[32260]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:22:27.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:27 smithi119 ceph-mon[36119]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 9 op/s 2024-03-12T04:22:27.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:27 smithi136 ceph-mon[35571]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 9 op/s 2024-03-12T04:22:27.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:27 smithi042 ceph-mon[32260]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 101 B/s wr, 9 op/s 2024-03-12T04:22:29.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:29 smithi119 ceph-mon[36119]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:22:29.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:29 smithi136 ceph-mon[35571]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:22:29.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:29 smithi042 ceph-mon[32260]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:22:31.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:31 smithi119 ceph-mon[36119]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:31.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:31 smithi136 ceph-mon[35571]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:31.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:31 smithi042 ceph-mon[32260]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:33.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:33 smithi119 ceph-mon[36119]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:33.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:33 smithi136 ceph-mon[35571]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:33.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:33 smithi042 ceph-mon[32260]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:34.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:34 smithi042 ceph-mon[32260]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:34.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:34 smithi119 ceph-mon[36119]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:34 smithi136 ceph-mon[35571]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:36.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:36 smithi119 ceph-mon[36119]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:36 smithi136 ceph-mon[35571]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:36.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:36 smithi042 ceph-mon[32260]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:38.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:22:38.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:22:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 -- 172.21.15.42:0/3768792978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 msgr2=0x7f02dc10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/3768792978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc10cf50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009ca0 tx=0x7f02d402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 -- 172.21.15.42:0/3768792978 shutdown_connections 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/3768792978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc10cf50 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/3768792978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02dc100c60 0x7f02dc104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/3768792978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02dc1002b0 0x7f02dc100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 -- 172.21.15.42:0/3768792978 >> 172.21.15.42:0/3768792978 conn(0x7f02dc077190 msgr2=0x7f02dc0775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 -- 172.21.15.42:0/3768792978 shutdown_connections 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.356+0000 7f02e3b4b640 1 -- 172.21.15.42:0/3768792978 wait complete. 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 Processor -- start 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 -- start start 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02dc1002b0 0x7f02dc19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02dc199660 0x7f02dc199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc19a050 con 0x7f02dc105380 2024-03-12T04:22:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc19a1c0 con 0x7f02dc1002b0 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e3b4b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc19a380 con 0x7f02dc199660 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e10e6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e18e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02dc1002b0 0x7f02dc19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e10e6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc19f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53426/0 (socket says 172.21.15.42:53426) 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e10e6640 1 -- 172.21.15.42:0/201839719 learned_addr learned my addr 172.21.15.42:0/201839719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.357+0000 7f02e20e8640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02dc199660 0x7f02dc199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e10e6640 1 -- 172.21.15.42:0/201839719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02dc199660 msgr2=0x7f02dc199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e10e6640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02dc199660 0x7f02dc199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e10e6640 1 -- 172.21.15.42:0/201839719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02dc1002b0 msgr2=0x7f02dc19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e10e6640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02dc1002b0 0x7f02dc19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e10e6640 1 -- 172.21.15.42:0/201839719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4009900 con 0x7f02dc105380 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e18e7640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02dc1002b0 0x7f02dc19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e10e6640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc19f9a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00b450 tx=0x7f02cc00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02d2ffd640 1 -- 172.21.15.42:0/201839719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc011830 con 0x7f02dc105380 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02d2ffd640 1 -- 172.21.15.42:0/201839719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02cc011e40 con 0x7f02dc105380 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc1a6320 con 0x7f02dc105380 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02d2ffd640 1 -- 172.21.15.42:0/201839719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00f2e0 con 0x7f02dc105380 2024-03-12T04:22:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.358+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc1a67e0 con 0x7f02dc105380 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.360+0000 7f02d2ffd640 1 -- 172.21.15.42:0/201839719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f02cc011990 con 0x7f02dc105380 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 Processor -- start 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 -- start start 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23d2b0 0x55b4fd23d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd246270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02d2ffd640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02c80a1ba0 0x7f02c80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd2467b0 0x55b4fd248ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd1ac280 con 0x55b4fd2467b0 2024-03-12T04:22:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd1a97b0 con 0x55b4fd23dbd0 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e67f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd1a4f90 con 0x55b4fd23d2b0 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd2467b0 0x55b4fd248ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e10e6640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02c80a1ba0 0x7f02c80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd2467b0 0x55b4fd248ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53438/0 (socket says 172.21.15.42:53438) 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e20e8640 1 -- 172.21.15.42:0/17843335 learned_addr learned my addr 172.21.15.42:0/17843335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.361+0000 7f02e18e7640 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd246270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e20e8640 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23d2b0 0x55b4fd23d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02d2ffd640 1 -- 172.21.15.42:0/201839719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f02cc0cd2c0 con 0x7f02dc105380 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e18e7640 1 -- 172.21.15.42:0/17843335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23d2b0 msgr2=0x55b4fd23d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e18e7640 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23d2b0 0x55b4fd23d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e18e7640 1 -- 172.21.15.42:0/17843335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd2467b0 msgr2=0x55b4fd248ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e18e7640 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd2467b0 0x55b4fd248ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e18e7640 1 -- 172.21.15.42:0/17843335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4fd163570 con 0x55b4fd23dbd0 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e20e8640 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd2467b0 0x55b4fd248ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e10e6640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02c80a1ba0 0x7f02c80a4060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f02cc099cc0 tx=0x7f02cc06c1c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02e18e7640 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd246270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02d800afc0 tx=0x7f02d800f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68338522fc07183f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02c7fff640 1 -- 172.21.15.42:0/17843335 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8005220 con 0x55b4fd23dbd0 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02c7fff640 1 -- 172.21.15.42:0/17843335 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02d80047a0 con 0x55b4fd23dbd0 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.362+0000 7f02c7fff640 1 -- 172.21.15.42:0/17843335 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d80109e0 con 0x55b4fd23dbd0 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 -- 172.21.15.42:0/17843335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 msgr2=0x55b4fd246270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd246270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02d800afc0 tx=0x7f02d800f850 comp rx=0 tx=0).stop 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 -- 172.21.15.42:0/17843335 shutdown_connections 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd2467b0 0x55b4fd248ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd246270 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 --2- 172.21.15.42:0/17843335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23d2b0 0x55b4fd23d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 -- 172.21.15.42:0/17843335 >> 172.21.15.42:0/17843335 conn(0x55b4fd237990 msgr2=0x55b4fd237d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 -- 172.21.15.42:0/17843335 shutdown_connections 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.363+0000 7f02e67f1740 1 -- 172.21.15.42:0/17843335 wait complete. 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 -- start start 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 Processor -- start 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd246160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd243560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd2467b0 0x55b4fd243b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fce78430 con 0x55b4fd23d2b0 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fce34e00 con 0x55b4fd23dbd0 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd129fd0 con 0x55b4fd2467b0 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e67f1740 10 client.0 _ll_register_callbacks cb 0x55b4fd17ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:38.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd246160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd246160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53446/0 (socket says 172.21.15.42:53446) 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e20e8640 1 -- 172.21.15.42:0/2868298766 learned_addr learned my addr 172.21.15.42:0/2868298766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e18e7640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd243560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e20e8640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd2467b0 0x55b4fd243b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e20e8640 1 -- 172.21.15.42:0/2868298766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd2467b0 msgr2=0x55b4fd243b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.364+0000 7f02e20e8640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd2467b0 0x55b4fd243b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e20e8640 1 -- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 msgr2=0x55b4fd243560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e20e8640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd243560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e20e8640 1 -- 172.21.15.42:0/2868298766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d800abf0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e18e7640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd243560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e20e8640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd246160 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f02d402c630 tx=0x7f02d402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7b57cf1798ad8e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02c57fa640 1 -- 172.21.15.42:0/2868298766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40329e0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02c57fa640 1 -- 172.21.15.42:0/2868298766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f02d4003970 con 0x55b4fd23d2b0 2024-03-12T04:22:38.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4fcea47e0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02c57fa640 1 -- 172.21.15.42:0/2868298766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d4003b30 con 0x55b4fd23d2b0 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 10 client.17535 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b4fd1a3610 con 0x55b4fd23d2b0 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02c57fa640 1 -- 172.21.15.42:0/2868298766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02d4052020 con 0x55b4fd23d2b0 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02c57fa640 1 -- 172.21.15.42:0/2868298766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02d4042420 con 0x55b4fd23d2b0 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 10 client.17535 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 20 client.17535 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd256930 0x55b4fd258d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 4 client.17535 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.365+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4fcee1720 con 0x55b4fd256930 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.366+0000 7f02e18e7640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd256930 0x55b4fd258d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.366+0000 7f02e18e7640 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd256930 0x55b4fd258d50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6808c3608e848e5a server_cookie=d8213037f548065f in_seq=0 out_seq=0 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.366+0000 7f02c57fa640 10 client.17535 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02c57fa640 1 -- 172.21.15.42:0/2868298766 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4fcee1720 con 0x55b4fd256930 2024-03-12T04:22:38.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02c57fa640 10 client.17535 handle_command_reply: tid=0 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 client.17535 shutdown 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 msgr2=0x55b4fd246160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd246160 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f02d402c630 tx=0x7f02d402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 shutdown_connections 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd256930 0x55b4fd258d50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd2467b0 0x55b4fd243b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd23dbd0 0x55b4fd243560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 --2- 172.21.15.42:0/2868298766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd246160 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02c57fa640 0 client.17535 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 >> 172.21.15.42:0/2868298766 conn(0x55b4fd237990 msgr2=0x55b4fd239730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 shutdown_connections 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 1 -- 172.21.15.42:0/2868298766 wait complete. 2024-03-12T04:22:38.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.367+0000 7f02e67f1740 20 client.17535 trim_cache size 0 max 16384 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.382+0000 7f02e67f1740 1 Processor -- start 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.382+0000 7f02e67f1740 1 -- start start 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.382+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd245c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.382+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd241e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.382+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 0x55b4fd242370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.382+0000 7f02e67f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd253460 con 0x55b4fd23d2b0 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e67f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd23ca80 con 0x55b4fd2467b0 2024-03-12T04:22:38.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e67f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd2428b0 con 0x55b4fd23dbd0 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd245c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd245c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53462/0 (socket says 172.21.15.42:53462) 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 -- 172.21.15.42:0/1982959937 learned_addr learned my addr 172.21.15.42:0/1982959937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 -- 172.21.15.42:0/1982959937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 msgr2=0x55b4fd241e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 --2- 172.21.15.42:0/1982959937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd241e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 -- 172.21.15.42:0/1982959937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 msgr2=0x55b4fd242370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 --2- 172.21.15.42:0/1982959937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 0x55b4fd242370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 -- 172.21.15.42:0/1982959937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4fd13fac0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02e20e8640 1 --2- 172.21.15.42:0/1982959937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd245c90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f02d4034f10 tx=0x7f02d4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557f87adf94ca098 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02c77fe640 1 -- 172.21.15.42:0/1982959937 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4042420 con 0x55b4fd23d2b0 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02c77fe640 1 -- 172.21.15.42:0/1982959937 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02d4052040 con 0x55b4fd23d2b0 2024-03-12T04:22:38.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.383+0000 7f02c77fe640 1 -- 172.21.15.42:0/1982959937 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d404e690 con 0x55b4fd23d2b0 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 -- 172.21.15.42:0/1982959937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 msgr2=0x55b4fd245c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 --2- 172.21.15.42:0/1982959937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd245c90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f02d4034f10 tx=0x7f02d4009800 comp rx=0 tx=0).stop 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 -- 172.21.15.42:0/1982959937 shutdown_connections 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 --2- 172.21.15.42:0/1982959937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 0x55b4fd242370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 --2- 172.21.15.42:0/1982959937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd241e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 --2- 172.21.15.42:0/1982959937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd245c90 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 -- 172.21.15.42:0/1982959937 >> 172.21.15.42:0/1982959937 conn(0x55b4fd237990 msgr2=0x55b4fd23e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 -- 172.21.15.42:0/1982959937 shutdown_connections 2024-03-12T04:22:38.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 -- 172.21.15.42:0/1982959937 wait complete. 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 -- start start 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.384+0000 7f02e67f1740 1 Processor -- start 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd24a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd24a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 0x55b4fd2ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd24b270 con 0x55b4fd23d2b0 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd2ad370 con 0x55b4fd2467b0 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fd2ad4b0 con 0x55b4fd23dbd0 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 10 client.0 _ll_register_callbacks cb 0x7f02d800a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd24a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd24a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53478/0 (socket says 172.21.15.42:53478) 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 -- 172.21.15.42:0/3308573455 learned_addr learned my addr 172.21.15.42:0/3308573455 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e18e7640 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd24a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 -- 172.21.15.42:0/3308573455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 msgr2=0x55b4fd24a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd24a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 -- 172.21.15.42:0/3308573455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 msgr2=0x55b4fd2ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 0x55b4fd2ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 -- 172.21.15.42:0/3308573455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4fd1a3610 con 0x55b4fd23d2b0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e18e7640 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd24a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e20e8640 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd24a430 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f02d4032e90 tx=0x7f02d4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac87c99b5b9f15d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02c57fa640 1 -- 172.21.15.42:0/3308573455 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4042bf0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02c57fa640 1 -- 172.21.15.42:0/3308573455 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f02d4053080 con 0x55b4fd23d2b0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.385+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d80047a0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02c57fa640 1 -- 172.21.15.42:0/3308573455 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d4042d50 con 0x55b4fd23d2b0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 10 client.17541 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b4fd1656a0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02c57fa640 1 -- 172.21.15.42:0/3308573455 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02d80047a0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02c57fa640 1 -- 172.21.15.42:0/3308573455 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02d40045e0 con 0x55b4fd23d2b0 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 10 client.17541 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 20 client.17541 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd2ae710 0x55b4fd2b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 4 client.17541 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b4fcf30890 con 0x55b4fd2ae710 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.386+0000 7f02e18e7640 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd2ae710 0x55b4fd2b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e18e7640 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd2ae710 0x55b4fd2b0b00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ceda3a27782161 server_cookie=5d28ae27bee235b8 in_seq=0 out_seq=0 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02c57fa640 10 client.17541 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02c57fa640 1 -- 172.21.15.42:0/3308573455 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b4fcf30890 con 0x55b4fd2ae710 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02c57fa640 10 client.17541 handle_command_reply: tid=0 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 client.17541 shutdown 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 msgr2=0x55b4fd24a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd24a430 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f02d4032e90 tx=0x7f02d4005bd0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 shutdown_connections 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55b4fd2ae710 0x55b4fd2b0b00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4fd2467b0 0x55b4fd2ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b4fd23dbd0 0x55b4fd24a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02c57fa640 0 client.17541 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 --2- 172.21.15.42:0/3308573455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fd23d2b0 0x55b4fd24a430 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 >> 172.21.15.42:0/3308573455 conn(0x55b4fd237990 msgr2=0x55b4fd23e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 shutdown_connections 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 1 -- 172.21.15.42:0/3308573455 wait complete. 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.387+0000 7f02e67f1740 20 client.17541 trim_cache size 0 max 16384 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:22:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:22:38.397 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:22:38.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02c80a1ba0 msgr2=0x7f02c80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02c80a1ba0 0x7f02c80a4060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f02cc099cc0 tx=0x7f02cc06c1c0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 msgr2=0x7f02dc19f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc19f9a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00b450 tx=0x7f02cc00b850 comp rx=0 tx=0).stop 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 shutdown_connections 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f02c80a1ba0 0x7f02c80a4060 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02dc199660 0x7f02dc199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02dc105380 0x7f02dc19f9a0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 --2- 172.21.15.42:0/201839719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02dc1002b0 0x7f02dc19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 >> 172.21.15.42:0/201839719 conn(0x7f02dc077190 msgr2=0x7f02dc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 shutdown_connections 2024-03-12T04:22:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.388+0000 7f02e3b4b640 1 -- 172.21.15.42:0/201839719 wait complete. 2024-03-12T04:22:38.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:22:38.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:22:38.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 -- 172.21.15.42:0/3007326255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc069aa0 msgr2=0x7f58bc069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3007326255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc069aa0 0x7f58bc069e80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009cd0 tx=0x7f58b402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 -- 172.21.15.42:0/3007326255 shutdown_connections 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3007326255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58bc1064a0 0x7f58bc10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3007326255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc06a450 0x7f58bc105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3007326255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc069aa0 0x7f58bc069e80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 -- 172.21.15.42:0/3007326255 >> 172.21.15.42:0/3007326255 conn(0x7f58bc077120 msgr2=0x7f58bc077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 -- 172.21.15.42:0/3007326255 shutdown_connections 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 -- 172.21.15.42:0/3007326255 wait complete. 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 Processor -- start 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.502+0000 7f58c2d20640 1 -- start start 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c2d20640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58bc069aa0 0x7f58bc197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c2d20640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc1064a0 0x7f58bc197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c2d20640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc19c020 0x7f58bc19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c2d20640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc19ca10 con 0x7f58bc19c020 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c2d20640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc19cb80 con 0x7f58bc1064a0 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c2d20640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc19ccf0 con 0x7f58bc069aa0 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc19c020 0x7f58bc19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc19c020 0x7f58bc19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53486/0 (socket says 172.21.15.42:53486) 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 -- 172.21.15.42:0/3265845515 learned_addr learned my addr 172.21.15.42:0/3265845515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c0abc640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58bc069aa0 0x7f58bc197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58bbfff640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc1064a0 0x7f58bc197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 -- 172.21.15.42:0/3265845515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58bc069aa0 msgr2=0x7f58bc197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58bc069aa0 0x7f58bc197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 -- 172.21.15.42:0/3265845515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc1064a0 msgr2=0x7f58bc197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc1064a0 0x7f58bc197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 -- 172.21.15.42:0/3265845515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4009900 con 0x7f58bc19c020 2024-03-12T04:22:38.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c12bd640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc19c020 0x7f58bc19c4d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e970 tx=0x7f58ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58b9ffb640 1 -- 172.21.15.42:0/3265845515 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac009720 con 0x7f58bc19c020 2024-03-12T04:22:38.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58b9ffb640 1 -- 172.21.15.42:0/3265845515 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58ac0054d0 con 0x7f58bc19c020 2024-03-12T04:22:38.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc1a42d0 con 0x7f58bc19c020 2024-03-12T04:22:38.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.503+0000 7f58b9ffb640 1 -- 172.21.15.42:0/3265845515 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac010430 con 0x7f58bc19c020 2024-03-12T04:22:38.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.504+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1a47d0 con 0x7f58bc19c020 2024-03-12T04:22:38.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.505+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc0683c0 con 0x7f58bc19c020 2024-03-12T04:22:38.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.505+0000 7f58b9ffb640 1 -- 172.21.15.42:0/3265845515 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f58ac004f10 con 0x7f58bc19c020 2024-03-12T04:22:38.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.506+0000 7f58b9ffb640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58a80a1c20 0x7f58a80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.506+0000 7f58c0abc640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58a80a1c20 0x7f58a80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.506+0000 7f58b9ffb640 1 -- 172.21.15.42:0/3265845515 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f58ac014070 con 0x7f58bc19c020 2024-03-12T04:22:38.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.507+0000 7f58c0abc640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58a80a1c20 0x7f58a80a40e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58b4004af0 tx=0x7f58b4005c50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.511+0000 7f58b9ffb640 1 -- 172.21.15.42:0/3265845515 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58ac096980 con 0x7f58bc19c020 2024-03-12T04:22:38.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:38 smithi042 ceph-mon[32260]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:38.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:38 smithi119 ceph-mon[36119]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:38.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:38 smithi136 ceph-mon[35571]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:38.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.729+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58bc109730 con 0x7f58bc19c020 2024-03-12T04:22:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.729+0000 7f58b9ffb640 1 -- 172.21.15.42:0/3265845515 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f58bc109730 con 0x7f58bc19c020 2024-03-12T04:22:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:22:38.732 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:22:38.733 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:22:38.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58a80a1c20 msgr2=0x7f58a80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58a80a1c20 0x7f58a80a40e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58b4004af0 tx=0x7f58b4005c50 comp rx=0 tx=0).stop 2024-03-12T04:22:38.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc19c020 msgr2=0x7f58bc19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc19c020 0x7f58bc19c4d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e970 tx=0x7f58ac00ed70 comp rx=0 tx=0).stop 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 shutdown_connections 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f58a80a1c20 0x7f58a80a40e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58bc19c020 0x7f58bc19c4d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc1064a0 0x7f58bc197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 --2- 172.21.15.42:0/3265845515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58bc069aa0 0x7f58bc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 >> 172.21.15.42:0/3265845515 conn(0x7f58bc077120 msgr2=0x7f58bc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 shutdown_connections 2024-03-12T04:22:38.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.731+0000 7f58c2d20640 1 -- 172.21.15.42:0/3265845515 wait complete. 2024-03-12T04:22:38.745 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:38.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:22:38.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 -- 172.21.15.42:0/3880437361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100cc0 msgr2=0x7fa66c104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 --2- 172.21.15.42:0/3880437361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100cc0 0x7fa66c104d00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009900 tx=0x7fa65c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:38.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 -- 172.21.15.42:0/3880437361 shutdown_connections 2024-03-12T04:22:38.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 --2- 172.21.15.42:0/3880437361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa66c1053e0 0x7fa66c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 --2- 172.21.15.42:0/3880437361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100cc0 0x7fa66c104d00 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 --2- 172.21.15.42:0/3880437361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c100310 0x7fa66c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 -- 172.21.15.42:0/3880437361 >> 172.21.15.42:0/3880437361 conn(0x7fa66c0771e0 msgr2=0x7fa66c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 -- 172.21.15.42:0/3880437361 shutdown_connections 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.849+0000 7fa674639640 1 -- 172.21.15.42:0/3880437361 wait complete. 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 Processor -- start 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 -- start start 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100310 0x7fa66c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1053e0 0x7fa66c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa66c19bfd0 0x7fa66c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c19c9c0 con 0x7fa66c100310 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c19cb30 con 0x7fa66c19bfd0 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa674639640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c19cca0 con 0x7fa66c1053e0 2024-03-12T04:22:38.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100310 0x7fa66c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100310 0x7fa66c197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53508/0 (socket says 172.21.15.42:53508) 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 -- 172.21.15.42:0/4090610225 learned_addr learned my addr 172.21.15.42:0/4090610225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa672bd6640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa66c19bfd0 0x7fa66c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 -- 172.21.15.42:0/4090610225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1053e0 msgr2=0x7fa66c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1053e0 0x7fa66c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 -- 172.21.15.42:0/4090610225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa66c19bfd0 msgr2=0x7fa66c19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa66c19bfd0 0x7fa66c19c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 -- 172.21.15.42:0/4090610225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c009560 con 0x7fa66c100310 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa672bd6640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa66c19bfd0 0x7fa66c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.850+0000 7fa6723d5640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100310 0x7fa66c197760 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c970 tx=0x7fa66800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.851+0000 7fa6637fe640 1 -- 172.21.15.42:0/4090610225 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668007bb0 con 0x7fa66c100310 2024-03-12T04:22:38.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.851+0000 7fa6637fe640 1 -- 172.21.15.42:0/4090610225 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa668007d10 con 0x7fa66c100310 2024-03-12T04:22:38.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.851+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c1a4280 con 0x7fa66c100310 2024-03-12T04:22:38.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.851+0000 7fa6637fe640 1 -- 172.21.15.42:0/4090610225 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668004e80 con 0x7fa66c100310 2024-03-12T04:22:38.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.851+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c1a4730 con 0x7fa66c100310 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.853+0000 7fa6637fe640 1 -- 172.21.15.42:0/4090610225 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa668005020 con 0x7fa66c100310 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.853+0000 7fa677209740 1 Processor -- start 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa677209740 1 -- start start 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa677209740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf76630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa677209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf7f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa677209740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf81b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa677209740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedf31960 con 0x557bedf7f750 2024-03-12T04:22:38.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa677209740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedee5310 con 0x557bedf76250 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa677209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedee2840 con 0x557bedf76b70 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa6637fe640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa6580a1c20 0x7fa6580a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf81b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa672bd6640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa6580a1c20 0x7fa6580a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf81b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53514/0 (socket says 172.21.15.42:53514) 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 -- 172.21.15.42:0/524690641 learned_addr learned my addr 172.21.15.42:0/524690641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa6723d5640 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf7f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa672bd6640 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf76630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa6637fe640 1 -- 172.21.15.42:0/4090610225 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa6680cd590 con 0x7fa66c100310 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 -- 172.21.15.42:0/524690641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 msgr2=0x557bedf7f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf7f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 -- 172.21.15.42:0/524690641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 msgr2=0x557bedf76630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf76630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa671bd4640 1 -- 172.21.15.42:0/524690641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bedea0060 con 0x557bedf7f750 2024-03-12T04:22:38.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.854+0000 7fa672bd6640 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf76630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.855+0000 7fa672bd6640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa6580a1c20 0x7fa6580a40e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa664005c40 tx=0x7fa664009d50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.855+0000 7fa671bd4640 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf81b40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa65c02c170 tx=0x7fa65c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472db3f0764b64a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.855+0000 7fa660bfe640 1 -- 172.21.15.42:0/524690641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0047c0 con 0x557bedf7f750 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.855+0000 7fa660bfe640 1 -- 172.21.15.42:0/524690641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa65c02de90 con 0x557bedf7f750 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.855+0000 7fa660bfe640 1 -- 172.21.15.42:0/524690641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0035f0 con 0x557bedf7f750 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.855+0000 7fa677209740 1 -- 172.21.15.42:0/524690641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 msgr2=0x557bedf81b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.855+0000 7fa677209740 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf81b40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa65c02c170 tx=0x7fa65c005dd0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- 172.21.15.42:0/524690641 shutdown_connections 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf81b40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf7f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 --2- 172.21.15.42:0/524690641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf76630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- 172.21.15.42:0/524690641 >> 172.21.15.42:0/524690641 conn(0x557bedf70980 msgr2=0x557bedf70d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- 172.21.15.42:0/524690641 shutdown_connections 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- 172.21.15.42:0/524690641 wait complete. 2024-03-12T04:22:38.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- start start 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 Processor -- start 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf76250 0x557bedf7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf7cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf7f750 0x557bedf7d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedede020 con 0x557bedf76250 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedbb1430 con 0x557bedf76b70 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedb6de00 con 0x557bedf7f750 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa677209740 10 client.0 _ll_register_callbacks cb 0x557bede85cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa6723d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf76250 0x557bedf7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa6723d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf76250 0x557bedf7c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53528/0 (socket says 172.21.15.42:53528) 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.856+0000 7fa6723d5640 1 -- 172.21.15.42:0/4238956361 learned_addr learned my addr 172.21.15.42:0/4238956361 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa671bd4640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf7cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa672bd6640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf7f750 0x557bedf7d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa6723d5640 1 -- 172.21.15.42:0/4238956361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf7f750 msgr2=0x557bedf7d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa6723d5640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf7f750 0x557bedf7d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa6723d5640 1 -- 172.21.15.42:0/4238956361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 msgr2=0x557bedf7cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa6723d5640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf7cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa6723d5640 1 -- 172.21.15.42:0/4238956361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c069910 con 0x557bedf76250 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa671bd4640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf7cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa672bd6640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf7f750 0x557bedf7d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa6723d5640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf76250 0x557bedf7c740 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa6680cad60 tx=0x7fa668099e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97bdeb920ca6ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa655ffb640 1 -- 172.21.15.42:0/4238956361 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66809e170 con 0x557bedf76250 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa655ffb640 1 -- 172.21.15.42:0/4238956361 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa66809e330 con 0x557bedf76250 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bedbdd7e0 con 0x557bedf76250 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa655ffb640 1 -- 172.21.15.42:0/4238956361 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6680a2980 con 0x557bedf76250 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa677209740 10 client.17559 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557bededc6a0 con 0x557bedf76250 2024-03-12T04:22:38.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa655ffb640 1 -- 172.21.15.42:0/4238956361 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557bedbdd7e0 con 0x557bedf76250 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa655ffb640 1 -- 172.21.15.42:0/4238956361 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa668016030 con 0x557bedf76250 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.857+0000 7fa677209740 10 client.17559 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa677209740 10 client.17559 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa677209740 20 client.17559 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa677209740 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedf8fb10 0x557bedf91f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa677209740 4 client.17559 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bedc1a720 con 0x557bedf8fb10 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa671bd4640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedf8fb10 0x557bedf91f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa671bd4640 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedf8fb10 0x557bedf91f20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=922e27c3dfb32c25 server_cookie=ae9ae1dcba5bdf87 in_seq=0 out_seq=0 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa655ffb640 10 client.17559 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa655ffb640 1 -- 172.21.15.42:0/4238956361 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bedc1a720 con 0x557bedf8fb10 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa655ffb640 10 client.17559 handle_command_reply: tid=0 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.858+0000 7fa677209740 1 client.17559 shutdown 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf76250 msgr2=0x557bedf7c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf76250 0x557bedf7c740 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa6680cad60 tx=0x7fa668099e40 comp rx=0 tx=0).stop 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 shutdown_connections 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf7f750 0x557bedf7d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf7cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedf8fb10 0x557bedf91f20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 --2- 172.21.15.42:0/4238956361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf76250 0x557bedf7c740 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa655ffb640 0 client.17559 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 >> 172.21.15.42:0/4238956361 conn(0x557bedf70980 msgr2=0x557bedf726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 shutdown_connections 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 1 -- 172.21.15.42:0/4238956361 wait complete. 2024-03-12T04:22:38.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.859+0000 7fa677209740 20 client.17559 trim_cache size 0 max 16384 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 Processor -- start 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 -- start start 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf7cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf6fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bede63130 con 0x557bedf7f750 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedf71980 con 0x557bedf76250 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa677209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedf75eb0 con 0x557bedf76b70 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf865a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53532/0 (socket says 172.21.15.42:53532) 2024-03-12T04:22:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 -- 172.21.15.42:0/3558970136 learned_addr learned my addr 172.21.15.42:0/3558970136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa671bd4640 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf6fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 -- 172.21.15.42:0/3558970136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 msgr2=0x557bedf6fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf6fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 -- 172.21.15.42:0/3558970136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 msgr2=0x557bedf7cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa6723d5640 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf7cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf7cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa672bd6640 1 -- 172.21.15.42:0/3558970136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bede78c20 con 0x557bedf7f750 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.871+0000 7fa671bd4640 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa672bd6640 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf865a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa664010af0 tx=0x7fa664004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268342544ce2d315 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa657fff640 1 -- 172.21.15.42:0/3558970136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664004b20 con 0x557bedf7f750 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa657fff640 1 -- 172.21.15.42:0/3558970136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa664024070 con 0x557bedf7f750 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa657fff640 1 -- 172.21.15.42:0/3558970136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640056c0 con 0x557bedf7f750 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa677209740 1 -- 172.21.15.42:0/3558970136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 msgr2=0x557bedf865a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa677209740 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf865a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa664010af0 tx=0x7fa664004ab0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa677209740 1 -- 172.21.15.42:0/3558970136 shutdown_connections 2024-03-12T04:22:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.872+0000 7fa677209740 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedf865a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76b70 0x557bedf6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 --2- 172.21.15.42:0/3558970136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76250 0x557bedf7cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 -- 172.21.15.42:0/3558970136 >> 172.21.15.42:0/3558970136 conn(0x557bedf70980 msgr2=0x557bedf6f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 -- 172.21.15.42:0/3558970136 shutdown_connections 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 -- 172.21.15.42:0/3558970136 wait complete. 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 -- start start 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 Processor -- start 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76250 0x557bedf85840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf85d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedfe5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedfe6520 con 0x557bedf7f750 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedfe6660 con 0x557bedf76b70 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bedfe67d0 con 0x557bedf76250 2024-03-12T04:22:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa677209740 10 client.0 _ll_register_callbacks cb 0x7fa65c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa672bd6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedfe5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa672bd6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedfe5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53544/0 (socket says 172.21.15.42:53544) 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa672bd6640 1 -- 172.21.15.42:0/3845779152 learned_addr learned my addr 172.21.15.42:0/3845779152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa6723d5640 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76250 0x557bedf85840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.873+0000 7fa671bd4640 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf85d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa672bd6640 1 -- 172.21.15.42:0/3845779152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76250 msgr2=0x557bedf85840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa672bd6640 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76250 0x557bedf85840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa672bd6640 1 -- 172.21.15.42:0/3845779152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 msgr2=0x557bedf85d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa672bd6640 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf85d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa672bd6640 1 -- 172.21.15.42:0/3845779152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6640096e0 con 0x557bedf7f750 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa672bd6640 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedfe5fe0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa664005c10 tx=0x7fa664004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bee7bad1c50781 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa655ffb640 1 -- 172.21.15.42:0/3845779152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66401ee40 con 0x557bedf7f750 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa655ffb640 1 -- 172.21.15.42:0/3845779152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa664026070 con 0x557bedf7f750 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa65c02de90 con 0x557bedf7f750 2024-03-12T04:22:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa655ffb640 1 -- 172.21.15.42:0/3845779152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa664005300 con 0x557bedf7f750 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 10 client.17565 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557bede9e8c0 con 0x557bedf7f750 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa655ffb640 1 -- 172.21.15.42:0/3845779152 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa65c02de90 con 0x557bedf7f750 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa655ffb640 1 -- 172.21.15.42:0/3845779152 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa664020850 con 0x557bedf7f750 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 10 client.17565 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 10 client.17565 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 20 client.17565 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedfe7f20 0x557bedfea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 4 client.17565 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.874+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557bedc69890 con 0x557bedfe7f20 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa6723d5640 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedfe7f20 0x557bedfea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa6723d5640 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedfe7f20 0x557bedfea340 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9105b355566cf784 server_cookie=fc7683e51fc465f1 in_seq=0 out_seq=0 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa655ffb640 10 client.17565 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa655ffb640 1 -- 172.21.15.42:0/3845779152 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557bedc69890 con 0x557bedfe7f20 2024-03-12T04:22:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa655ffb640 10 client.17565 handle_command_reply: tid=0 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 client.17565 shutdown 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 msgr2=0x557bedfe5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedfe5fe0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa664005c10 tx=0x7fa664004e90 comp rx=0 tx=0).stop 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 shutdown_connections 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x557bedfe7f20 0x557bedfea340 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bedf7f750 0x557bedfe5fe0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557bedf76b70 0x557bedf85d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa655ffb640 0 client.17565 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 --2- 172.21.15.42:0/3845779152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557bedf76250 0x557bedf85840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 >> 172.21.15.42:0/3845779152 conn(0x557bedf70980 msgr2=0x557bedf6fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 shutdown_connections 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 1 -- 172.21.15.42:0/3845779152 wait complete. 2024-03-12T04:22:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.875+0000 7fa677209740 20 client.17565 trim_cache size 0 max 16384 2024-03-12T04:22:38.886 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:38.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa6580a1c20 msgr2=0x7fa6580a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa6580a1c20 0x7fa6580a40e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa664005c40 tx=0x7fa664009d50 comp rx=0 tx=0).stop 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100310 msgr2=0x7fa66c197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100310 0x7fa66c197760 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c970 tx=0x7fa66800ce30 comp rx=0 tx=0).stop 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 shutdown_connections 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fa6580a1c20 0x7fa6580a40e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa66c19bfd0 0x7fa66c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1053e0 0x7fa66c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 --2- 172.21.15.42:0/4090610225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c100310 0x7fa66c197760 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 >> 172.21.15.42:0/4090610225 conn(0x7fa66c0771e0 msgr2=0x7fa66c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 shutdown_connections 2024-03-12T04:22:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.876+0000 7fa674639640 1 -- 172.21.15.42:0/4090610225 wait complete. 2024-03-12T04:22:38.889 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:38.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:22:38.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 -- 172.21.15.42:0/3452972639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c068fa0 msgr2=0x7fdf6c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 --2- 172.21.15.42:0/3452972639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c068fa0 0x7fdf6c105de0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009930 tx=0x7fdf5402bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:38.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 -- 172.21.15.42:0/3452972639 shutdown_connections 2024-03-12T04:22:38.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 --2- 172.21.15.42:0/3452972639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6c1064c0 0x7fdf6c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 --2- 172.21.15.42:0/3452972639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c068fa0 0x7fdf6c105de0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 --2- 172.21.15.42:0/3452972639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf6c0685f0 0x7fdf6c0689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 -- 172.21.15.42:0/3452972639 >> 172.21.15.42:0/3452972639 conn(0x7fdf6c077180 msgr2=0x7fdf6c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 -- 172.21.15.42:0/3452972639 shutdown_connections 2024-03-12T04:22:38.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.991+0000 7fdf710df640 1 -- 172.21.15.42:0/3452972639 wait complete. 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 Processor -- start 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 -- start start 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf6c0685f0 0x7fdf6c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6c1064c0 0x7fdf6c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c19c010 0x7fdf6c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c19ca00 con 0x7fdf6c19c010 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c19cb70 con 0x7fdf6c1064c0 2024-03-12T04:22:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf710df640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c19cce0 con 0x7fdf6c0685f0 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c19c010 0x7fdf6c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c19c010 0x7fdf6c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53556/0 (socket says 172.21.15.42:53556) 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf6b59e640 1 -- 172.21.15.42:0/482151790 learned_addr learned my addr 172.21.15.42:0/482151790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.992+0000 7fdf6a59c640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6c1064c0 0x7fdf6c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf6c0685f0 0x7fdf6c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf6b59e640 1 -- 172.21.15.42:0/482151790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf6c0685f0 msgr2=0x7fdf6c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf6c0685f0 0x7fdf6c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf6b59e640 1 -- 172.21.15.42:0/482151790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6c1064c0 msgr2=0x7fdf6c197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6c1064c0 0x7fdf6c197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf6b59e640 1 -- 172.21.15.42:0/482151790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54009560 con 0x7fdf6c19c010 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c19c010 0x7fdf6c19c4c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d750 tx=0x7fdf6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf63fff640 1 -- 172.21.15.42:0/482151790 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64009bd0 con 0x7fdf6c19c010 2024-03-12T04:22:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf63fff640 1 -- 172.21.15.42:0/482151790 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf6400de60 con 0x7fdf6c19c010 2024-03-12T04:22:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c1a42c0 con 0x7fdf6c19c010 2024-03-12T04:22:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf63fff640 1 -- 172.21.15.42:0/482151790 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64004e00 con 0x7fdf6c19c010 2024-03-12T04:22:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.993+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c1a4740 con 0x7fdf6c19c010 2024-03-12T04:22:39.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.995+0000 7fdf63fff640 1 -- 172.21.15.42:0/482151790 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fdf64004f60 con 0x7fdf6c19c010 2024-03-12T04:22:39.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 Processor -- start 2024-03-12T04:22:39.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 -- start start 2024-03-12T04:22:39.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf63fff640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdf580a1c20 0x7fdf580a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf6a59c640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdf580a1c20 0x7fdf580a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf63fff640 1 -- 172.21.15.42:0/482151790 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdf640cd1a0 con 0x7fdf6c19c010 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472eb250 0x5601472eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472ebb70 0x5601472f4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601472a6960 con 0x5601472f4750 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014725a310 con 0x5601472ebb70 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560147257840 con 0x5601472eb250 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53564/0 (socket says 172.21.15.42:53564) 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf6b59e640 1 -- 172.21.15.42:0/741197723 learned_addr learned my addr 172.21.15.42:0/741197723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472eb250 0x5601472eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.996+0000 7fdf6a59c640 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472ebb70 0x5601472f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf6a59c640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdf580a1c20 0x7fdf580a40e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009230 tx=0x7fdf54005fd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf6b59e640 1 -- 172.21.15.42:0/741197723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472eb250 msgr2=0x5601472eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472eb250 0x5601472eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf6b59e640 1 -- 172.21.15.42:0/741197723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472ebb70 msgr2=0x5601472f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472ebb70 0x5601472f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf6b59e640 1 -- 172.21.15.42:0/741197723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560147215060 con 0x5601472f4750 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f6b40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdf64096760 tx=0x7fdf64099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e945537fe80a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf617fa640 1 -- 172.21.15.42:0/741197723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640a2530 con 0x5601472f4750 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf617fa640 1 -- 172.21.15.42:0/741197723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf64009d30 con 0x5601472f4750 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.997+0000 7fdf617fa640 1 -- 172.21.15.42:0/741197723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64096400 con 0x5601472f4750 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- 172.21.15.42:0/741197723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 msgr2=0x5601472f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f6b40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdf64096760 tx=0x7fdf64099d90 comp rx=0 tx=0).stop 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- 172.21.15.42:0/741197723 shutdown_connections 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f6b40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472ebb70 0x5601472f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/741197723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472eb250 0x5601472eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- 172.21.15.42:0/741197723 >> 172.21.15.42:0/741197723 conn(0x5601472e5980 msgr2=0x5601472e5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- 172.21.15.42:0/741197723 shutdown_connections 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- 172.21.15.42:0/741197723 wait complete. 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- start start 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 Processor -- start 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560147253020 con 0x5601472f4750 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560146f26430 con 0x5601472eb250 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560146ee2e00 con 0x5601472ebb70 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf73d8f740 10 client.0 _ll_register_callbacks cb 0x5601471facb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.998+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53566/0 (socket says 172.21.15.42:53566) 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 -- 172.21.15.42:0/714953478 learned_addr learned my addr 172.21.15.42:0/714953478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6a59c640 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 -- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 msgr2=0x5601472f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 -- 172.21.15.42:0/714953478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 msgr2=0x5601472f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 -- 172.21.15.42:0/714953478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c1a4740 con 0x5601472f4750 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f21b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009610 tx=0x7fdf640999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ae778ff79d9577 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf527fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf527fc640 1 -- 172.21.15.42:0/714953478 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640b82c0 con 0x5601472f4750 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf527fc640 1 -- 172.21.15.42:0/714953478 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdf640a2530 con 0x5601472f4750 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560146f527e0 con 0x5601472f4750 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf527fc640 1 -- 172.21.15.42:0/714953478 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf64005210 con 0x5601472f4750 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf73d8f740 10 client.17577 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5601472516a0 con 0x5601472f4750 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf527fc640 1 -- 172.21.15.42:0/714953478 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x560146f527e0 con 0x5601472f4750 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:38.999+0000 7fdf527fc640 1 -- 172.21.15.42:0/714953478 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf64095ae0 con 0x5601472f4750 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf73d8f740 10 client.17577 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf73d8f740 10 client.17577 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf73d8f740 20 client.17577 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560147304610 0x560147306a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf73d8f740 4 client.17577 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560146f8f720 con 0x560147304610 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560147304610 0x560147306a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560147304610 0x560147306a00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8201e1da8b5df53 server_cookie=4a98f089d34d375c in_seq=0 out_seq=0 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.000+0000 7fdf527fc640 10 client.17577 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf527fc640 1 -- 172.21.15.42:0/714953478 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560146f8f720 con 0x560147304610 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf527fc640 10 client.17577 handle_command_reply: tid=0 2024-03-12T04:22:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 client.17577 shutdown 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 msgr2=0x5601472f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f21b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009610 tx=0x7fdf640999d0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 shutdown_connections 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x560147304610 0x560147306a00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x5601472f21b0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf527fc640 0 client.17577 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/714953478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 >> 172.21.15.42:0/714953478 conn(0x5601472e5980 msgr2=0x5601472e76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 shutdown_connections 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 1 -- 172.21.15.42:0/714953478 wait complete. 2024-03-12T04:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.001+0000 7fdf73d8f740 20 client.17577 trim_cache size 0 max 16384 2024-03-12T04:22:39.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 Processor -- start 2024-03-12T04:22:39.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 -- start start 2024-03-12T04:22:39.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472eb250 0x5601472f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472f4750 0x5601472ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601471d8130 con 0x5601472eb250 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014730bbe0 con 0x5601472f4750 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601472e1080 con 0x5601472ebb70 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf6ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472eb250 0x5601472f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf6ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472eb250 0x5601472f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53580/0 (socket says 172.21.15.42:53580) 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf6ad9d640 1 -- 172.21.15.42:0/1165216862 learned_addr learned my addr 172.21.15.42:0/1165216862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.013+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472f4750 0x5601472ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf6ad9d640 1 -- 172.21.15.42:0/1165216862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 msgr2=0x5601472f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf6ad9d640 1 -- 172.21.15.42:0/1165216862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472f4750 msgr2=0x5601472ee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472f4750 0x5601472ee9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf6ad9d640 1 -- 172.21.15.42:0/1165216862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601471edc20 con 0x5601472eb250 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472eb250 0x5601472f3e60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x5601473072b0 tx=0x7fdf5c0139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffeb16e846b1715a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf60ff9640 1 -- 172.21.15.42:0/1165216862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c017040 con 0x5601472eb250 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf60ff9640 1 -- 172.21.15.42:0/1165216862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf5c022070 con 0x5601472eb250 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf60ff9640 1 -- 172.21.15.42:0/1165216862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c010c80 con 0x5601472eb250 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1165216862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472eb250 msgr2=0x5601472f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.014+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472eb250 0x5601472f3e60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x5601473072b0 tx=0x7fdf5c0139f0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1165216862 shutdown_connections 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472f4750 0x5601472ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1165216862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472eb250 0x5601472f3e60 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1165216862 >> 172.21.15.42:0/1165216862 conn(0x5601472e5980 msgr2=0x5601472eca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1165216862 shutdown_connections 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1165216862 wait complete. 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- start start 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 Processor -- start 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472fac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x56014735aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014735b000 con 0x5601472f4750 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014735b140 con 0x5601472eb250 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56014735b2b0 con 0x5601472ebb70 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf73d8f740 10 client.0 _ll_register_callbacks cb 0x7fdf64099b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x56014735aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf6b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x56014735aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53584/0 (socket says 172.21.15.42:53584) 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf6b59e640 1 -- 172.21.15.42:0/1348282010 learned_addr learned my addr 172.21.15.42:0/1348282010 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf6a59c640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472fac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.015+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf6b59e640 1 -- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 msgr2=0x5601472fac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472fac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf6b59e640 1 -- 172.21.15.42:0/1348282010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 msgr2=0x5601472fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472fa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf6b59e640 1 -- 172.21.15.42:0/1348282010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c009e10 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf6a59c640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf6b59e640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x56014735aac0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fdf6409a640 tx=0x7fdf640b2b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d0addc6f4a6b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf527fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf527fc640 1 -- 172.21.15.42:0/1348282010 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64096030 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf527fc640 1 -- 172.21.15.42:0/1348282010 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdf640b82c0 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf64009d30 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf527fc640 1 -- 172.21.15.42:0/1348282010 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf640a2530 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 10 client.17583 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5601472138c0 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf527fc640 1 -- 172.21.15.42:0/1348282010 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf64009d30 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf527fc640 1 -- 172.21.15.42:0/1348282010 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf64098c50 con 0x5601472f4750 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 10 client.17583 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 10 client.17583 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 20 client.17583 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56014735c5a0 0x56014735e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 4 client.17583 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.016+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560146fde890 con 0x56014735c5a0 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56014735c5a0 0x56014735e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf527fc640 10 client.17583 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf6ad9d640 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56014735c5a0 0x56014735e990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=872fe49e15d745e2 server_cookie=be478c7e9d966a15 in_seq=0 out_seq=0 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf527fc640 1 -- 172.21.15.42:0/1348282010 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560146fde890 con 0x56014735c5a0 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf527fc640 10 client.17583 handle_command_reply: tid=0 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 client.17583 shutdown 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 msgr2=0x56014735aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x56014735aac0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fdf6409a640 tx=0x7fdf640b2b40 comp rx=0 tx=0).stop 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 shutdown_connections 2024-03-12T04:22:39.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56014735c5a0 0x56014735e990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601472f4750 0x56014735aac0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601472ebb70 0x5601472fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf527fc640 0 client.17583 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 --2- 172.21.15.42:0/1348282010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601472eb250 0x5601472fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 >> 172.21.15.42:0/1348282010 conn(0x5601472e5980 msgr2=0x5601472e6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 shutdown_connections 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.017+0000 7fdf73d8f740 1 -- 172.21.15.42:0/1348282010 wait complete. 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf73d8f740 20 client.17583 trim_cache size 0 max 16384 2024-03-12T04:22:39.027 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:39.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdf580a1c20 msgr2=0x7fdf580a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdf580a1c20 0x7fdf580a40e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009230 tx=0x7fdf54005fd0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c19c010 msgr2=0x7fdf6c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c19c010 0x7fdf6c19c4c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d750 tx=0x7fdf6400dc10 comp rx=0 tx=0).stop 2024-03-12T04:22:39.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 shutdown_connections 2024-03-12T04:22:39.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fdf580a1c20 0x7fdf580a40e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c19c010 0x7fdf6c19c4c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6c1064c0 0x7fdf6c197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 --2- 172.21.15.42:0/482151790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf6c0685f0 0x7fdf6c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 >> 172.21.15.42:0/482151790 conn(0x7fdf6c077180 msgr2=0x7fdf6c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 shutdown_connections 2024-03-12T04:22:39.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.018+0000 7fdf710df640 1 -- 172.21.15.42:0/482151790 wait complete. 2024-03-12T04:22:39.031 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:39.031 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:22:39.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- 172.21.15.42:0/892713000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 msgr2=0x7f0e1c0ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 --2- 172.21.15.42:0/892713000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c0ffcd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009930 tx=0x7f0e1802bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- 172.21.15.42:0/892713000 shutdown_connections 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 --2- 172.21.15.42:0/892713000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c106500 0x7f0e1c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 --2- 172.21.15.42:0/892713000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e1c1002a0 0x7f0e1c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 --2- 172.21.15.42:0/892713000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c0ffcd0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- 172.21.15.42:0/892713000 >> 172.21.15.42:0/892713000 conn(0x7f0e1c076640 msgr2=0x7f0e1c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- 172.21.15.42:0/892713000 shutdown_connections 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- 172.21.15.42:0/892713000 wait complete. 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 Processor -- start 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- start start 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c106500 0x7f0e1c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e1c197660 0x7f0e1c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c198130 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1982a0 con 0x7f0e1c106500 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.134+0000 7f0e2413e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c198460 con 0x7f0e1c197660 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53594/0 (socket says 172.21.15.42:53594) 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 -- 172.21.15.42:0/3909324898 learned_addr learned my addr 172.21.15.42:0/3909324898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e216d9640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c106500 0x7f0e1c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e226db640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e1c197660 0x7f0e1c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 -- 172.21.15.42:0/3909324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e1c197660 msgr2=0x7f0e1c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e1c197660 0x7f0e1c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 -- 172.21.15.42:0/3909324898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c106500 msgr2=0x7f0e1c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c106500 0x7f0e1c19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 -- 172.21.15.42:0/3909324898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e18009560 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e216d9640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c106500 0x7f0e1c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e21eda640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c19d5a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009230 tx=0x7f0e180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e12ffd640 1 -- 172.21.15.42:0/3909324898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180044c0 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e12ffd640 1 -- 172.21.15.42:0/3909324898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e1802de90 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c1a43a0 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e12ffd640 1 -- 172.21.15.42:0/3909324898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18005400 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.135+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c1a4880 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.137+0000 7f0e12ffd640 1 -- 172.21.15.42:0/3909324898 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0e18005560 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 Processor -- start 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 -- start start 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d7477630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477b70 0x5556d7480210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7480750 0x5556d7482b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d7432960 con 0x5556d7477b70 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d73e6310 con 0x5556d7477250 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e26def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d73e3840 con 0x5556d7480750 2024-03-12T04:22:39.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e12ffd640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e080a1ba0 0x7f0e080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e21eda640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477b70 0x5556d7480210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e21eda640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477b70 0x5556d7480210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53604/0 (socket says 172.21.15.42:53604) 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e216d9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7480750 0x5556d7482b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e21eda640 1 -- 172.21.15.42:0/3821486870 learned_addr learned my addr 172.21.15.42:0/3821486870 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e226db640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e080a1ba0 0x7f0e080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e226db640 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d7477630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e12ffd640 1 -- 172.21.15.42:0/3909324898 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0e180ef860 con 0x7f0e1c0ff8f0 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e216d9640 1 -- 172.21.15.42:0/3821486870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 msgr2=0x5556d7477630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.138+0000 7f0e216d9640 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d7477630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e216d9640 1 -- 172.21.15.42:0/3821486870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477b70 msgr2=0x5556d7480210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e216d9640 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477b70 0x5556d7480210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e216d9640 1 -- 172.21.15.42:0/3821486870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556d73a1060 con 0x5556d7480750 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e226db640 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d7477630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e21eda640 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477b70 0x5556d7480210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:22:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e226db640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e080a1ba0 0x7f0e080a4060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005cc0 tx=0x7f0e1400a340 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e216d9640 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7480750 0x5556d7482b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c198eb0 tx=0x7f0e0c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7638d0193f2a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e07fff640 1 -- 172.21.15.42:0/3821486870 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00be80 con 0x5556d7480750 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e07fff640 1 -- 172.21.15.42:0/3821486870 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e0c0118f0 con 0x5556d7480750 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.139+0000 7f0e07fff640 1 -- 172.21.15.42:0/3821486870 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c010870 con 0x5556d7480750 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 -- 172.21.15.42:0/3821486870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7480750 msgr2=0x5556d7482b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7480750 0x5556d7482b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c198eb0 tx=0x7f0e0c00f730 comp rx=0 tx=0).stop 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 -- 172.21.15.42:0/3821486870 shutdown_connections 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7480750 0x5556d7482b40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477b70 0x5556d7480210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 --2- 172.21.15.42:0/3821486870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d7477630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 -- 172.21.15.42:0/3821486870 >> 172.21.15.42:0/3821486870 conn(0x5556d7471980 msgr2=0x5556d7471d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 -- 172.21.15.42:0/3821486870 shutdown_connections 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 -- 172.21.15.42:0/3821486870 wait complete. 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 -- start start 2024-03-12T04:22:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.140+0000 7f0e26def740 1 Processor -- start 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477250 0x5556d747d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d747dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7480750 0x5556d747e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d73df020 con 0x5556d7477250 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d70b2430 con 0x5556d7480750 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d706ee00 con 0x5556d7477b70 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 10 client.0 _ll_register_callbacks cb 0x5556d7386cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477250 0x5556d747d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477250 0x5556d747d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53612/0 (socket says 172.21.15.42:53612) 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 -- 172.21.15.42:0/2869809587 learned_addr learned my addr 172.21.15.42:0/2869809587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e216d9640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d747dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e226db640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7480750 0x5556d747e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 -- 172.21.15.42:0/2869809587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 msgr2=0x5556d747dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d747dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 -- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7480750 msgr2=0x5556d747e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7480750 0x5556d747e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 -- 172.21.15.42:0/2869809587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c00b8e0 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e226db640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7480750 0x5556d747e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e21eda640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477250 0x5556d747d7a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0e18006010 tx=0x7f0e18009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d99f582772477e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e057fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e057fa640 1 -- 172.21.15.42:0/2869809587 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180bccc0 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e057fa640 1 -- 172.21.15.42:0/2869809587 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0e180bc5c0 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.141+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556d70de7e0 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e057fa640 1 -- 172.21.15.42:0/2869809587 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e180c0710 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 10 client.17595 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556d73dd6a0 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e057fa640 1 -- 172.21.15.42:0/2869809587 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5556d70de7e0 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e057fa640 1 -- 172.21.15.42:0/2869809587 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e1803c070 con 0x5556d7477250 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 10 client.17595 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 10 client.17595 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 20 client.17595 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d7490920 0x5556d7492d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 4 client.17595 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556d711b720 con 0x5556d7490920 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.142+0000 7f0e216d9640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d7490920 0x5556d7492d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e216d9640 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d7490920 0x5556d7492d10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e5b7724746a7b9e server_cookie=5079e520a37d360a in_seq=0 out_seq=0 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e057fa640 10 client.17595 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e057fa640 1 -- 172.21.15.42:0/2869809587 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556d711b720 con 0x5556d7490920 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e057fa640 10 client.17595 handle_command_reply: tid=0 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e26def740 1 client.17595 shutdown 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477250 msgr2=0x5556d747d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e26def740 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477250 0x5556d747d7a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0e18006010 tx=0x7f0e18009260 comp rx=0 tx=0).stop 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 shutdown_connections 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.143+0000 7f0e26def740 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d7490920 0x5556d7492d10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e26def740 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7480750 0x5556d747e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e26def740 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d747dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e26def740 1 --2- 172.21.15.42:0/2869809587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7477250 0x5556d747d7a0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e057fa640 0 client.17595 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:39.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 >> 172.21.15.42:0/2869809587 conn(0x5556d7471980 msgr2=0x5556d74736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 shutdown_connections 2024-03-12T04:22:39.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e26def740 1 -- 172.21.15.42:0/2869809587 wait complete. 2024-03-12T04:22:39.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.144+0000 7f0e26def740 20 client.17595 trim_cache size 0 max 16384 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 Processor -- start 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 -- start start 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d747fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d747e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d7364130 con 0x5556d7480750 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d7472980 con 0x5556d7477250 2024-03-12T04:22:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.155+0000 7f0e26def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d7490720 con 0x5556d7477b70 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d747e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d747e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53620/0 (socket says 172.21.15.42:53620) 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 -- 172.21.15.42:0/717888718 learned_addr learned my addr 172.21.15.42:0/717888718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e21eda640 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d747fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e216d9640 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 -- 172.21.15.42:0/717888718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 msgr2=0x5556d74879e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 -- 172.21.15.42:0/717888718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 msgr2=0x5556d747fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d747fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 -- 172.21.15.42:0/717888718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556d7379c20 con 0x5556d7480750 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e226db640 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d747e040 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f0e140131a0 tx=0x7f0e14005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80d00525aef1954 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e077fe640 1 -- 172.21.15.42:0/717888718 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400ee70 con 0x5556d7480750 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e077fe640 1 -- 172.21.15.42:0/717888718 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e140118f0 con 0x5556d7480750 2024-03-12T04:22:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.156+0000 7f0e077fe640 1 -- 172.21.15.42:0/717888718 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14010950 con 0x5556d7480750 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 -- 172.21.15.42:0/717888718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 msgr2=0x5556d747e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d747e040 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f0e140131a0 tx=0x7f0e14005bd0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 -- 172.21.15.42:0/717888718 shutdown_connections 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d747e040 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 --2- 172.21.15.42:0/717888718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d747fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 -- 172.21.15.42:0/717888718 >> 172.21.15.42:0/717888718 conn(0x5556d7471980 msgr2=0x5556d7497670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 -- 172.21.15.42:0/717888718 shutdown_connections 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 -- 172.21.15.42:0/717888718 wait complete. 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 -- start start 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.157+0000 7f0e26def740 1 Processor -- start 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d74e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d74dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d74e0130 con 0x5556d7480750 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d74e02a0 con 0x5556d7477250 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d74e0460 con 0x5556d7477b70 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 10 client.0 _ll_register_callbacks cb 0x7f0e0c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d74dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d74dfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53626/0 (socket says 172.21.15.42:53626) 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 -- 172.21.15.42:0/1188328780 learned_addr learned my addr 172.21.15.42:0/1188328780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 -- 172.21.15.42:0/1188328780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 msgr2=0x5556d74e5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e216d9640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e21eda640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d74e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74e5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 -- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 msgr2=0x5556d74e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d74e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 -- 172.21.15.42:0/1188328780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e14018070 con 0x5556d7480750 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e216d9640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74e5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e226db640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d74dfb70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f0e14013170 tx=0x7f0e14011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee4ba38b96bd07a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e057fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e057fa640 1 -- 172.21.15.42:0/1188328780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14010430 con 0x5556d7480750 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e0c0118f0 con 0x5556d7480750 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e057fa640 1 -- 172.21.15.42:0/1188328780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0e1400ee70 con 0x5556d7480750 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e057fa640 1 -- 172.21.15.42:0/1188328780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e14004c50 con 0x5556d7480750 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.158+0000 7f0e26def740 10 client.17601 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556d739f8c0 con 0x5556d7480750 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e057fa640 1 -- 172.21.15.42:0/1188328780 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e0c0118f0 con 0x5556d7480750 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e057fa640 1 -- 172.21.15.42:0/1188328780 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e1402c030 con 0x5556d7480750 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e26def740 10 client.17601 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e26def740 10 client.17601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e26def740 20 client.17601 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e26def740 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d74ed600 0x5556d74ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e26def740 4 client.17601 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5556d716a890 con 0x5556d74ed600 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.159+0000 7f0e21eda640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d74ed600 0x5556d74ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e21eda640 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d74ed600 0x5556d74ef9f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20789e291bcd5775 server_cookie=95becf05ed1a4ee1 in_seq=0 out_seq=0 2024-03-12T04:22:39.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e057fa640 10 client.17601 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e057fa640 1 -- 172.21.15.42:0/1188328780 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5556d716a890 con 0x5556d74ed600 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e057fa640 10 client.17601 handle_command_reply: tid=0 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 client.17601 shutdown 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 msgr2=0x5556d74dfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d74dfb70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f0e14013170 tx=0x7f0e14011d10 comp rx=0 tx=0).stop 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 shutdown_connections 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x5556d74ed600 0x5556d74ef9f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556d7480750 0x5556d74dfb70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5556d7477b70 0x5556d74e5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e057fa640 0 client.17601 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 --2- 172.21.15.42:0/1188328780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556d7477250 0x5556d74e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 >> 172.21.15.42:0/1188328780 conn(0x5556d7471980 msgr2=0x5556d74702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 shutdown_connections 2024-03-12T04:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 1 -- 172.21.15.42:0/1188328780 wait complete. 2024-03-12T04:22:39.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.160+0000 7f0e26def740 20 client.17601 trim_cache size 0 max 16384 2024-03-12T04:22:39.169 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e080a1ba0 msgr2=0x7f0e080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e080a1ba0 0x7f0e080a4060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005cc0 tx=0x7f0e1400a340 comp rx=0 tx=0).stop 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 msgr2=0x7f0e1c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c19d5a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009230 tx=0x7f0e180040c0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 shutdown_connections 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f0e080a1ba0 0x7f0e080a4060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e1c197660 0x7f0e1c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c106500 0x7f0e1c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 --2- 172.21.15.42:0/3909324898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c0ff8f0 0x7f0e1c19d5a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 >> 172.21.15.42:0/3909324898 conn(0x7f0e1c076640 msgr2=0x7f0e1c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 shutdown_connections 2024-03-12T04:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.161+0000 7f0e2413e640 1 -- 172.21.15.42:0/3909324898 wait complete. 2024-03-12T04:22:39.174 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:39.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:22:39.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 -- 172.21.15.42:0/4219004974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400101770 msgr2=0x7f8400104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 --2- 172.21.15.42:0/4219004974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400101770 0x7f8400104d10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009930 tx=0x7f83f002bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:39.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 -- 172.21.15.42:0/4219004974 shutdown_connections 2024-03-12T04:22:39.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 --2- 172.21.15.42:0/4219004974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8400105440 0x7f840010cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 --2- 172.21.15.42:0/4219004974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400101770 0x7f8400104d10 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 --2- 172.21.15.42:0/4219004974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8400100dc0 0x7f84001011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 -- 172.21.15.42:0/4219004974 >> 172.21.15.42:0/4219004974 conn(0x7f84000753c0 msgr2=0x7f84000757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 -- 172.21.15.42:0/4219004974 shutdown_connections 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.276+0000 7f8408696640 1 -- 172.21.15.42:0/4219004974 wait complete. 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 Processor -- start 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 -- start start 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400100dc0 0x7f84001976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8400105440 0x7f8400197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f840019bf60 0x7f840019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840019c950 con 0x7f8400100dc0 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840019cac0 con 0x7f8400105440 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8408696640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840019cc30 con 0x7f840019bf60 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400100dc0 0x7f84001976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400100dc0 0x7f84001976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53658/0 (socket says 172.21.15.42:53658) 2024-03-12T04:22:39.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 -- 172.21.15.42:0/3601062089 learned_addr learned my addr 172.21.15.42:0/3601062089 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406c33640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f840019bf60 0x7f840019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 -- 172.21.15.42:0/3601062089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f840019bf60 msgr2=0x7f840019c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f840019bf60 0x7f840019c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 -- 172.21.15.42:0/3601062089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8400105440 msgr2=0x7f8400197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8400105440 0x7f8400197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.277+0000 7f8406432640 1 -- 172.21.15.42:0/3601062089 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f0009560 con 0x7f8400100dc0 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.278+0000 7f8406432640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400100dc0 0x7f84001976a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c920 tx=0x7f83fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.278+0000 7f83f77fe640 1 -- 172.21.15.42:0/3601062089 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc007c70 con 0x7f8400100dc0 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.278+0000 7f83f77fe640 1 -- 172.21.15.42:0/3601062089 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83fc007dd0 con 0x7f8400100dc0 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.278+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84001a4210 con 0x7f8400100dc0 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.278+0000 7f83f77fe640 1 -- 172.21.15.42:0/3601062089 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0187c0 con 0x7f8400100dc0 2024-03-12T04:22:39.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.278+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84001a46c0 con 0x7f8400100dc0 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f83f77fe640 1 -- 172.21.15.42:0/3601062089 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f83fc00f450 con 0x7f8400100dc0 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 Processor -- start 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 -- start start 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f52f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f538210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f53ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f4ea960 con 0x55999f52fb70 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f49e310 con 0x55999f52f250 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f840b262740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f49b840 con 0x55999f538750 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f83f77fe640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f83ec0a1ba0 0x7f83ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.280+0000 7f8406432640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f538210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f538210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53674/0 (socket says 172.21.15.42:53674) 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 -- 172.21.15.42:0/2877539177 learned_addr learned my addr 172.21.15.42:0/2877539177 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406c33640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f83ec0a1ba0 0x7f83ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406c33640 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f52f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8405c31640 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f53ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f83f77fe640 1 -- 172.21.15.42:0/3601062089 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f83fc0cd9e0 con 0x7f8400100dc0 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 -- 172.21.15.42:0/2877539177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 msgr2=0x55999f53ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f53ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 -- 172.21.15.42:0/2877539177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 msgr2=0x55999f52f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f52f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 -- 172.21.15.42:0/2877539177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999f459060 con 0x55999f52fb70 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406432640 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f538210 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f83fc097000 tx=0x7f83fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac54a4c2a0c9e5ae server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f8406c33640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f83ec0a1ba0 0x7f83ec0a4060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005c40 tx=0x7f83f800ac90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f83f4ff9640 1 -- 172.21.15.42:0/2877539177 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc09ad30 con 0x55999f52fb70 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f83f4ff9640 1 -- 172.21.15.42:0/2877539177 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83fc0b0040 con 0x55999f52fb70 2024-03-12T04:22:39.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.281+0000 7f83f4ff9640 1 -- 172.21.15.42:0/2877539177 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0a2dd0 con 0x55999f52fb70 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 -- 172.21.15.42:0/2877539177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 msgr2=0x55999f538210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f538210 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f83fc097000 tx=0x7f83fc00cf90 comp rx=0 tx=0).stop 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 -- 172.21.15.42:0/2877539177 shutdown_connections 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f53ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f538210 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 --2- 172.21.15.42:0/2877539177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f52f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 -- 172.21.15.42:0/2877539177 >> 172.21.15.42:0/2877539177 conn(0x55999f529980 msgr2=0x55999f529d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 -- 172.21.15.42:0/2877539177 shutdown_connections 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 -- 172.21.15.42:0/2877539177 wait complete. 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 -- start start 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.282+0000 7f840b262740 1 Processor -- start 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f840b262740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f535810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f840b262740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f535d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f840b262740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f536290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f840b262740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f497020 con 0x55999f52fb70 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f840b262740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f16a430 con 0x55999f52f250 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f840b262740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f126e00 con 0x55999f538750 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f840b262740 10 client.0 _ll_register_callbacks cb 0x55999f43ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f535d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f535d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53680/0 (socket says 172.21.15.42:53680) 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 -- 172.21.15.42:0/4125473221 learned_addr learned my addr 172.21.15.42:0/4125473221 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8406432640 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f535810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8406c33640 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f536290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 -- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 msgr2=0x55999f536290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f536290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 -- 172.21.15.42:0/4125473221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 msgr2=0x55999f535810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f535810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 -- 172.21.15.42:0/4125473221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84001a46c0 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f8405c31640 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f535d50 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009900 tx=0x7f83f002c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b63b31bef7977d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f83e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.283+0000 7f83e9ffb640 1 -- 172.21.15.42:0/4125473221 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0004450 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f83e9ffb640 1 -- 172.21.15.42:0/4125473221 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f83f002cb90 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55999f1967e0 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f83e9ffb640 1 -- 172.21.15.42:0/4125473221 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83f0004dc0 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 10 client.17613 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55999f4956a0 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f83e9ffb640 1 -- 172.21.15.42:0/4125473221 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55999f1967e0 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f83e9ffb640 1 -- 172.21.15.42:0/4125473221 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f83f003c040 con 0x55999f52fb70 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 10 client.17613 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 10 client.17613 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:22:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 20 client.17613 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f548b50 0x55999f54af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 4 client.17613 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55999f1d3720 con 0x55999f548b50 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.284+0000 7f8406432640 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f548b50 0x55999f54af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.285+0000 7f83e9ffb640 10 client.17613 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.285+0000 7f8406432640 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f548b50 0x55999f54af70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aed4829f48482121 server_cookie=b14427379dc77f90 in_seq=0 out_seq=0 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.285+0000 7f83e9ffb640 1 -- 172.21.15.42:0/4125473221 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55999f1d3720 con 0x55999f548b50 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.285+0000 7f83e9ffb640 10 client.17613 handle_command_reply: tid=0 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 client.17613 shutdown 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 msgr2=0x55999f535d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f535d50 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009900 tx=0x7f83f002c9a0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 shutdown_connections 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f548b50 0x55999f54af70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f538750 0x55999f536290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52fb70 0x55999f535d50 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f83e9ffb640 0 client.17613 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 --2- 172.21.15.42:0/4125473221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f535810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 >> 172.21.15.42:0/4125473221 conn(0x55999f529980 msgr2=0x55999f52b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 shutdown_connections 2024-03-12T04:22:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 1 -- 172.21.15.42:0/4125473221 wait complete. 2024-03-12T04:22:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.286+0000 7f840b262740 20 client.17613 trim_cache size 0 max 16384 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 Processor -- start 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 -- start start 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52f250 0x55999f5360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f528a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f538750 0x55999f53f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f41c130 con 0x55999f52f250 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f52e9f0 con 0x55999f538750 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f840b262740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f52e670 con 0x55999f52fb70 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52f250 0x55999f5360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52f250 0x55999f5360b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53692/0 (socket says 172.21.15.42:53692) 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 -- 172.21.15.42:0/153436136 learned_addr learned my addr 172.21.15.42:0/153436136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8405c31640 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f528a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 -- 172.21.15.42:0/153436136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 msgr2=0x55999f528a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f528a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 -- 172.21.15.42:0/153436136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f538750 msgr2=0x55999f53f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f538750 0x55999f53f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 -- 172.21.15.42:0/153436136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999f431c20 con 0x55999f52f250 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8405c31640 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f528a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f8406432640 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52f250 0x55999f5360b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00ffd0 tx=0x7f83fc09e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f627b5417955cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.298+0000 7f83ebfff640 1 -- 172.21.15.42:0/153436136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0cdca0 con 0x55999f52f250 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f83ebfff640 1 -- 172.21.15.42:0/153436136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83fc09e7f0 con 0x55999f52f250 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f83ebfff640 1 -- 172.21.15.42:0/153436136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0cb3d0 con 0x55999f52f250 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 -- 172.21.15.42:0/153436136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52f250 msgr2=0x55999f5360b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52f250 0x55999f5360b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00ffd0 tx=0x7f83fc09e780 comp rx=0 tx=0).stop 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 -- 172.21.15.42:0/153436136 shutdown_connections 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f538750 0x55999f53f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f528a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 --2- 172.21.15.42:0/153436136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f52f250 0x55999f5360b0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 -- 172.21.15.42:0/153436136 >> 172.21.15.42:0/153436136 conn(0x55999f529980 msgr2=0x55999f5282f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 -- 172.21.15.42:0/153436136 shutdown_connections 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 -- 172.21.15.42:0/153436136 wait complete. 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.299+0000 7f840b262740 1 -- start start 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 1 Processor -- start 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f53e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f53e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f538750 0x55999f53ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f59f210 con 0x55999f538750 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f59f380 con 0x55999f52f250 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999f59f540 con 0x55999f52fb70 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f840b262740 10 client.0 _ll_register_callbacks cb 0x7f83fc09a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f538750 0x55999f53ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f538750 0x55999f53ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53706/0 (socket says 172.21.15.42:53706) 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 -- 172.21.15.42:0/2482000939 learned_addr learned my addr 172.21.15.42:0/2482000939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8405c31640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f53e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406432640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f53e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 -- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 msgr2=0x55999f53e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f53e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 -- 172.21.15.42:0/2482000939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 msgr2=0x55999f53e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f53e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 -- 172.21.15.42:0/2482000939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55999f459060 con 0x55999f538750 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8405c31640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f53e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f8406c33640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f538750 0x55999f53ed50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f83f8010ca0 tx=0x7f83f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1337dfcb25df727 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.300+0000 7f83e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f83e9ffb640 1 -- 172.21.15.42:0/2482000939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8010ee0 con 0x55999f538750 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55999f222890 con 0x55999f538750 2024-03-12T04:22:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f83e9ffb640 1 -- 172.21.15.42:0/2482000939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f83f8026070 con 0x55999f538750 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f83e9ffb640 1 -- 172.21.15.42:0/2482000939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83f8003b70 con 0x55999f538750 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 10 client.17619 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55999f4578c0 con 0x55999f538750 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f83e9ffb640 1 -- 172.21.15.42:0/2482000939 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55999f222890 con 0x55999f538750 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f83e9ffb640 1 -- 172.21.15.42:0/2482000939 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f83f8020480 con 0x55999f538750 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 10 client.17619 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 10 client.17619 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 20 client.17619 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f5a0ca0 0x55999f5a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 4 client.17619 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55999f142700 con 0x55999f5a0ca0 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.301+0000 7f8406432640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f5a0ca0 0x55999f5a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f83e9ffb640 10 client.17619 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f8406432640 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f5a0ca0 0x55999f5a30c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87138c520c76dbcd server_cookie=ee9b37a5e65bcedf in_seq=0 out_seq=0 2024-03-12T04:22:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f83e9ffb640 1 -- 172.21.15.42:0/2482000939 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55999f142700 con 0x55999f5a0ca0 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f83e9ffb640 10 client.17619 handle_command_reply: tid=0 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 client.17619 shutdown 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f538750 msgr2=0x55999f53ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f538750 0x55999f53ed50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f83f8010ca0 tx=0x7f83f8004a40 comp rx=0 tx=0).stop 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 shutdown_connections 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55999f5a0ca0 0x55999f5a30c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55999f538750 0x55999f53ed50 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f83e9ffb640 0 client.17619 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55999f52fb70 0x55999f53e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 --2- 172.21.15.42:0/2482000939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55999f52f250 0x55999f53e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 >> 172.21.15.42:0/2482000939 conn(0x55999f529980 msgr2=0x55999f52b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 shutdown_connections 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 1 -- 172.21.15.42:0/2482000939 wait complete. 2024-03-12T04:22:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.302+0000 7f840b262740 20 client.17619 trim_cache size 0 max 16384 2024-03-12T04:22:39.313 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:39.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f83ec0a1ba0 msgr2=0x7f83ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f83ec0a1ba0 0x7f83ec0a4060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005c40 tx=0x7f83f800ac90 comp rx=0 tx=0).stop 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400100dc0 msgr2=0x7f84001976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400100dc0 0x7f84001976a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c920 tx=0x7f83fc00cde0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 shutdown_connections 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f83ec0a1ba0 0x7f83ec0a4060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f840019bf60 0x7f840019c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8400105440 0x7f8400197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 --2- 172.21.15.42:0/3601062089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8400100dc0 0x7f84001976a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 >> 172.21.15.42:0/3601062089 conn(0x7f84000753c0 msgr2=0x7f84000fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 shutdown_connections 2024-03-12T04:22:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.303+0000 7f8408696640 1 -- 172.21.15.42:0/3601062089 wait complete. 2024-03-12T04:22:39.316 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:22:39.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:22:39.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 -- 172.21.15.42:0/2345244823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8100c10 msgr2=0x7f8af8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 --2- 172.21.15.42:0/2345244823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8100c10 0x7f8af8104c50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009930 tx=0x7f8ae802bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 -- 172.21.15.42:0/2345244823 shutdown_connections 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 --2- 172.21.15.42:0/2345244823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8af8105330 0x7f8af810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 --2- 172.21.15.42:0/2345244823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8100c10 0x7f8af8104c50 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 --2- 172.21.15.42:0/2345244823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af8100260 0x7f8af8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 -- 172.21.15.42:0/2345244823 >> 172.21.15.42:0/2345244823 conn(0x7f8af8077180 msgr2=0x7f8af8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 -- 172.21.15.42:0/2345244823 shutdown_connections 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.419+0000 7f8aff82a640 1 -- 172.21.15.42:0/2345244823 wait complete. 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 Processor -- start 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 -- start start 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af8100260 0x7f8af8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8af8105330 0x7f8af8102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8102770 0x7f8af81a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af81a5ee0 con 0x7f8af8102770 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af81a6050 con 0x7f8af8100260 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8aff82a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af81a6210 con 0x7f8af8105330 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8102770 0x7f8af81a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8102770 0x7f8af81a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53730/0 (socket says 172.21.15.42:53730) 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 -- 172.21.15.42:0/1712597023 learned_addr learned my addr 172.21.15.42:0/1712597023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8af8105330 0x7f8af8102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af8100260 0x7f8af8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 -- 172.21.15.42:0/1712597023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8af8105330 msgr2=0x7f8af8102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8af8105330 0x7f8af8102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 -- 172.21.15.42:0/1712597023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af8100260 msgr2=0x7f8af8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af8100260 0x7f8af8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 -- 172.21.15.42:0/1712597023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8009560 con 0x7f8af8102770 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af8100260 0x7f8af8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8af8105330 0x7f8af8102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.420+0000 7f8afddc7640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8102770 0x7f8af81a5910 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8af000eaa0 tx=0x7f8af000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.421+0000 7f8aee7fc640 1 -- 172.21.15.42:0/1712597023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af000cdb0 con 0x7f8af8102770 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.421+0000 7f8aee7fc640 1 -- 172.21.15.42:0/1712597023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8af0005020 con 0x7f8af8102770 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.421+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af81a64f0 con 0x7f8af8102770 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.421+0000 7f8aee7fc640 1 -- 172.21.15.42:0/1712597023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00116b0 con 0x7f8af8102770 2024-03-12T04:22:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.421+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af81a6a00 con 0x7f8af8102770 2024-03-12T04:22:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8aee7fc640 1 -- 172.21.15.42:0/1712597023 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8af0005190 con 0x7f8af8102770 2024-03-12T04:22:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 Processor -- start 2024-03-12T04:22:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 -- start start 2024-03-12T04:22:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77378630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 0x557e77381210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77381750 0x557e77383b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e77333960 con 0x557e77381750 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8aee7fc640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8ae40a1bd0 0x7f8ae40a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e772e7310 con 0x557e77378b70 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.423+0000 7f8b023ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e772e4840 con 0x557e77378250 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77381750 0x557e77383b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afddc7640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8ae40a1bd0 0x7f8ae40a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77381750 0x557e77383b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53744/0 (socket says 172.21.15.42:53744) 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 -- 172.21.15.42:0/2809792060 learned_addr learned my addr 172.21.15.42:0/2809792060 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 0x557e77381210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77378630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8aee7fc640 1 -- 172.21.15.42:0/1712597023 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8af00cd9d0 con 0x7f8af8102770 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 -- 172.21.15.42:0/2809792060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 msgr2=0x557e77378630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77378630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 -- 172.21.15.42:0/2809792060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 msgr2=0x557e77381210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 0x557e77381210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 -- 172.21.15.42:0/2809792060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e772a2060 con 0x557e77381750 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afddc7640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8ae40a1bd0 0x7f8ae40a4090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8af00cb1a0 tx=0x7f8af0000f40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77381750 0x557e77383b40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8ae802c710 tx=0x7f8ae8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716dda0a80562321 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8ae37fe640 1 -- 172.21.15.42:0/2809792060 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8004030 con 0x557e77381750 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8ae37fe640 1 -- 172.21.15.42:0/2809792060 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ae802de90 con 0x557e77381750 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.424+0000 7f8ae37fe640 1 -- 172.21.15.42:0/2809792060 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8004dc0 con 0x557e77381750 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 -- 172.21.15.42:0/2809792060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77381750 msgr2=0x557e77383b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77381750 0x557e77383b40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8ae802c710 tx=0x7f8ae8000bc0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 -- 172.21.15.42:0/2809792060 shutdown_connections 2024-03-12T04:22:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77381750 0x557e77383b40 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 0x557e77381210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 --2- 172.21.15.42:0/2809792060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77378630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 -- 172.21.15.42:0/2809792060 >> 172.21.15.42:0/2809792060 conn(0x557e77372980 msgr2=0x557e77372d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 -- 172.21.15.42:0/2809792060 shutdown_connections 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 -- 172.21.15.42:0/2809792060 wait complete. 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 -- start start 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.425+0000 7f8b023ff740 1 Processor -- start 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378250 0x557e7737e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 0x557e7737ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77381750 0x557e7737f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8b023ff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e772e0020 con 0x557e77378250 2024-03-12T04:22:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8b023ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e76fb3430 con 0x557e77378b70 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8b023ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e76f6fe00 con 0x557e77381750 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8b023ff740 10 client.0 _ll_register_callbacks cb 0x557e77287cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378250 0x557e7737e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378250 0x557e7737e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53746/0 (socket says 172.21.15.42:53746) 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 -- 172.21.15.42:0/3190489817 learned_addr learned my addr 172.21.15.42:0/3190489817 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77381750 0x557e7737f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 -- 172.21.15.42:0/3190489817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77381750 msgr2=0x557e7737f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77381750 0x557e7737f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 -- 172.21.15.42:0/3190489817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 msgr2=0x557e7737ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 0x557e7737ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 -- 172.21.15.42:0/3190489817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af810d710 con 0x557e77378250 2024-03-12T04:22:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378250 0x557e7737e810 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006010 tx=0x7f8ae802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa586df47de6ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.426+0000 7f8ae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/3190489817 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80040c0 con 0x557e77378250 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e76fdf7e0 con 0x557e77378250 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/3190489817 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8ae8000e40 con 0x557e77378250 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/3190489817 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ae8004220 con 0x557e77378250 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 10 client.17631 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557e772de6a0 con 0x557e77378250 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/3190489817 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ae8060020 con 0x557e77378250 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/3190489817 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ae804e040 con 0x557e77378250 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 10 client.17631 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 20 client.17631 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773917f0 0x557e77393c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 4 client.17631 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e7701c720 con 0x557e773917f0 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.427+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773917f0 0x557e77393c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.428+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773917f0 0x557e77393c10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3647ea9efdd9e9f0 server_cookie=d2ea0a28badd53b0 in_seq=0 out_seq=0 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.428+0000 7f8ae0ff9640 10 client.17631 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/3190489817 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e7701c720 con 0x557e773917f0 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8ae0ff9640 10 client.17631 handle_command_reply: tid=0 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 client.17631 shutdown 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378250 msgr2=0x557e7737e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378250 0x557e7737e810 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006010 tx=0x7f8ae802bc30 comp rx=0 tx=0).stop 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 shutdown_connections 2024-03-12T04:22:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773917f0 0x557e77393c10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77381750 0x557e7737f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77378b70 0x557e7737ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8ae0ff9640 0 client.17631 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 --2- 172.21.15.42:0/3190489817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378250 0x557e7737e810 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 >> 172.21.15.42:0/3190489817 conn(0x557e77372980 msgr2=0x557e773746d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 shutdown_connections 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 1 -- 172.21.15.42:0/3190489817 wait complete. 2024-03-12T04:22:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.429+0000 7f8b023ff740 20 client.17631 trim_cache size 0 max 16384 2024-03-12T04:22:39.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 Processor -- start 2024-03-12T04:22:39.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 -- start start 2024-03-12T04:22:39.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77380f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e77379a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77379fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e77265130 con 0x557e77378b70 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e77377ef0 con 0x557e77381750 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8b023ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e77398e60 con 0x557e77378250 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e77379a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e77379a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53752/0 (socket says 172.21.15.42:53752) 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 -- 172.21.15.42:0/1724942136 learned_addr learned my addr 172.21.15.42:0/1724942136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77379fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77380f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 -- 172.21.15.42:0/1724942136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 msgr2=0x557e77380f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77380f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 -- 172.21.15.42:0/1724942136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 msgr2=0x557e77379fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77379fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 -- 172.21.15.42:0/1724942136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e7727ac20 con 0x557e77378b70 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77379fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e77379a90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x557e7737e070 tx=0x7f8af400b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2893dd48e4faff06 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.441+0000 7f8ae2ffd640 1 -- 172.21.15.42:0/1724942136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400be60 con 0x557e77378b70 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8ae2ffd640 1 -- 172.21.15.42:0/1724942136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8af40187d0 con 0x557e77378b70 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8ae2ffd640 1 -- 172.21.15.42:0/1724942136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4020960 con 0x557e77378b70 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 -- 172.21.15.42:0/1724942136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 msgr2=0x557e77379a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e77379a90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x557e7737e070 tx=0x7f8af400b770 comp rx=0 tx=0).stop 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 -- 172.21.15.42:0/1724942136 shutdown_connections 2024-03-12T04:22:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77379fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e77379a90 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 --2- 172.21.15.42:0/1724942136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e77380f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 -- 172.21.15.42:0/1724942136 >> 172.21.15.42:0/1724942136 conn(0x557e77372980 msgr2=0x557e7736da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 -- 172.21.15.42:0/1724942136 shutdown_connections 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 -- 172.21.15.42:0/1724942136 wait complete. 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 -- start start 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.442+0000 7f8b023ff740 1 Processor -- start 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e773e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e773e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8b023ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77387a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8b023ff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e77387fe0 con 0x557e77378b70 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8b023ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e77388150 con 0x557e77381750 2024-03-12T04:22:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8b023ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e77388310 con 0x557e77378250 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8b023ff740 10 client.0 _ll_register_callbacks cb 0x7f8ae8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e773e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e773e2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53764/0 (socket says 172.21.15.42:53764) 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 -- 172.21.15.42:0/227609630 learned_addr learned my addr 172.21.15.42:0/227609630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e773e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77387a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 -- 172.21.15.42:0/227609630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 msgr2=0x557e773e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e773e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 -- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 msgr2=0x557e77387a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77387a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 -- 172.21.15.42:0/227609630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af400b900 con 0x557e77378b70 2024-03-12T04:22:39.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e773e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8afd5c6640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e773e2890 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8af400ba90 tx=0x7f8af4018e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee946a7a69673dc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.443+0000 7f8ae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/227609630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4005030 con 0x557e77378b70 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e7737e640 con 0x557e77378b70 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/227609630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8af4015050 con 0x557e77378b70 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/227609630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8af4005190 con 0x557e77378b70 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 10 client.17637 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557e772a08c0 con 0x557e77378b70 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/227609630 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557e7737e640 con 0x557e77378b70 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/227609630 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8af401e020 con 0x557e77378b70 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 10 client.17637 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:39.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 20 client.17637 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773ea1c0 0x557e773ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 4 client.17637 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8ae802de90 con 0x557e773ea1c0 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.444+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773ea1c0 0x557e773ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8afcdc5640 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773ea1c0 0x557e773ec5e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802cc351f4e57913 server_cookie=6d79b808ee5d8624 in_seq=0 out_seq=0 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8ae0ff9640 10 client.17637 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8ae0ff9640 1 -- 172.21.15.42:0/227609630 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8ae802de90 con 0x557e773ea1c0 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8ae0ff9640 10 client.17637 handle_command_reply: tid=0 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 client.17637 shutdown 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 msgr2=0x557e773e2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e773e2890 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8af400ba90 tx=0x7f8af4018e10 comp rx=0 tx=0).stop 2024-03-12T04:22:39.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 shutdown_connections 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557e773ea1c0 0x557e773ec5e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e77381750 0x557e77387a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8ae0ff9640 0 client.17637 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e77378b70 0x557e773e2890 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 --2- 172.21.15.42:0/227609630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e77378250 0x557e773e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 >> 172.21.15.42:0/227609630 conn(0x557e77372980 msgr2=0x557e77379a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 shutdown_connections 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 1 -- 172.21.15.42:0/227609630 wait complete. 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.445+0000 7f8b023ff740 20 client.17637 trim_cache size 0 max 16384 2024-03-12T04:22:39.456 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:22:39.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8ae40a1bd0 msgr2=0x7f8ae40a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8ae40a1bd0 0x7f8ae40a4090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8af00cb1a0 tx=0x7f8af0000f40 comp rx=0 tx=0).stop 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8102770 msgr2=0x7f8af81a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8102770 0x7f8af81a5910 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8af000eaa0 tx=0x7f8af000ef60 comp rx=0 tx=0).stop 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 shutdown_connections 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8ae40a1bd0 0x7f8ae40a4090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8af8102770 0x7f8af81a5910 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8af8105330 0x7f8af8102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 --2- 172.21.15.42:0/1712597023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af8100260 0x7f8af8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 >> 172.21.15.42:0/1712597023 conn(0x7f8af8077180 msgr2=0x7f8af80687c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 shutdown_connections 2024-03-12T04:22:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:39.446+0000 7f8aff82a640 1 -- 172.21.15.42:0/1712597023 wait complete. 2024-03-12T04:22:39.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:39 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3265845515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:22:39.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:39 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3265845515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:22:39.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:39 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3265845515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:22:40.460 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:22:40.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:22:40.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.577+0000 7f9138c64640 1 -- 172.21.15.42:0/3025982717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9134102d10 msgr2=0x7f913410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.577+0000 7f9138c64640 1 --2- 172.21.15.42:0/3025982717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9134102d10 0x7f913410b1f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9128009900 tx=0x7f912802bcd0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 -- 172.21.15.42:0/3025982717 shutdown_connections 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 --2- 172.21.15.42:0/3025982717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f913410b7c0 0x7f913410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 --2- 172.21.15.42:0/3025982717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9134102d10 0x7f913410b1f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 --2- 172.21.15.42:0/3025982717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91341023f0 0x7f91341027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 -- 172.21.15.42:0/3025982717 >> 172.21.15.42:0/3025982717 conn(0x7f91340771e0 msgr2=0x7f91340775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 -- 172.21.15.42:0/3025982717 shutdown_connections 2024-03-12T04:22:40.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 -- 172.21.15.42:0/3025982717 wait complete. 2024-03-12T04:22:40.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 Processor -- start 2024-03-12T04:22:40.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.578+0000 7f9138c64640 1 -- start start 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9138c64640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91341023f0 0x7f9134100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9138c64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9134102d10 0x7f9134100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9138c64640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913410b7c0 0x7f9134100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9138c64640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91341a59f0 con 0x7f913410b7c0 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9138c64640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91341a5b60 con 0x7f9134102d10 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9138c64640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91341a5d20 con 0x7f91341023f0 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913410b7c0 0x7f9134100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913410b7c0 0x7f9134100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53780/0 (socket says 172.21.15.42:53780) 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 -- 172.21.15.42:0/1220997111 learned_addr learned my addr 172.21.15.42:0/1220997111 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9131d9b640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9134102d10 0x7f9134100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f913259c640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91341023f0 0x7f9134100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 -- 172.21.15.42:0/1220997111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91341023f0 msgr2=0x7f9134100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91341023f0 0x7f9134100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 -- 172.21.15.42:0/1220997111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9134102d10 msgr2=0x7f9134100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9134102d10 0x7f9134100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 -- 172.21.15.42:0/1220997111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128009560 con 0x7f913410b7c0 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.579+0000 7f9132d9d640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913410b7c0 0x7f9134100ac0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f912c00cce0 tx=0x7f912c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.580+0000 7f91237fe640 1 -- 172.21.15.42:0/1220997111 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00f450 con 0x7f913410b7c0 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.580+0000 7f91237fe640 1 -- 172.21.15.42:0/1220997111 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f912c00fa60 con 0x7f913410b7c0 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.580+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91341a6000 con 0x7f913410b7c0 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.580+0000 7f91237fe640 1 -- 172.21.15.42:0/1220997111 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c018b10 con 0x7f913410b7c0 2024-03-12T04:22:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.580+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91341a6490 con 0x7f913410b7c0 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.582+0000 7f91237fe640 1 -- 172.21.15.42:0/1220997111 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f912c00fbd0 con 0x7f913410b7c0 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.582+0000 7f913b839740 1 Processor -- start 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.582+0000 7f913b839740 1 -- start start 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913b839740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0c380 0x56249ec0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913b839740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0cd30 0x56249ec152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913b839740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec17d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f91237fe640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f911c0a1c50 0x7f911c0a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913b839740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249eb7af90 con 0x56249ec0c380 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913b839740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249eb784c0 con 0x56249ec0cd30 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913b839740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249eb73ca0 con 0x56249ec158b0 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0c380 0x56249ec0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0c380 0x56249ec0c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53796/0 (socket says 172.21.15.42:53796) 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 -- 172.21.15.42:0/4575324 learned_addr learned my addr 172.21.15.42:0/4575324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9132d9d640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f911c0a1c50 0x7f911c0a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913259c640 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0cd30 0x56249ec152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f91237fe640 1 -- 172.21.15.42:0/1220997111 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f912c0cdae0 con 0x7f913410b7c0 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec17d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 -- 172.21.15.42:0/4575324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 msgr2=0x56249ec17d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec17d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 -- 172.21.15.42:0/4575324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0cd30 msgr2=0x56249ec152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0cd30 0x56249ec152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 -- 172.21.15.42:0/4575324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56249eb360b0 con 0x56249ec0c380 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f913259c640 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0cd30 0x56249ec152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9132d9d640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f911c0a1c50 0x7f911c0a4110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f912c0cb850 tx=0x7f912c06c930 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0c380 0x56249ec0c760 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9128005c40 tx=0x7f9128004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb07adfede7fc22 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.583+0000 7f9120ff9640 1 -- 172.21.15.42:0/4575324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912802c9e0 con 0x56249ec0c380 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f9120ff9640 1 -- 172.21.15.42:0/4575324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91280044e0 con 0x56249ec0c380 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f9120ff9640 1 -- 172.21.15.42:0/4575324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128004dc0 con 0x56249ec0c380 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 -- 172.21.15.42:0/4575324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0c380 msgr2=0x56249ec0c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0c380 0x56249ec0c760 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9128005c40 tx=0x7f9128004820 comp rx=0 tx=0).stop 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 -- 172.21.15.42:0/4575324 shutdown_connections 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec17d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0cd30 0x56249ec152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 --2- 172.21.15.42:0/4575324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0c380 0x56249ec0c760 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 -- 172.21.15.42:0/4575324 >> 172.21.15.42:0/4575324 conn(0x56249ec06840 msgr2=0x56249ec06c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 -- 172.21.15.42:0/4575324 shutdown_connections 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.584+0000 7f913b839740 1 -- 172.21.15.42:0/4575324 wait complete. 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 -- start start 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 Processor -- start 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec12890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec12dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249eb18f40 con 0x56249ec0cd30 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249e8474b0 con 0x56249ec0c380 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249eaf9670 con 0x56249ec158b0 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913b839740 10 client.0 _ll_register_callbacks cb 0x56249eb66f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec12dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec12dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53806/0 (socket says 172.21.15.42:53806) 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913259c640 1 -- 172.21.15.42:0/879526307 learned_addr learned my addr 172.21.15.42:0/879526307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f9131d9b640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec12890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f913259c640 1 -- 172.21.15.42:0/879526307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 msgr2=0x56249ec13310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.585+0000 7f9131d9b640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913259c640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec13310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913259c640 1 -- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 msgr2=0x56249ec12890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913259c640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec12890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913259c640 1 -- 172.21.15.42:0/879526307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913410e8b0 con 0x56249ec0cd30 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f9131d9b640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec13310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913259c640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec12dd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f912400cea0 tx=0x7f912400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113da9ac89a6a4b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f9119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f9119ffb640 1 -- 172.21.15.42:0/879526307 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124011970 con 0x56249ec0cd30 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f9119ffb640 1 -- 172.21.15.42:0/879526307 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9124011ad0 con 0x56249ec0cd30 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56249eb4df40 con 0x56249ec0cd30 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f9119ffb640 1 -- 172.21.15.42:0/879526307 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9124011c90 con 0x56249ec0cd30 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913b839740 10 client.17649 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56249eb72320 con 0x56249ec0cd30 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f9119ffb640 1 -- 172.21.15.42:0/879526307 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56249eb4df40 con 0x56249ec0cd30 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f9119ffb640 1 -- 172.21.15.42:0/879526307 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9124019470 con 0x56249ec0cd30 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913b839740 10 client.17649 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913b839740 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.586+0000 7f913b839740 20 client.17649 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.587+0000 7f913b839740 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec25fb0 0x56249ec283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.587+0000 7f913b839740 4 client.17649 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.587+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56249eb6a310 con 0x56249ec25fb0 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.587+0000 7f9131d9b640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec25fb0 0x56249ec283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.587+0000 7f9131d9b640 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec25fb0 0x56249ec283d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9dcf6186bf42e0 server_cookie=6c59da294ca0112 in_seq=0 out_seq=0 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.587+0000 7f9119ffb640 10 client.17649 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f9119ffb640 1 -- 172.21.15.42:0/879526307 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56249eb6a310 con 0x56249ec25fb0 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f9119ffb640 10 client.17649 handle_command_reply: tid=0 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 client.17649 shutdown 2024-03-12T04:22:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 msgr2=0x56249ec12dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec12dd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f912400cea0 tx=0x7f912400cf90 comp rx=0 tx=0).stop 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 shutdown_connections 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec25fb0 0x56249ec283d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec13310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec12dd0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 --2- 172.21.15.42:0/879526307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec12890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f9119ffb640 0 client.17649 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 >> 172.21.15.42:0/879526307 conn(0x56249ec06840 msgr2=0x56249ec087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 shutdown_connections 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 1 -- 172.21.15.42:0/879526307 wait complete. 2024-03-12T04:22:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.588+0000 7f913b839740 20 client.17649 trim_cache size 0 max 16384 2024-03-12T04:22:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.603+0000 7f913b839740 1 Processor -- start 2024-03-12T04:22:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.603+0000 7f913b839740 1 -- start start 2024-03-12T04:22:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913b839740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec0c380 0x56249ec13130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913b839740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913b839740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec158b0 0x56249ec1ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913b839740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249eb4df70 con 0x56249ec0cd30 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913b839740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249ec07670 con 0x56249ec158b0 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913b839740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249ec022c0 con 0x56249ec0c380 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53816/0 (socket says 172.21.15.42:53816) 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 -- 172.21.15.42:0/2905494081 learned_addr learned my addr 172.21.15.42:0/2905494081 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f9131d9b640 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec0c380 0x56249ec13130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 -- 172.21.15.42:0/2905494081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec0c380 msgr2=0x56249ec13130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec0c380 0x56249ec13130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f9131d9b640 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec158b0 0x56249ec1ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 -- 172.21.15.42:0/2905494081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec158b0 msgr2=0x56249ec1ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec158b0 0x56249ec1ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 -- 172.21.15.42:0/2905494081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56249eb0ec00 con 0x56249ec0cd30 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f913259c640 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1c8e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f91240058c0 tx=0x7f9124003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647e4a9eacb25ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f911bfff640 1 -- 172.21.15.42:0/2905494081 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124019a30 con 0x56249ec0cd30 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f911bfff640 1 -- 172.21.15.42:0/2905494081 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9124019b90 con 0x56249ec0cd30 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.604+0000 7f911bfff640 1 -- 172.21.15.42:0/2905494081 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240119d0 con 0x56249ec0cd30 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- 172.21.15.42:0/2905494081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 msgr2=0x56249ec1c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1c8e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f91240058c0 tx=0x7f9124003ed0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- 172.21.15.42:0/2905494081 shutdown_connections 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec158b0 0x56249ec1ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1c8e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 --2- 172.21.15.42:0/2905494081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec0c380 0x56249ec13130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- 172.21.15.42:0/2905494081 >> 172.21.15.42:0/2905494081 conn(0x56249ec06840 msgr2=0x56249ec05050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- 172.21.15.42:0/2905494081 shutdown_connections 2024-03-12T04:22:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- 172.21.15.42:0/2905494081 wait complete. 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- start start 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 Processor -- start 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec1ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec7c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249ec7ca20 con 0x56249ec0cd30 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249ec7cb60 con 0x56249ec0c380 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.605+0000 7f913b839740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249ec7cca0 con 0x56249ec158b0 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913b839740 10 client.0 _ll_register_callbacks cb 0x7f9128005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53822/0 (socket says 172.21.15.42:53822) 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 -- 172.21.15.42:0/4028007459 learned_addr learned my addr 172.21.15.42:0/4028007459 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec1ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 -- 172.21.15.42:0/4028007459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 msgr2=0x56249ec7c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec7c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 -- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 msgr2=0x56249ec1ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec1ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 -- 172.21.15.42:0/4028007459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56249eb72320 con 0x56249ec0cd30 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec1ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913259c640 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1bf80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9124019550 tx=0x7f9124011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422b60aa5302399c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f9119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f9119ffb640 1 -- 172.21.15.42:0/4028007459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124034670 con 0x56249ec0cd30 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f9119ffb640 1 -- 172.21.15.42:0/4028007459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9124000e40 con 0x56249ec0cd30 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56249eb99900 con 0x56249ec0cd30 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f9119ffb640 1 -- 172.21.15.42:0/4028007459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91240056e0 con 0x56249ec0cd30 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913b839740 10 client.17655 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.606+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56249eb34910 con 0x56249ec0cd30 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f9119ffb640 1 -- 172.21.15.42:0/4028007459 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56249eb99900 con 0x56249ec0cd30 2024-03-12T04:22:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f9119ffb640 1 -- 172.21.15.42:0/4028007459 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9124023460 con 0x56249ec0cd30 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f913b839740 10 client.17655 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f913b839740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f913b839740 20 client.17655 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f913b839740 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec7e140 0x56249ec80560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f913b839740 4 client.17655 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56249e873860 con 0x56249ec7e140 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.607+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec7e140 0x56249ec80560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.608+0000 7f9131d9b640 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec7e140 0x56249ec80560 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12a2f64cc75251d server_cookie=980f110caa16fc47 in_seq=0 out_seq=0 2024-03-12T04:22:40.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.608+0000 7f9119ffb640 10 client.17655 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f9119ffb640 1 -- 172.21.15.42:0/4028007459 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56249e873860 con 0x56249ec7e140 2024-03-12T04:22:40.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f9119ffb640 10 client.17655 handle_command_reply: tid=0 2024-03-12T04:22:40.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 client.17655 shutdown 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 msgr2=0x56249ec1bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1bf80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9124019550 tx=0x7f9124011bf0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 shutdown_connections 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56249ec7e140 0x56249ec80560 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f9119ffb640 0 client.17655 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56249ec158b0 0x56249ec7c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56249ec0cd30 0x56249ec1bf80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 --2- 172.21.15.42:0/4028007459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56249ec0c380 0x56249ec1ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 >> 172.21.15.42:0/4028007459 conn(0x56249ec06840 msgr2=0x56249ec05050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 shutdown_connections 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 1 -- 172.21.15.42:0/4028007459 wait complete. 2024-03-12T04:22:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.609+0000 7f913b839740 20 client.17655 trim_cache size 0 max 16384 2024-03-12T04:22:40.618 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:22:40.618 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:22:40.618 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "8b314c9f-fa23-4562-8a36-cd5e0a6aa183", 2024-03-12T04:22:40.618 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:22:40.618 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f911c0a1c50 msgr2=0x7f911c0a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f911c0a1c50 0x7f911c0a4110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f912c0cb850 tx=0x7f912c06c930 comp rx=0 tx=0).stop 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913410b7c0 msgr2=0x7f9134100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913410b7c0 0x7f9134100ac0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f912c00cce0 tx=0x7f912c00a620 comp rx=0 tx=0).stop 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 shutdown_connections 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f911c0a1c50 0x7f911c0a4110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913410b7c0 0x7f9134100ac0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9134102d10 0x7f9134100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 --2- 172.21.15.42:0/1220997111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91341023f0 0x7f9134100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 >> 172.21.15.42:0/1220997111 conn(0x7f91340771e0 msgr2=0x7f91340fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 shutdown_connections 2024-03-12T04:22:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.610+0000 7f9138c64640 1 -- 172.21.15.42:0/1220997111 wait complete. 2024-03-12T04:22:40.622 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:40 smithi042 ceph-mon[32260]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:40.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:22:40.623 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:22:40.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:40 smithi119 ceph-mon[36119]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:40.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:40 smithi136 ceph-mon[35571]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 -- 172.21.15.42:0/3586246825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc105340 msgr2=0x7f51fc10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 --2- 172.21.15.42:0/3586246825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc105340 0x7f51fc10cf10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009ca0 tx=0x7f51f402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 -- 172.21.15.42:0/3586246825 shutdown_connections 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 --2- 172.21.15.42:0/3586246825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc105340 0x7f51fc10cf10 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 --2- 172.21.15.42:0/3586246825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51fc100c20 0x7f51fc104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 --2- 172.21.15.42:0/3586246825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51fc100270 0x7f51fc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 -- 172.21.15.42:0/3586246825 >> 172.21.15.42:0/3586246825 conn(0x7f51fc077230 msgr2=0x7f51fc077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 -- 172.21.15.42:0/3586246825 shutdown_connections 2024-03-12T04:22:40.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 -- 172.21.15.42:0/3586246825 wait complete. 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 Processor -- start 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.726+0000 7f5202aae640 1 -- start start 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f5202aae640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc100270 0x7f51fc197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f5202aae640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51fc105340 0x7f51fc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f5202aae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51fc19be40 0x7f51fc19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f5202aae640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19c830 con 0x7f51fc100270 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f5202aae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19c9a0 con 0x7f51fc19be40 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f5202aae640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19cb10 con 0x7f51fc105340 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc100270 0x7f51fc197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc100270 0x7f51fc197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53840/0 (socket says 172.21.15.42:53840) 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 -- 172.21.15.42:0/732206272 learned_addr learned my addr 172.21.15.42:0/732206272 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520104b640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51fc19be40 0x7f51fc19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 -- 172.21.15.42:0/732206272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51fc105340 msgr2=0x7f51fc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f51fbfff640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51fc105340 0x7f51fc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51fc105340 0x7f51fc197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 -- 172.21.15.42:0/732206272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51fc19be40 msgr2=0x7f51fc19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51fc19be40 0x7f51fc19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 -- 172.21.15.42:0/732206272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4009900 con 0x7f51fc100270 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520104b640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51fc19be40 0x7f51fc19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f520084a640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc100270 0x7f51fc197580 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c890 tx=0x7f51f000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f51f9ffb640 1 -- 172.21.15.42:0/732206272 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004220 con 0x7f51fc100270 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f51f9ffb640 1 -- 172.21.15.42:0/732206272 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51f0007e60 con 0x7f51fc100270 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc1a40f0 con 0x7f51fc100270 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.727+0000 7f51f9ffb640 1 -- 172.21.15.42:0/732206272 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004a00 con 0x7f51fc100270 2024-03-12T04:22:40.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.728+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc1a45f0 con 0x7f51fc100270 2024-03-12T04:22:40.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f51f9ffb640 1 -- 172.21.15.42:0/732206272 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f51f00079b0 con 0x7f51fc100270 2024-03-12T04:22:40.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 Processor -- start 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f51f9ffb640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51ec0a1c20 0x7f51ec0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 -- start start 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f520104b640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51ec0a1c20 0x7f51ec0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f51f9ffb640 1 -- 172.21.15.42:0/732206272 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f51f00cd510 con 0x7f51fc100270 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aed2b0 0x555a94aed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94af67b0 0x555a94af8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94a5c280 con 0x555a94aed2b0 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94a597b0 con 0x555a94aedbd0 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.730+0000 7f5205683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94a54f90 con 0x555a94af67b0 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aed2b0 0x555a94aed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aed2b0 0x555a94aed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53852/0 (socket says 172.21.15.42:53852) 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 -- 172.21.15.42:0/3378947450 learned_addr learned my addr 172.21.15.42:0/3378947450 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f51fbfff640 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 -- 172.21.15.42:0/3378947450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94af67b0 msgr2=0x555a94af8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94af67b0 0x555a94af8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 -- 172.21.15.42:0/3378947450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 msgr2=0x555a94af6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 -- 172.21.15.42:0/3378947450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a94a13570 con 0x555a94aed2b0 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f51fbfff640 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520104b640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51ec0a1c20 0x7f51ec0a40e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006010 tx=0x7f51f4032d70 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f520084a640 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aed2b0 0x555a94aed690 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f51f0096b30 tx=0x7f51f009a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1b27ca43394351 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f51e6ffd640 1 -- 172.21.15.42:0/3378947450 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004380 con 0x555a94aed2b0 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f51e6ffd640 1 -- 172.21.15.42:0/3378947450 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51f00b2e90 con 0x555a94aed2b0 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.731+0000 7f51e6ffd640 1 -- 172.21.15.42:0/3378947450 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00967d0 con 0x555a94aed2b0 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 -- 172.21.15.42:0/3378947450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aed2b0 msgr2=0x555a94aed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aed2b0 0x555a94aed690 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f51f0096b30 tx=0x7f51f009a9a0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 -- 172.21.15.42:0/3378947450 shutdown_connections 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94af67b0 0x555a94af8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 --2- 172.21.15.42:0/3378947450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aed2b0 0x555a94aed690 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 -- 172.21.15.42:0/3378947450 >> 172.21.15.42:0/3378947450 conn(0x555a94ae7990 msgr2=0x555a94ae7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 -- 172.21.15.42:0/3378947450 shutdown_connections 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 -- 172.21.15.42:0/3378947450 wait complete. 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 -- start start 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.732+0000 7f5205683740 1 Processor -- start 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f5205683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f5205683740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f5205683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f5205683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94728430 con 0x555a94af67b0 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f5205683740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a946e4e00 con 0x555a94aedbd0 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f5205683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a949d9fd0 con 0x555a94aed2b0 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f5205683740 10 client.0 _ll_register_callbacks cb 0x555a94a2ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53854/0 (socket says 172.21.15.42:53854) 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 -- 172.21.15.42:0/747536215 learned_addr learned my addr 172.21.15.42:0/747536215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f51fbfff640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 -- 172.21.15.42:0/747536215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 msgr2=0x555a94af61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520084a640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 -- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 msgr2=0x555a94af3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 -- 172.21.15.42:0/747536215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc1a45f0 con 0x555a94af67b0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f51fbfff640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520084a640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f520104b640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af3cb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009ca0 tx=0x7f51f40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4465fc41f7c93d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f51dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.733+0000 7f51dbfff640 1 -- 172.21.15.42:0/747536215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4003d80 con 0x555a94af67b0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f51dbfff640 1 -- 172.21.15.42:0/747536215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f51f4047040 con 0x555a94af67b0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a947547e0 con 0x555a94af67b0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f51dbfff640 1 -- 172.21.15.42:0/747536215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f4004040 con 0x555a94af67b0 2024-03-12T04:22:40.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 10 client.17667 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555a94a53610 con 0x555a94af67b0 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f51dbfff640 1 -- 172.21.15.42:0/747536215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x555a947547e0 con 0x555a94af67b0 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f51dbfff640 1 -- 172.21.15.42:0/747536215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f51f403e040 con 0x555a94af67b0 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 10 client.17667 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 20 client.17667 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b06910 0x555a94b08d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 4 client.17667 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a94791720 con 0x555a94b06910 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.734+0000 7f520084a640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b06910 0x555a94b08d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.735+0000 7f51dbfff640 10 client.17667 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.735+0000 7f520084a640 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b06910 0x555a94b08d30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3a60ca5c6b70f2 server_cookie=d9ed8ca38da5c1ca in_seq=0 out_seq=0 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.735+0000 7f51dbfff640 1 -- 172.21.15.42:0/747536215 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a94791720 con 0x555a94b06910 2024-03-12T04:22:40.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.735+0000 7f51dbfff640 10 client.17667 handle_command_reply: tid=0 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.735+0000 7f5205683740 1 client.17667 shutdown 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 msgr2=0x555a94af3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af3cb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009ca0 tx=0x7f51f40095c0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 shutdown_connections 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b06910 0x555a94b08d30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af3cb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f51dbfff640 0 client.17667 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 --2- 172.21.15.42:0/747536215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 >> 172.21.15.42:0/747536215 conn(0x555a94ae7990 msgr2=0x555a94ae9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 shutdown_connections 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 1 -- 172.21.15.42:0/747536215 wait complete. 2024-03-12T04:22:40.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.736+0000 7f5205683740 20 client.17667 trim_cache size 0 max 16384 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 Processor -- start 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 -- start start 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94ae3410 con 0x555a94af67b0 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94ae77f0 con 0x555a94aedbd0 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f5205683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94b06690 con 0x555a94aed2b0 2024-03-12T04:22:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53864/0 (socket says 172.21.15.42:53864) 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 -- 172.21.15.42:0/1642097957 learned_addr learned my addr 172.21.15.42:0/1642097957 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 -- 172.21.15.42:0/1642097957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 msgr2=0x555a94af5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 --2- 172.21.15.42:0/1642097957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 -- 172.21.15.42:0/1642097957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 msgr2=0x555a94af61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 --2- 172.21.15.42:0/1642097957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 -- 172.21.15.42:0/1642097957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a949efac0 con 0x555a94af67b0 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f520104b640 1 --2- 172.21.15.42:0/1642097957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af1e40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f51f4056040 tx=0x7f51f4032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ae2ae48e5a3b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.751+0000 7f51e67fc640 1 -- 172.21.15.42:0/1642097957 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4042650 con 0x555a94af67b0 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f51e67fc640 1 -- 172.21.15.42:0/1642097957 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51f404b440 con 0x555a94af67b0 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f51e67fc640 1 -- 172.21.15.42:0/1642097957 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4054690 con 0x555a94af67b0 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 -- 172.21.15.42:0/1642097957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 msgr2=0x555a94af1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 --2- 172.21.15.42:0/1642097957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af1e40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f51f4056040 tx=0x7f51f4032fb0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 -- 172.21.15.42:0/1642097957 shutdown_connections 2024-03-12T04:22:40.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 --2- 172.21.15.42:0/1642097957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94af67b0 0x555a94af1e40 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 --2- 172.21.15.42:0/1642097957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94aedbd0 0x555a94af61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 --2- 172.21.15.42:0/1642097957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94af5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 -- 172.21.15.42:0/1642097957 >> 172.21.15.42:0/1642097957 conn(0x555a94ae7990 msgr2=0x555a94b0d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 -- 172.21.15.42:0/1642097957 shutdown_connections 2024-03-12T04:22:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 -- 172.21.15.42:0/1642097957 wait complete. 2024-03-12T04:22:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 -- start start 2024-03-12T04:22:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.752+0000 7f5205683740 1 Processor -- start 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f5205683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94afd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f5205683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aedbd0 0x555a94af9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f5205683740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94af67b0 0x555a94afa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f5205683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94afa690 con 0x555a94aedbd0 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f5205683740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94afa800 con 0x555a94af67b0 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f5205683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94afa9c0 con 0x555a94aed2b0 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f5205683740 10 client.0 _ll_register_callbacks cb 0x7f51f0099f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aedbd0 0x555a94af9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aedbd0 0x555a94af9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53878/0 (socket says 172.21.15.42:53878) 2024-03-12T04:22:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 -- 172.21.15.42:0/3308651648 learned_addr learned my addr 172.21.15.42:0/3308651648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f520084a640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94afd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 -- 172.21.15.42:0/3308651648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 msgr2=0x555a94afd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94afd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f520104b640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94af67b0 0x555a94afa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 -- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94af67b0 msgr2=0x555a94afa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94af67b0 0x555a94afa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 -- 172.21.15.42:0/3308651648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a94a53610 con 0x555a94aedbd0 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f520084a640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94afd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f520104b640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94af67b0 0x555a94afa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51fbfff640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aedbd0 0x555a94af9c10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005940 tx=0x7f51e800f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86bc9e1d77979a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.753+0000 7f51dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f51dbfff640 1 -- 172.21.15.42:0/3308651648 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e800b790 con 0x555a94aedbd0 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51f00b2e90 con 0x555a94aedbd0 2024-03-12T04:22:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f51dbfff640 1 -- 172.21.15.42:0/3308651648 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f51e800b8f0 con 0x555a94aedbd0 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f51dbfff640 1 -- 172.21.15.42:0/3308651648 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e800bab0 con 0x555a94aedbd0 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 10 client.17673 fetch_fsmap learned FSMap version 27 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555a94a156a0 con 0x555a94aedbd0 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f51dbfff640 1 -- 172.21.15.42:0/3308651648 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51e8024020 con 0x555a94aedbd0 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f51dbfff640 1 -- 172.21.15.42:0/3308651648 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f51e801f390 con 0x555a94aedbd0 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 10 client.17673 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 20 client.17673 populate_metadata read hostname 'smithi042' 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b5e380 0x555a94b607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 4 client.17673 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a947e0890 con 0x555a94b5e380 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.754+0000 7f520084a640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b5e380 0x555a94b607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f520084a640 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b5e380 0x555a94b607a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad5067ac3e4dbaaf server_cookie=dc89392377eb2177 in_seq=0 out_seq=0 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f51dbfff640 10 client.17673 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f51dbfff640 1 -- 172.21.15.42:0/3308651648 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555a947e0890 con 0x555a94b5e380 2024-03-12T04:22:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f51dbfff640 10 client.17673 handle_command_reply: tid=0 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 client.17673 shutdown 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aedbd0 msgr2=0x555a94af9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aedbd0 0x555a94af9c10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005940 tx=0x7f51e800f5d0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 shutdown_connections 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x555a94b5e380 0x555a94b607a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555a94af67b0 0x555a94afa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a94aedbd0 0x555a94af9c10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 --2- 172.21.15.42:0/3308651648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a94aed2b0 0x555a94afd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f51dbfff640 0 client.17673 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 >> 172.21.15.42:0/3308651648 conn(0x555a94ae7990 msgr2=0x555a94af1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 shutdown_connections 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 1 -- 172.21.15.42:0/3308651648 wait complete. 2024-03-12T04:22:40.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.755+0000 7f5205683740 20 client.17673 trim_cache size 0 max 16384 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: "8b314c9f-fa23-4562-8a36-cd5e0a6aa183": { 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "8b314c9f-fa23-4562-8a36-cd5e0a6aa183", 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:22:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:22:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51ec0a1c20 msgr2=0x7f51ec0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51ec0a1c20 0x7f51ec0a40e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006010 tx=0x7f51f4032d70 comp rx=0 tx=0).stop 2024-03-12T04:22:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc100270 msgr2=0x7f51fc197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:22:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc100270 0x7f51fc197580 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c890 tx=0x7f51f000cd50 comp rx=0 tx=0).stop 2024-03-12T04:22:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 shutdown_connections 2024-03-12T04:22:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f51ec0a1c20 0x7f51ec0a40e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51fc19be40 0x7f51fc19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51fc105340 0x7f51fc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 --2- 172.21.15.42:0/732206272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51fc100270 0x7f51fc197580 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:22:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 >> 172.21.15.42:0/732206272 conn(0x7f51fc077230 msgr2=0x7f51fc0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:22:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 shutdown_connections 2024-03-12T04:22:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:22:40.756+0000 7f5202aae640 1 -- 172.21.15.42:0/732206272 wait complete. 2024-03-12T04:22:40.769 INFO:tasks.cephfs.filesystem:scrub status for tag:8b314c9f-fa23-4562-8a36-cd5e0a6aa183 - {'path': '/', 'tag': '8b314c9f-fa23-4562-8a36-cd5e0a6aa183', 'options': 'recursive,force'} 2024-03-12T04:22:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:41 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:22:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:41 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:41 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:22:41.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:41 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:41 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:22:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:41 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:41 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:22:41.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:41 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:41 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:22:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:41 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:41 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:22:41.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:41 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:22:42.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:42 smithi119 ceph-mon[36119]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:42 smithi136 ceph-mon[35571]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:42.717 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:42 smithi042 ceph-mon[32260]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:44 smithi119 ceph-mon[36119]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:44 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:22:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:44 smithi136 ceph-mon[35571]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:44 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:22:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:44 smithi042 ceph-mon[32260]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:44 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:22:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:46 smithi119 ceph-mon[36119]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:46 smithi136 ceph-mon[35571]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:46 smithi042 ceph-mon[32260]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:48 smithi119 ceph-mon[36119]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:48 smithi136 ceph-mon[35571]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:48.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:48 smithi042 ceph-mon[32260]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:50.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:50 smithi119 ceph-mon[36119]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:50.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:50 smithi136 ceph-mon[35571]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:50.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:50 smithi042 ceph-mon[32260]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:52.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:52 smithi119 ceph-mon[36119]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:52 smithi136 ceph-mon[35571]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:52.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:52 smithi042 ceph-mon[32260]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:22:54.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:54 smithi119 ceph-mon[36119]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:54 smithi136 ceph-mon[35571]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:54 smithi042 ceph-mon[32260]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:22:56.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:56 smithi119 ceph-mon[36119]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:56.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:56 smithi136 ceph-mon[35571]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:56.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:56 smithi042 ceph-mon[32260]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:22:58.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:22:58 smithi119 ceph-mon[36119]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:58.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:22:58 smithi136 ceph-mon[35571]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:22:58.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:22:58 smithi042 ceph-mon[32260]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:00.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:00 smithi119 ceph-mon[36119]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:00.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:00 smithi136 ceph-mon[35571]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:00.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:00 smithi042 ceph-mon[32260]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:02 smithi119 ceph-mon[36119]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:02.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:02 smithi136 ceph-mon[35571]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:02.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:02 smithi042 ceph-mon[32260]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:04.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:04 smithi119 ceph-mon[36119]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:23:04.710 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:04 smithi136 ceph-mon[35571]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:23:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:04 smithi042 ceph-mon[32260]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:23:06.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:06 smithi119 ceph-mon[36119]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:23:06.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:06 smithi136 ceph-mon[35571]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:23:06.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:06 smithi042 ceph-mon[32260]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:23:08.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:08 smithi119 ceph-mon[36119]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:23:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:08 smithi136 ceph-mon[35571]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:23:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:08 smithi042 ceph-mon[32260]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:23:10.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:10 smithi119 ceph-mon[36119]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:23:10.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:10 smithi119 ceph-mon[36119]: osdmap e93: 12 total, 12 up, 12 in 2024-03-12T04:23:10.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:10 smithi136 ceph-mon[35571]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:23:10.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:10 smithi136 ceph-mon[35571]: osdmap e93: 12 total, 12 up, 12 in 2024-03-12T04:23:10.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:23:10.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:23:10.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:10 smithi042 ceph-mon[32260]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:23:10.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:10 smithi042 ceph-mon[32260]: osdmap e93: 12 total, 12 up, 12 in 2024-03-12T04:23:10.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 -- 172.21.15.42:0/896801040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba84100190 msgr2=0x7fba84100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/896801040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba84100190 0x7fba84100570 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fba80009900 tx=0x7fba8002bcd0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 -- 172.21.15.42:0/896801040 shutdown_connections 2024-03-12T04:23:10.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/896801040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba841054b0 0x7fba84105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/896801040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba84100ab0 0x7fba84104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/896801040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba84100190 0x7fba84100570 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 -- 172.21.15.42:0/896801040 >> 172.21.15.42:0/896801040 conn(0x7fba84077230 msgr2=0x7fba84077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 -- 172.21.15.42:0/896801040 shutdown_connections 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 -- 172.21.15.42:0/896801040 wait complete. 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 Processor -- start 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.889+0000 7fba8b3f6640 1 -- start start 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba8b3f6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba84100190 0x7fba841a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba8b3f6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba84100ab0 0x7fba841a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba8b3f6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba841054b0 0x7fba8419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba8b3f6640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8419e2a0 con 0x7fba841054b0 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba8b3f6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8419e410 con 0x7fba84100ab0 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba8b3f6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8419e5d0 con 0x7fba84100190 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba841054b0 0x7fba8419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba841054b0 0x7fba8419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50550/0 (socket says 172.21.15.42:50550) 2024-03-12T04:23:10.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 -- 172.21.15.42:0/564423636 learned_addr learned my addr 172.21.15.42:0/564423636 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba88991640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba84100ab0 0x7fba841a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89192640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba84100190 0x7fba841a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 -- 172.21.15.42:0/564423636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba84100190 msgr2=0x7fba841a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba84100190 0x7fba841a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 -- 172.21.15.42:0/564423636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba84100ab0 msgr2=0x7fba841a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba84100ab0 0x7fba841a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 -- 172.21.15.42:0/564423636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80009560 con 0x7fba841054b0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89192640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba84100190 0x7fba841a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba89993640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba841054b0 0x7fba8419db10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b970 tx=0x7fba7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.890+0000 7fba7a7fc640 1 -- 172.21.15.42:0/564423636 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00cb40 con 0x7fba841054b0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.891+0000 7fba7a7fc640 1 -- 172.21.15.42:0/564423636 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba7c00cca0 con 0x7fba841054b0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.891+0000 7fba7a7fc640 1 -- 172.21.15.42:0/564423636 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0076e0 con 0x7fba841054b0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.891+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8419e8b0 con 0x7fba841054b0 2024-03-12T04:23:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.891+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba841aa9d0 con 0x7fba841054b0 2024-03-12T04:23:10.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.893+0000 7fba7a7fc640 1 -- 172.21.15.42:0/564423636 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fba7c014440 con 0x7fba841054b0 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.893+0000 7fba8dfc5740 1 Processor -- start 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba8dfc5740 1 -- start start 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bcabd0 0x560754bd3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba7a7fc640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fba700a1c20 0x7fba700a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bd37b0 0x560754bd5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754b39280 con 0x560754bca2b0 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754b367b0 con 0x560754bcabd0 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754b31f90 con 0x560754bd37b0 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50556/0 (socket says 172.21.15.42:50556) 2024-03-12T04:23:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 -- 172.21.15.42:0/3144661 learned_addr learned my addr 172.21.15.42:0/3144661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba7a7fc640 1 -- 172.21.15.42:0/564423636 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fba7c0ccf20 con 0x7fba841054b0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fba700a1c20 0x7fba700a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba89993640 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bd37b0 0x560754bd5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba89192640 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bcabd0 0x560754bd3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 -- 172.21.15.42:0/3144661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bd37b0 msgr2=0x560754bd5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bd37b0 0x560754bd5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 -- 172.21.15.42:0/3144661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bcabd0 msgr2=0x560754bd3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bcabd0 0x560754bd3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba88991640 1 -- 172.21.15.42:0/3144661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560754af0570 con 0x560754bca2b0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.894+0000 7fba89192640 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bcabd0 0x560754bd3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.895+0000 7fba88991640 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bca690 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fba74011ab0 tx=0x7fba74011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb3f6bed9555af server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.895+0000 7fba88991640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fba700a1c20 0x7fba700a40e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba7400f850 tx=0x7fba7401e000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.895+0000 7fba6f7fe640 1 -- 172.21.15.42:0/3144661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74011050 con 0x560754bca2b0 2024-03-12T04:23:10.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.895+0000 7fba6f7fe640 1 -- 172.21.15.42:0/3144661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba740044e0 con 0x560754bca2b0 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.895+0000 7fba6f7fe640 1 -- 172.21.15.42:0/3144661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74005600 con 0x560754bca2b0 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3144661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 msgr2=0x560754bca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bca690 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fba74011ab0 tx=0x7fba74011f70 comp rx=0 tx=0).stop 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3144661 shutdown_connections 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bd37b0 0x560754bd5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bcabd0 0x560754bd3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3144661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bca690 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3144661 >> 172.21.15.42:0/3144661 conn(0x560754bc4990 msgr2=0x560754bc4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3144661 shutdown_connections 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3144661 wait complete. 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- start start 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 Processor -- start 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bd3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bd0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bd37b0 0x560754bd0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754805430 con 0x560754bca2b0 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607547c1e00 con 0x560754bd37b0 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754ab6fd0 con 0x560754bcabd0 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.896+0000 7fba8dfc5740 10 client.0 _ll_register_callbacks cb 0x560754b0cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bd3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bd3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50558/0 (socket says 172.21.15.42:50558) 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 -- 172.21.15.42:0/3910139417 learned_addr learned my addr 172.21.15.42:0/3910139417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba88991640 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bd0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89993640 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bd37b0 0x560754bd0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 -- 172.21.15.42:0/3910139417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 msgr2=0x560754bd0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bd0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 -- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bd37b0 msgr2=0x560754bd0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bd37b0 0x560754bd0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 -- 172.21.15.42:0/3910139417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba74019040 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba89192640 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bd3200 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fba8002c3c0 tx=0x7fba8002cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b901d275693bc16d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba6cff9640 1 -- 172.21.15.42:0/3910139417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80004450 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba6cff9640 1 -- 172.21.15.42:0/3910139417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fba80044070 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607548317e0 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba6cff9640 1 -- 172.21.15.42:0/3910139417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba80005d50 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba8dfc5740 10 client.17685 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.897+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560754b30610 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba6cff9640 1 -- 172.21.15.42:0/3910139417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5607548317e0 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba6cff9640 1 -- 172.21.15.42:0/3910139417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fba8004f020 con 0x560754bca2b0 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba8dfc5740 10 client.17685 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba8dfc5740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba8dfc5740 20 client.17685 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754be39a0 0x560754be5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba8dfc5740 4 client.17685 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56075486e720 con 0x560754be39a0 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.898+0000 7fba88991640 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754be39a0 0x560754be5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.899+0000 7fba6cff9640 10 client.17685 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.899+0000 7fba88991640 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754be39a0 0x560754be5dc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d5d67ec692ec1b2 server_cookie=954f8a2d5cb59e5a in_seq=0 out_seq=0 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.899+0000 7fba6cff9640 1 -- 172.21.15.42:0/3910139417 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56075486e720 con 0x560754be39a0 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.899+0000 7fba6cff9640 10 client.17685 handle_command_reply: tid=0 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.899+0000 7fba8dfc5740 1 client.17685 shutdown 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 msgr2=0x560754bd3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bd3200 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fba8002c3c0 tx=0x7fba8002cb40 comp rx=0 tx=0).stop 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 shutdown_connections 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754be39a0 0x560754be5dc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bd37b0 0x560754bd0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bd0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba6cff9640 0 client.17685 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:10.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/3910139417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bca2b0 0x560754bd3200 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 >> 172.21.15.42:0/3910139417 conn(0x560754bc4990 msgr2=0x560754bc6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:10.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 shutdown_connections 2024-03-12T04:23:10.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 1 -- 172.21.15.42:0/3910139417 wait complete. 2024-03-12T04:23:10.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.900+0000 7fba8dfc5740 20 client.17685 trim_cache size 0 max 16384 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 Processor -- start 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 -- start start 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bda600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bdab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075486e720 con 0x560754bd37b0 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754bcb420 con 0x560754bca2b0 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754bc9f10 con 0x560754bcabd0 2024-03-12T04:23:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba89993640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba89993640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50562/0 (socket says 172.21.15.42:50562) 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.915+0000 7fba89993640 1 -- 172.21.15.42:0/1547363177 learned_addr learned my addr 172.21.15.42:0/1547363177 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba89192640 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bda600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba88991640 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bdab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba89993640 1 -- 172.21.15.42:0/1547363177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 msgr2=0x560754bdab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba89993640 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bdab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba89993640 1 -- 172.21.15.42:0/1547363177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 msgr2=0x560754bda600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba89993640 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bda600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba89993640 1 -- 172.21.15.42:0/1547363177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560754accac0 con 0x560754bd37b0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba88991640 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bdab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba89993640 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd2cc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b940 tx=0x7fba7c0a6ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f635e21ebe4fdad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba6effd640 1 -- 172.21.15.42:0/1547363177 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c09ddb0 con 0x560754bd37b0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba6effd640 1 -- 172.21.15.42:0/1547363177 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba7c0a2400 con 0x560754bd37b0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.916+0000 7fba6effd640 1 -- 172.21.15.42:0/1547363177 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0be510 con 0x560754bd37b0 2024-03-12T04:23:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1547363177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 msgr2=0x560754bd2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd2cc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b940 tx=0x7fba7c0a6ed0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1547363177 shutdown_connections 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd2cc0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bdab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1547363177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bda600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1547363177 >> 172.21.15.42:0/1547363177 conn(0x560754bc4990 msgr2=0x560754bcb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1547363177 shutdown_connections 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1547363177 wait complete. 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- start start 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 Processor -- start 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bdab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bd8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754bd9cf0 con 0x560754bd37b0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754c39e90 con 0x560754bca2b0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.917+0000 7fba8dfc5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560754c39fd0 con 0x560754bcabd0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba8dfc5740 10 client.0 _ll_register_callbacks cb 0x7fba74009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50574/0 (socket says 172.21.15.42:50574) 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 -- 172.21.15.42:0/1380830409 learned_addr learned my addr 172.21.15.42:0/1380830409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89192640 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bdab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 -- 172.21.15.42:0/1380830409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 msgr2=0x560754bd8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bd8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 -- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 msgr2=0x560754bdab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bdab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 -- 172.21.15.42:0/1380830409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba841aa9d0 con 0x560754bd37b0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89192640 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bdab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba89993640 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd9620 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009a00 tx=0x7fba7c099e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4120a33e86f9336 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:10.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba6cff9640 1 -- 172.21.15.42:0/1380830409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c09ddb0 con 0x560754bd37b0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba6cff9640 1 -- 172.21.15.42:0/1380830409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fba7c0146f0 con 0x560754bd37b0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba740044e0 con 0x560754bd37b0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba6cff9640 1 -- 172.21.15.42:0/1380830409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba7c0be470 con 0x560754bd37b0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba8dfc5740 10 client.17691 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560754af26a0 con 0x560754bd37b0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.918+0000 7fba6cff9640 1 -- 172.21.15.42:0/1380830409 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba7c0b8030 con 0x560754bd37b0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba6cff9640 1 -- 172.21.15.42:0/1380830409 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fba7c0a2090 con 0x560754bd37b0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba8dfc5740 10 client.17691 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba8dfc5740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba8dfc5740 20 client.17691 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754c3b350 0x560754c3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba8dfc5740 4 client.17691 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5607548bd890 con 0x560754c3b350 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba89192640 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754c3b350 0x560754c3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba89192640 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754c3b350 0x560754c3d740 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2340fee273628c7 server_cookie=351a73cd62e85bac in_seq=0 out_seq=0 2024-03-12T04:23:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.919+0000 7fba6cff9640 10 client.17691 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba6cff9640 1 -- 172.21.15.42:0/1380830409 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5607548bd890 con 0x560754c3b350 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba6cff9640 10 client.17691 handle_command_reply: tid=0 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 client.17691 shutdown 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 msgr2=0x560754bd9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd9620 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009a00 tx=0x7fba7c099e50 comp rx=0 tx=0).stop 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 shutdown_connections 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560754c3b350 0x560754c3d740 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560754bd37b0 0x560754bd9620 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560754bcabd0 0x560754bd8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba6cff9640 0 client.17691 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 --2- 172.21.15.42:0/1380830409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560754bca2b0 0x560754bdab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 >> 172.21.15.42:0/1380830409 conn(0x560754bc4990 msgr2=0x560754bcfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 shutdown_connections 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 1 -- 172.21.15.42:0/1380830409 wait complete. 2024-03-12T04:23:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8dfc5740 20 client.17691 trim_cache size 0 max 16384 2024-03-12T04:23:10.929 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:23:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:23:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:23:10.929 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fba700a1c20 msgr2=0x7fba700a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fba700a1c20 0x7fba700a40e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba7400f850 tx=0x7fba7401e000 comp rx=0 tx=0).stop 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba841054b0 msgr2=0x7fba8419db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba841054b0 0x7fba8419db10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b970 tx=0x7fba7c00be30 comp rx=0 tx=0).stop 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 shutdown_connections 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fba700a1c20 0x7fba700a40e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba841054b0 0x7fba8419db10 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba84100ab0 0x7fba841a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 --2- 172.21.15.42:0/564423636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba84100190 0x7fba841a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:10.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.920+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 >> 172.21.15.42:0/564423636 conn(0x7fba84077230 msgr2=0x7fba84102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:10.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.921+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 shutdown_connections 2024-03-12T04:23:10.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:10.921+0000 7fba8b3f6640 1 -- 172.21.15.42:0/564423636 wait complete. 2024-03-12T04:23:10.933 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:23:10.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:23:11.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 -- 172.21.15.42:0/452587793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 msgr2=0x7f75a010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 --2- 172.21.15.42:0/452587793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a010cf50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f759c009cd0 tx=0x7f759c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 -- 172.21.15.42:0/452587793 shutdown_connections 2024-03-12T04:23:11.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 --2- 172.21.15.42:0/452587793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a010cf50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 --2- 172.21.15.42:0/452587793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a0100c60 0x7f75a0104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 --2- 172.21.15.42:0/452587793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75a01002b0 0x7f75a0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 -- 172.21.15.42:0/452587793 >> 172.21.15.42:0/452587793 conn(0x7f75a0077120 msgr2=0x7f75a0077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 -- 172.21.15.42:0/452587793 shutdown_connections 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 -- 172.21.15.42:0/452587793 wait complete. 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 Processor -- start 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.075+0000 7f75a77e2640 1 -- start start 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a01002b0 0x7f75a0101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a0102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75a01027c0 0x7f75a01a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01a5e90 con 0x7f75a0105380 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01a6000 con 0x7f75a01002b0 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01a61c0 con 0x7f75a01027c0 2024-03-12T04:23:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a0102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a0102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50592/0 (socket says 172.21.15.42:50592) 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 -- 172.21.15.42:0/801607099 learned_addr learned my addr 172.21.15.42:0/801607099 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a557e640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a01002b0 0x7f75a0101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a5d7f640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75a01027c0 0x7f75a01a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 -- 172.21.15.42:0/801607099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75a01027c0 msgr2=0x7f75a01a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75a01027c0 0x7f75a01a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 -- 172.21.15.42:0/801607099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a01002b0 msgr2=0x7f75a0101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a01002b0 0x7f75a0101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 -- 172.21.15.42:0/801607099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c009900 con 0x7f75a0105380 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a557e640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a01002b0 0x7f75a0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a4d7d640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a0102280 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7590005fd0 tx=0x7f759000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75967fc640 1 -- 172.21.15.42:0/801607099 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590009c30 con 0x7f75a0105380 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75967fc640 1 -- 172.21.15.42:0/801607099 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75900044d0 con 0x7f75a0105380 2024-03-12T04:23:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a01a64a0 con 0x7f75a0105380 2024-03-12T04:23:11.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75967fc640 1 -- 172.21.15.42:0/801607099 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590003e50 con 0x7f75a0105380 2024-03-12T04:23:11.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.076+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a01a6a00 con 0x7f75a0105380 2024-03-12T04:23:11.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.078+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a00683c0 con 0x7f75a0105380 2024-03-12T04:23:11.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.079+0000 7f75967fc640 1 -- 172.21.15.42:0/801607099 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7590003fb0 con 0x7f75a0105380 2024-03-12T04:23:11.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.079+0000 7f75967fc640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f758c0a1c50 0x7f758c0a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.079+0000 7f75a557e640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f758c0a1c50 0x7f758c0a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.079+0000 7f75967fc640 1 -- 172.21.15.42:0/801607099 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f75900cd420 con 0x7f75a0105380 2024-03-12T04:23:11.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.080+0000 7f75a557e640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f758c0a1c50 0x7f758c0a4110 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7598009b50 tx=0x7f7598005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.084+0000 7f75967fc640 1 -- 172.21.15.42:0/801607099 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75900962f0 con 0x7f75a0105380 2024-03-12T04:23:11.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.305+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75a0103270 con 0x7f75a0105380 2024-03-12T04:23:11.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.306+0000 7f75967fc640 1 -- 172.21.15.42:0/801607099 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f75900999b0 con 0x7f75a0105380 2024-03-12T04:23:11.308 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:23:11.308 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:23:11.310 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f758c0a1c50 msgr2=0x7f758c0a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f758c0a1c50 0x7f758c0a4110 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7598009b50 tx=0x7f7598005f90 comp rx=0 tx=0).stop 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 msgr2=0x7f75a0102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a0102280 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7590005fd0 tx=0x7f759000dde0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 shutdown_connections 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f758c0a1c50 0x7f758c0a4110 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75a01027c0 0x7f75a01a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75a0105380 0x7f75a0102280 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 --2- 172.21.15.42:0/801607099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a01002b0 0x7f75a0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 >> 172.21.15.42:0/801607099 conn(0x7f75a0077120 msgr2=0x7f75a0069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 shutdown_connections 2024-03-12T04:23:11.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.308+0000 7f75a77e2640 1 -- 172.21.15.42:0/801607099 wait complete. 2024-03-12T04:23:11.320 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:11.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:23:11.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 -- 172.21.15.42:0/1545683470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264106460 msgr2=0x7f626410e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 --2- 172.21.15.42:0/1545683470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264106460 0x7f626410e030 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6260009cd0 tx=0x7f626002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 -- 172.21.15.42:0/1545683470 shutdown_connections 2024-03-12T04:23:11.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 --2- 172.21.15.42:0/1545683470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264106460 0x7f626410e030 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 --2- 172.21.15.42:0/1545683470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6264101d40 0x7f6264105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 --2- 172.21.15.42:0/1545683470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6264101390 0x7f6264101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 -- 172.21.15.42:0/1545683470 >> 172.21.15.42:0/1545683470 conn(0x7f62640fcbb0 msgr2=0x7f62640fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 -- 172.21.15.42:0/1545683470 shutdown_connections 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.424+0000 7f626c14f640 1 -- 172.21.15.42:0/1545683470 wait complete. 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 Processor -- start 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 -- start start 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264101390 0x7f6264197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6264106460 0x7f6264197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626419c010 0x7f626419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626419ca00 con 0x7f6264101390 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626419cb70 con 0x7f626419c010 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626c14f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626419cce0 con 0x7f6264106460 2024-03-12T04:23:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f6269eeb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264101390 0x7f6264197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f6269eeb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264101390 0x7f6264197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51352/0 (socket says 172.21.15.42:51352) 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f6269eeb640 1 -- 172.21.15.42:0/1680217974 learned_addr learned my addr 172.21.15.42:0/1680217974 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f62696ea640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6264106460 0x7f6264197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.425+0000 7f626a6ec640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626419c010 0x7f626419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f6269eeb640 1 -- 172.21.15.42:0/1680217974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6264106460 msgr2=0x7f6264197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6264106460 0x7f6264197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f6269eeb640 1 -- 172.21.15.42:0/1680217974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626419c010 msgr2=0x7f626419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626419c010 0x7f626419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f6269eeb640 1 -- 172.21.15.42:0/1680217974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260009900 con 0x7f6264101390 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264101390 0x7f6264197750 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d950 tx=0x7f625c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f625affd640 1 -- 172.21.15.42:0/1680217974 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c009c30 con 0x7f6264101390 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f625affd640 1 -- 172.21.15.42:0/1680217974 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f625c009d90 con 0x7f6264101390 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62641a42c0 con 0x7f6264101390 2024-03-12T04:23:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f625affd640 1 -- 172.21.15.42:0/1680217974 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c00f6b0 con 0x7f6264101390 2024-03-12T04:23:11.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.426+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62641a4770 con 0x7f6264101390 2024-03-12T04:23:11.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.428+0000 7f625affd640 1 -- 172.21.15.42:0/1680217974 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f625c010aa0 con 0x7f6264101390 2024-03-12T04:23:11.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.428+0000 7f626ee03740 1 Processor -- start 2024-03-12T04:23:11.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.428+0000 7f626ee03740 1 -- start start 2024-03-12T04:23:11.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626ee03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffb250 0x5636cdffb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626ee03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffbb70 0x5636ce004210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626ee03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce006b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626ee03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdfb6960 con 0x5636ce004750 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f625affd640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62500a1ba0 0x7f62500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626ee03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdf6a310 con 0x5636cdffb250 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626ee03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdf67840 con 0x5636cdffbb70 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626a6ec640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62500a1ba0 0x7f62500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce006b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce006b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51354/0 (socket says 172.21.15.42:51354) 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 -- 172.21.15.42:0/1393619857 learned_addr learned my addr 172.21.15.42:0/1393619857 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f625affd640 1 -- 172.21.15.42:0/1680217974 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f625c0cd620 con 0x7f6264101390 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626a6ec640 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffb250 0x5636cdffb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffbb70 0x5636ce004210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 -- 172.21.15.42:0/1393619857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffbb70 msgr2=0x5636ce004210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffbb70 0x5636ce004210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 -- 172.21.15.42:0/1393619857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffb250 msgr2=0x5636cdffb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffb250 0x5636cdffb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 -- 172.21.15.42:0/1393619857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636cdf25060 con 0x5636ce004750 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffbb70 0x5636ce004210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f626a6ec640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62500a1ba0 0x7f62500a4060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6260006010 tx=0x7f62600324d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.429+0000 7f62696ea640 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce006b40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f625400b7c0 tx=0x7f625400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2a709fb1e8e95e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f624ffff640 1 -- 172.21.15.42:0/1393619857 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400d950 con 0x5636ce004750 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f624ffff640 1 -- 172.21.15.42:0/1393619857 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6254013480 con 0x5636ce004750 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f624ffff640 1 -- 172.21.15.42:0/1393619857 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254012640 con 0x5636ce004750 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 -- 172.21.15.42:0/1393619857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 msgr2=0x5636ce006b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce006b40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f625400b7c0 tx=0x7f625400bc80 comp rx=0 tx=0).stop 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 -- 172.21.15.42:0/1393619857 shutdown_connections 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce006b40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffbb70 0x5636ce004210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 --2- 172.21.15.42:0/1393619857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffb250 0x5636cdffb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 -- 172.21.15.42:0/1393619857 >> 172.21.15.42:0/1393619857 conn(0x5636cdff5980 msgr2=0x5636cdff5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 -- 172.21.15.42:0/1393619857 shutdown_connections 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.430+0000 7f626ee03740 1 -- 172.21.15.42:0/1393619857 wait complete. 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 -- start start 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 Processor -- start 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce001730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce0021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdf63020 con 0x5636ce004750 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdc36430 con 0x5636cdffbb70 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdbf2e00 con 0x5636cdffb250 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626ee03740 10 client.0 _ll_register_callbacks cb 0x5636cdf0acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626a6ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce0021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626a6ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce0021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51364/0 (socket says 172.21.15.42:51364) 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626a6ec640 1 -- 172.21.15.42:0/1667830039 learned_addr learned my addr 172.21.15.42:0/1667830039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce001730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f62696ea640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.431+0000 7f626a6ec640 1 -- 172.21.15.42:0/1667830039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 msgr2=0x5636ce001730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626a6ec640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce001730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626a6ec640 1 -- 172.21.15.42:0/1667830039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 msgr2=0x5636ce001c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626a6ec640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626a6ec640 1 -- 172.21.15.42:0/1667830039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625400e040 con 0x5636ce004750 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f62696ea640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626a6ec640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce0021b0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6260032800 tx=0x7f6260046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e21a90c6f3ca3ac server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f624d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f624d7fa640 1 -- 172.21.15.42:0/1667830039 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260004040 con 0x5636ce004750 2024-03-12T04:23:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f624d7fa640 1 -- 172.21.15.42:0/1667830039 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f62600041a0 con 0x5636ce004750 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636cdc627e0 con 0x5636ce004750 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f624d7fa640 1 -- 172.21.15.42:0/1667830039 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6260004360 con 0x5636ce004750 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 10 client.17709 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5636cdf616a0 con 0x5636ce004750 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f624d7fa640 1 -- 172.21.15.42:0/1667830039 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f626005b020 con 0x5636ce004750 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f624d7fa640 1 -- 172.21.15.42:0/1667830039 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f626003e040 con 0x5636ce004750 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 10 client.17709 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 10 client.17709 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 20 client.17709 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce014b10 0x5636ce016f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 4 client.17709 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.432+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636cdc9f720 con 0x5636ce014b10 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce014b10 0x5636ce016f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f624d7fa640 10 client.17709 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f6269eeb640 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce014b10 0x5636ce016f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da83f27eaf3e8658 server_cookie=8cd7df7c2dc569db in_seq=0 out_seq=0 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f624d7fa640 1 -- 172.21.15.42:0/1667830039 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636cdc9f720 con 0x5636ce014b10 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f624d7fa640 10 client.17709 handle_command_reply: tid=0 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f626ee03740 1 client.17709 shutdown 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 msgr2=0x5636ce0021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.433+0000 7f626ee03740 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce0021b0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6260032800 tx=0x7f6260046040 comp rx=0 tx=0).stop 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 shutdown_connections 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce014b10 0x5636ce016f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce0021b0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f624d7fa640 0 client.17709 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 --2- 172.21.15.42:0/1667830039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce001730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 >> 172.21.15.42:0/1667830039 conn(0x5636cdff5980 msgr2=0x5636cdff76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 shutdown_connections 2024-03-12T04:23:11.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 1 -- 172.21.15.42:0/1667830039 wait complete. 2024-03-12T04:23:11.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.434+0000 7f626ee03740 20 client.17709 trim_cache size 0 max 16384 2024-03-12T04:23:11.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.445+0000 7f626ee03740 1 Processor -- start 2024-03-12T04:23:11.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.445+0000 7f626ee03740 1 -- start start 2024-03-12T04:23:11.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626ee03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636cdffb250 0x5636ce003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626ee03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626ee03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636ce004750 0x5636cdffe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626ee03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdee8130 con 0x5636cdffb250 2024-03-12T04:23:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626ee03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636cdff15c0 con 0x5636cdffbb70 2024-03-12T04:23:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626ee03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ce01c160 con 0x5636ce004750 2024-03-12T04:23:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636cdffb250 0x5636ce003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636cdffb250 0x5636ce003e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51368/0 (socket says 172.21.15.42:51368) 2024-03-12T04:23:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 -- 172.21.15.42:0/873328611 learned_addr learned my addr 172.21.15.42:0/873328611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f62696ea640 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626a6ec640 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636ce004750 0x5636cdffe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 -- 172.21.15.42:0/873328611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636ce004750 msgr2=0x5636cdffe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636ce004750 0x5636cdffe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 -- 172.21.15.42:0/873328611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 msgr2=0x5636ce001fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 -- 172.21.15.42:0/873328611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636cdefdc20 con 0x5636cdffb250 2024-03-12T04:23:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f62696ea640 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f626a6ec640 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636ce004750 0x5636cdffe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f6269eeb640 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636cdffb250 0x5636ce003e60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f625c096be0 tx=0x7f625c0965e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedceb01d8215942 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f624f7fe640 1 -- 172.21.15.42:0/873328611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c09e170 con 0x5636cdffb250 2024-03-12T04:23:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f624f7fe640 1 -- 172.21.15.42:0/873328611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f625c0a29b0 con 0x5636cdffb250 2024-03-12T04:23:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.446+0000 7f624f7fe640 1 -- 172.21.15.42:0/873328611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c095f60 con 0x5636cdffb250 2024-03-12T04:23:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 -- 172.21.15.42:0/873328611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636cdffb250 msgr2=0x5636ce003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636cdffb250 0x5636ce003e60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f625c096be0 tx=0x7f625c0965e0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 -- 172.21.15.42:0/873328611 shutdown_connections 2024-03-12T04:23:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636ce004750 0x5636cdffe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce001fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 --2- 172.21.15.42:0/873328611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636cdffb250 0x5636ce003e60 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 -- 172.21.15.42:0/873328611 >> 172.21.15.42:0/873328611 conn(0x5636cdff5980 msgr2=0x5636cdffca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 -- 172.21.15.42:0/873328611 shutdown_connections 2024-03-12T04:23:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 -- 172.21.15.42:0/873328611 wait complete. 2024-03-12T04:23:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 -- start start 2024-03-12T04:23:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.447+0000 7f626ee03740 1 Processor -- start 2024-03-12T04:23:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626ee03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce00a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626ee03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce00ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626ee03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce06b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626ee03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ce00b290 con 0x5636ce004750 2024-03-12T04:23:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626ee03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ce06b580 con 0x5636cdffbb70 2024-03-12T04:23:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626ee03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ce06b6c0 con 0x5636cdffb250 2024-03-12T04:23:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626ee03740 10 client.0 _ll_register_callbacks cb 0x7f625400b1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce06b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51382/0 (socket says 172.21.15.42:51382) 2024-03-12T04:23:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 -- 172.21.15.42:0/2593212326 learned_addr learned my addr 172.21.15.42:0/2593212326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f62696ea640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce00ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f6269eeb640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce00a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 -- 172.21.15.42:0/2593212326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 msgr2=0x5636ce00a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce00a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 -- 172.21.15.42:0/2593212326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 msgr2=0x5636ce00ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce00ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 -- 172.21.15.42:0/2593212326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62641a4770 con 0x5636ce004750 2024-03-12T04:23:11.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f62696ea640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce00ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f626a6ec640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce06b040 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6260043d10 tx=0x7f6260009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bec74437212ebc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.448+0000 7f624d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f624d7fa640 1 -- 172.21.15.42:0/2593212326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260034440 con 0x5636ce004750 2024-03-12T04:23:11.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6254013480 con 0x5636ce004750 2024-03-12T04:23:11.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f624d7fa640 1 -- 172.21.15.42:0/2593212326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f626005c080 con 0x5636ce004750 2024-03-12T04:23:11.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f624d7fa640 1 -- 172.21.15.42:0/2593212326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6260043430 con 0x5636ce004750 2024-03-12T04:23:11.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 10 client.17715 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5636cdf238c0 con 0x5636ce004750 2024-03-12T04:23:11.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f624d7fa640 1 -- 172.21.15.42:0/2593212326 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6254013480 con 0x5636ce004750 2024-03-12T04:23:11.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f624d7fa640 1 -- 172.21.15.42:0/2593212326 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6260004430 con 0x5636ce004750 2024-03-12T04:23:11.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 10 client.17715 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 10 client.17715 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:23:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 20 client.17715 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce06cbc0 0x5636ce06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 4 client.17715 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636cdcee890 con 0x5636ce06cbc0 2024-03-12T04:23:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.449+0000 7f6269eeb640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce06cbc0 0x5636ce06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f6269eeb640 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce06cbc0 0x5636ce06efe0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=457b7462c23f59d7 server_cookie=abb69f81166bdb9d in_seq=0 out_seq=0 2024-03-12T04:23:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f624d7fa640 10 client.17715 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f624d7fa640 1 -- 172.21.15.42:0/2593212326 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636cdcee890 con 0x5636ce06cbc0 2024-03-12T04:23:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f624d7fa640 10 client.17715 handle_command_reply: tid=0 2024-03-12T04:23:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 client.17715 shutdown 2024-03-12T04:23:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 msgr2=0x5636ce06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce06b040 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6260043d10 tx=0x7f6260009c90 comp rx=0 tx=0).stop 2024-03-12T04:23:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 shutdown_connections 2024-03-12T04:23:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x5636ce06cbc0 0x5636ce06efe0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636ce004750 0x5636ce06b040 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636cdffbb70 0x5636ce00ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f624d7fa640 0 client.17715 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 --2- 172.21.15.42:0/2593212326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636cdffb250 0x5636ce00a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 >> 172.21.15.42:0/2593212326 conn(0x5636cdff5980 msgr2=0x5636cdff42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 shutdown_connections 2024-03-12T04:23:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 1 -- 172.21.15.42:0/2593212326 wait complete. 2024-03-12T04:23:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.450+0000 7f626ee03740 20 client.17715 trim_cache size 0 max 16384 2024-03-12T04:23:11.473 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:11.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62500a1ba0 msgr2=0x7f62500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62500a1ba0 0x7f62500a4060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6260006010 tx=0x7f62600324d0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264101390 msgr2=0x7f6264197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264101390 0x7f6264197750 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d950 tx=0x7f625c00de10 comp rx=0 tx=0).stop 2024-03-12T04:23:11.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 shutdown_connections 2024-03-12T04:23:11.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62500a1ba0 0x7f62500a4060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626419c010 0x7f626419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6264106460 0x7f6264197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 --2- 172.21.15.42:0/1680217974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6264101390 0x7f6264197750 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 >> 172.21.15.42:0/1680217974 conn(0x7f62640fcbb0 msgr2=0x7f62640fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 shutdown_connections 2024-03-12T04:23:11.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.451+0000 7f626c14f640 1 -- 172.21.15.42:0/1680217974 wait complete. 2024-03-12T04:23:11.477 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:11.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:23:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 -- 172.21.15.42:0/236257172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d00ff910 msgr2=0x7f55d00ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 --2- 172.21.15.42:0/236257172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d00ff910 0x7f55d00ffcf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c002bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 -- 172.21.15.42:0/236257172 shutdown_connections 2024-03-12T04:23:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 --2- 172.21.15.42:0/236257172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d0106520 0x7f55d010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 --2- 172.21.15.42:0/236257172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55d01002c0 0x7f55d0105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 --2- 172.21.15.42:0/236257172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d00ff910 0x7f55d00ffcf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 -- 172.21.15.42:0/236257172 >> 172.21.15.42:0/236257172 conn(0x7f55d0076640 msgr2=0x7f55d00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 -- 172.21.15.42:0/236257172 shutdown_connections 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.580+0000 7f55d5adf640 1 -- 172.21.15.42:0/236257172 wait complete. 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 Processor -- start 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 -- start start 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d00ff910 0x7f55d0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d0106520 0x7f55d0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55d019bea0 0x7f55d019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019c890 con 0x7f55d0106520 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019ca00 con 0x7f55d019bea0 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55d5adf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019cb70 con 0x7f55d00ff910 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d0106520 0x7f55d0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d0106520 0x7f55d0197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51402/0 (socket says 172.21.15.42:51402) 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.581+0000 7f55ceffd640 1 -- 172.21.15.42:0/2537070502 learned_addr learned my addr 172.21.15.42:0/2537070502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d00ff910 0x7f55d0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55cffff640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55d019bea0 0x7f55d019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ceffd640 1 -- 172.21.15.42:0/2537070502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d00ff910 msgr2=0x7f55d0197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ceffd640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d00ff910 0x7f55d0197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ceffd640 1 -- 172.21.15.42:0/2537070502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55d019bea0 msgr2=0x7f55d019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ceffd640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55d019bea0 0x7f55d019c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ceffd640 1 -- 172.21.15.42:0/2537070502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0009560 con 0x7f55d0106520 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55cffff640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55d019bea0 0x7f55d019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d00ff910 0x7f55d0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ceffd640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d0106520 0x7f55d0197b70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c970 tx=0x7f55c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ccff9640 1 -- 172.21.15.42:0/2537070502 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4004440 con 0x7f55d0106520 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ccff9640 1 -- 172.21.15.42:0/2537070502 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f55c4007d10 con 0x7f55d0106520 2024-03-12T04:23:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d01a4150 con 0x7f55d0106520 2024-03-12T04:23:11.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55ccff9640 1 -- 172.21.15.42:0/2537070502 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c400f450 con 0x7f55d0106520 2024-03-12T04:23:11.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.582+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d01a4600 con 0x7f55d0106520 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.584+0000 7f55ccff9640 1 -- 172.21.15.42:0/2537070502 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f55c4003680 con 0x7f55d0106520 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 Processor -- start 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 -- start start 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c84019360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c84026620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55ccff9640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55b80a1ba0 0x7f55b80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 0x558c84028f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c83fd4960 con 0x558c84018f80 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c83f88310 con 0x558c840198a0 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55d878f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c83f85840 con 0x558c84026b60 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c84019360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c84019360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51418/0 (socket says 172.21.15.42:51418) 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 -- 172.21.15.42:0/1173211642 learned_addr learned my addr 172.21.15.42:0/1173211642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55ceffd640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55b80a1ba0 0x7f55b80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55ccff9640 1 -- 172.21.15.42:0/2537070502 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f55c40cda60 con 0x7f55d0106520 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c84026620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 0x558c84028f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 -- 172.21.15.42:0/1173211642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 msgr2=0x558c84028f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 0x558c84028f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 -- 172.21.15.42:0/1173211642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 msgr2=0x558c84026620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c84026620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.585+0000 7f55cffff640 1 -- 172.21.15.42:0/1173211642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c83f43060 con 0x558c84018f80 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55ceffd640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55b80a1ba0 0x7f55b80a4060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55c409a620 tx=0x7f55c40035f0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55cffff640 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c84019360 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f55c80134f0 tx=0x7f55c8009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d22ac7d8a52156 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55b5ffb640 1 -- 172.21.15.42:0/1173211642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8004d20 con 0x558c84018f80 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55b5ffb640 1 -- 172.21.15.42:0/1173211642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f55c8004e80 con 0x558c84018f80 2024-03-12T04:23:11.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55b5ffb640 1 -- 172.21.15.42:0/1173211642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8005480 con 0x558c84018f80 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 -- 172.21.15.42:0/1173211642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 msgr2=0x558c84019360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c84019360 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f55c80134f0 tx=0x7f55c8009890 comp rx=0 tx=0).stop 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 -- 172.21.15.42:0/1173211642 shutdown_connections 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 0x558c84028f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c84026620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 --2- 172.21.15.42:0/1173211642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c84019360 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 -- 172.21.15.42:0/1173211642 >> 172.21.15.42:0/1173211642 conn(0x558c84013980 msgr2=0x558c84013d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 -- 172.21.15.42:0/1173211642 shutdown_connections 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.586+0000 7f55d878f740 1 -- 172.21.15.42:0/1173211642 wait complete. 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 -- start start 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 Processor -- start 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84018f80 0x558c84023b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c840198a0 0x558c840240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 0x558c840245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c83f81020 con 0x558c840198a0 2024-03-12T04:23:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c83c54430 con 0x558c84026b60 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c83c10e00 con 0x558c84018f80 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55d878f740 10 client.0 _ll_register_callbacks cb 0x558c83f28cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c840198a0 0x558c840240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c840198a0 0x558c840240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51432/0 (socket says 172.21.15.42:51432) 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55cf7fe640 1 -- 172.21.15.42:0/2577669199 learned_addr learned my addr 172.21.15.42:0/2577669199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.587+0000 7f55cffff640 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 0x558c840245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55cffff640 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84018f80 0x558c84023b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55cf7fe640 1 -- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84018f80 msgr2=0x558c84023b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84018f80 0x558c84023b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55cf7fe640 1 -- 172.21.15.42:0/2577669199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 msgr2=0x558c840245f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 0x558c840245f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55cf7fe640 1 -- 172.21.15.42:0/2577669199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c8009c50 con 0x558c840198a0 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c840198a0 0x558c840240b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c002cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56dd0fa43b1b144 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55aeffd640 1 -- 172.21.15.42:0/2577669199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0004510 con 0x558c840198a0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55aeffd640 1 -- 172.21.15.42:0/2577669199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f55c00035f0 con 0x558c840198a0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c83cbd720 con 0x558c840198a0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55aeffd640 1 -- 172.21.15.42:0/2577669199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c00037b0 con 0x558c840198a0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55d878f740 10 client.17727 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558c83f7f6a0 con 0x558c840198a0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55aeffd640 1 -- 172.21.15.42:0/2577669199 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558c83cbd720 con 0x558c840198a0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.588+0000 7f55aeffd640 1 -- 172.21.15.42:0/2577669199 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55c003c040 con 0x558c840198a0 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55d878f740 10 client.17727 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55d878f740 10 client.17727 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55d878f740 20 client.17727 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55d878f740 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c84036f40 0x558c84039360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55d878f740 4 client.17727 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c83d0c890 con 0x558c84036f40 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55cffff640 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c84036f40 0x558c84039360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55aeffd640 10 client.17727 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.589+0000 7f55cffff640 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c84036f40 0x558c84039360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60183f5b2f6f8c1e server_cookie=7e79d3a86688322f in_seq=0 out_seq=0 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55aeffd640 1 -- 172.21.15.42:0/2577669199 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c83d0c890 con 0x558c84036f40 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55aeffd640 10 client.17727 handle_command_reply: tid=0 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 client.17727 shutdown 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c840198a0 msgr2=0x558c840240b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c840198a0 0x558c840240b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c002cae0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 shutdown_connections 2024-03-12T04:23:11.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c84036f40 0x558c84039360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 0x558c840245f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55aeffd640 0 client.17727 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c840198a0 0x558c840240b0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 --2- 172.21.15.42:0/2577669199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84018f80 0x558c84023b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 >> 172.21.15.42:0/2577669199 conn(0x558c84013980 msgr2=0x558c840156d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 shutdown_connections 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.590+0000 7f55d878f740 1 -- 172.21.15.42:0/2577669199 wait complete. 2024-03-12T04:23:11.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.591+0000 7f55d878f740 20 client.17727 trim_cache size 0 max 16384 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.602+0000 7f55d878f740 1 Processor -- start 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.602+0000 7f55d878f740 1 -- start start 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55d878f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c840262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55d878f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c840198a0 0x558c84024410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55d878f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 0x558c84020df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55d878f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c83f06130 con 0x558c84018f80 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55d878f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c8400f5c0 con 0x558c84026b60 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55d878f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c84033a90 con 0x558c840198a0 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c840262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c840262a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51438/0 (socket says 172.21.15.42:51438) 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 -- 172.21.15.42:0/2863392355 learned_addr learned my addr 172.21.15.42:0/2863392355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 -- 172.21.15.42:0/2863392355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c840198a0 msgr2=0x558c84024410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 --2- 172.21.15.42:0/2863392355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c840198a0 0x558c84024410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 -- 172.21.15.42:0/2863392355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 msgr2=0x558c84020df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 --2- 172.21.15.42:0/2863392355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 0x558c84020df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 -- 172.21.15.42:0/2863392355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c83f1bc20 con 0x558c84018f80 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55cffff640 1 --2- 172.21.15.42:0/2863392355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c840262a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005fd0 tx=0x7f55c80050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422934ae35df8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55b57fa640 1 -- 172.21.15.42:0/2863392355 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c802b380 con 0x558c84018f80 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55b57fa640 1 -- 172.21.15.42:0/2863392355 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f55c8024040 con 0x558c84018f80 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.603+0000 7f55b57fa640 1 -- 172.21.15.42:0/2863392355 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8005870 con 0x558c84018f80 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 -- 172.21.15.42:0/2863392355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 msgr2=0x558c840262a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 --2- 172.21.15.42:0/2863392355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c840262a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005fd0 tx=0x7f55c80050c0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 -- 172.21.15.42:0/2863392355 shutdown_connections 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 --2- 172.21.15.42:0/2863392355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c84026b60 0x558c84020df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 --2- 172.21.15.42:0/2863392355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c840198a0 0x558c84024410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 --2- 172.21.15.42:0/2863392355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c840262a0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 -- 172.21.15.42:0/2863392355 >> 172.21.15.42:0/2863392355 conn(0x558c84013980 msgr2=0x558c8401eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 -- 172.21.15.42:0/2863392355 shutdown_connections 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 -- 172.21.15.42:0/2863392355 wait complete. 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 -- start start 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.604+0000 7f55d878f740 1 Processor -- start 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c8402cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c8402d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 0x558c8408d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c8408d970 con 0x558c84018f80 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c8408dab0 con 0x558c840198a0 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c8408dbf0 con 0x558c84026b60 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 10 client.0 _ll_register_callbacks cb 0x7f55c80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c8402cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c8402cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51446/0 (socket says 172.21.15.42:51446) 2024-03-12T04:23:11.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 -- 172.21.15.42:0/4087318719 learned_addr learned my addr 172.21.15.42:0/4087318719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c8402d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 -- 172.21.15.42:0/4087318719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 msgr2=0x558c8408d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 0x558c8408d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 -- 172.21.15.42:0/4087318719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 msgr2=0x558c8402d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c8402d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 -- 172.21.15.42:0/4087318719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c8026040 con 0x558c84018f80 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55cffff640 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c8402cbe0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009ae0 tx=0x7f55c80058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51ab979c6a67659 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55aeffd640 1 -- 172.21.15.42:0/4087318719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8038070 con 0x558c84018f80 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55aeffd640 1 -- 172.21.15.42:0/4087318719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f55c80052e0 con 0x558c84018f80 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c83c807e0 con 0x558c84018f80 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55aeffd640 1 -- 172.21.15.42:0/4087318719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c80054a0 con 0x558c84018f80 2024-03-12T04:23:11.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 10 client.17733 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.605+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558c83f418c0 con 0x558c84018f80 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55aeffd640 1 -- 172.21.15.42:0/4087318719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558c83c807e0 con 0x558c84018f80 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55aeffd640 1 -- 172.21.15.42:0/4087318719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55c803b020 con 0x558c84018f80 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55d878f740 10 client.17733 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55d878f740 10 client.17733 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55d878f740 20 client.17733 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55d878f740 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c8408f070 0x558c84091460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55d878f740 4 client.17733 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558c83c2c700 con 0x558c8408f070 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c8408f070 0x558c84091460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55cf7fe640 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c8408f070 0x558c84091460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8287094cd5048b2 server_cookie=9e492ba5b27efde4 in_seq=0 out_seq=0 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.606+0000 7f55aeffd640 10 client.17733 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55aeffd640 1 -- 172.21.15.42:0/4087318719 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558c83c2c700 con 0x558c8408f070 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55aeffd640 10 client.17733 handle_command_reply: tid=0 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 client.17733 shutdown 2024-03-12T04:23:11.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 msgr2=0x558c8402cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c8402cbe0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009ae0 tx=0x7f55c80058f0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 shutdown_connections 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x558c8408f070 0x558c84091460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c84026b60 0x558c8408d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c840198a0 0x558c8402d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 --2- 172.21.15.42:0/4087318719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c84018f80 0x558c8402cbe0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55aeffd640 0 client.17733 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 >> 172.21.15.42:0/4087318719 conn(0x558c84013980 msgr2=0x558c84014db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 shutdown_connections 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 1 -- 172.21.15.42:0/4087318719 wait complete. 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d878f740 20 client.17733 trim_cache size 0 max 16384 2024-03-12T04:23:11.615 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:11.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55b80a1ba0 msgr2=0x7f55b80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d5adf640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55b80a1ba0 0x7f55b80a4060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55c409a620 tx=0x7f55c40035f0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d0106520 msgr2=0x7f55d0197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.607+0000 7f55d5adf640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d0106520 0x7f55d0197b70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c970 tx=0x7f55c400ce30 comp rx=0 tx=0).stop 2024-03-12T04:23:11.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 shutdown_connections 2024-03-12T04:23:11.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55b80a1ba0 0x7f55b80a4060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55d019bea0 0x7f55d019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d0106520 0x7f55d0197b70 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 --2- 172.21.15.42:0/2537070502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d00ff910 0x7f55d0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 >> 172.21.15.42:0/2537070502 conn(0x7f55d0076640 msgr2=0x7f55d00fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 shutdown_connections 2024-03-12T04:23:11.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.608+0000 7f55d5adf640 1 -- 172.21.15.42:0/2537070502 wait complete. 2024-03-12T04:23:11.619 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:11 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/801607099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:23:11.620 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:11.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:23:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:11 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/801607099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:23:11.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:11 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/801607099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 -- 172.21.15.42:0/3095405733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 msgr2=0x7fd754100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 --2- 172.21.15.42:0/3095405733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd754100640 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd744009930 tx=0x7fd74402bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 -- 172.21.15.42:0/3095405733 shutdown_connections 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 --2- 172.21.15.42:0/3095405733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd754105330 0x7fd75410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 --2- 172.21.15.42:0/3095405733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd754100c10 0x7fd754104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 --2- 172.21.15.42:0/3095405733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd754100640 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 -- 172.21.15.42:0/3095405733 >> 172.21.15.42:0/3095405733 conn(0x7fd754077180 msgr2=0x7fd754077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 -- 172.21.15.42:0/3095405733 shutdown_connections 2024-03-12T04:23:11.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 -- 172.21.15.42:0/3095405733 wait complete. 2024-03-12T04:23:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.724+0000 7fd759af4640 1 Processor -- start 2024-03-12T04:23:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd759af4640 1 -- start start 2024-03-12T04:23:11.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd759af4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd7541976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd759af4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd754105330 0x7fd754197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd759af4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd75419bf90 0x7fd75419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd759af4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75419c980 con 0x7fd754100260 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd759af4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75419caf0 con 0x7fd754105330 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd759af4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75419cc60 con 0x7fd75419bf90 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd7541976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd7541976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51464/0 (socket says 172.21.15.42:51464) 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 -- 172.21.15.42:0/3396158142 learned_addr learned my addr 172.21.15.42:0/3396158142 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd752ffd640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd754105330 0x7fd754197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd753fff640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd75419bf90 0x7fd75419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 -- 172.21.15.42:0/3396158142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd75419bf90 msgr2=0x7fd75419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd75419bf90 0x7fd75419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 -- 172.21.15.42:0/3396158142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd754105330 msgr2=0x7fd754197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd754105330 0x7fd754197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 -- 172.21.15.42:0/3396158142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd744009560 con 0x7fd754100260 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd753fff640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd75419bf90 0x7fd75419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd7537fe640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd7541976d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd744009230 tx=0x7fd7440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.725+0000 7fd750ff9640 1 -- 172.21.15.42:0/3396158142 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440044e0 con 0x7fd754100260 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.726+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7541a4240 con 0x7fd754100260 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.726+0000 7fd750ff9640 1 -- 172.21.15.42:0/3396158142 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd74402de90 con 0x7fd754100260 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.726+0000 7fd750ff9640 1 -- 172.21.15.42:0/3396158142 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744005390 con 0x7fd754100260 2024-03-12T04:23:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.726+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7541a46c0 con 0x7fd754100260 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd750ff9640 1 -- 172.21.15.42:0/3396158142 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd7440054f0 con 0x7fd754100260 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 Processor -- start 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 -- start start 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1b70 0x55d3168fa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3168ac960 con 0x55d3168f1b70 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d316860310 con 0x55d3168f1250 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d31685d840 con 0x55d3168fa750 2024-03-12T04:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd750ff9640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd73c0a1ba0 0x7fd73c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd7537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1b70 0x55d3168fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd7537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1b70 0x55d3168fa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51480/0 (socket says 172.21.15.42:51480) 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd7537fe640 1 -- 172.21.15.42:0/1029778391 learned_addr learned my addr 172.21.15.42:0/1029778391 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd753fff640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd73c0a1ba0 0x7fd73c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.728+0000 7fd753fff640 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd752ffd640 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd750ff9640 1 -- 172.21.15.42:0/3396158142 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd7440ef970 con 0x7fd754100260 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd7537fe640 1 -- 172.21.15.42:0/1029778391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 msgr2=0x55d3168fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd7537fe640 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd7537fe640 1 -- 172.21.15.42:0/1029778391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 msgr2=0x55d3168f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd7537fe640 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd7537fe640 1 -- 172.21.15.42:0/1029778391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d31681b060 con 0x55d3168f1b70 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd7537fe640 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1b70 0x55d3168fa210 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd74402c4e0 tx=0x7fd74402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9342317b73eca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd753fff640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd73c0a1ba0 0x7fd73c0a4060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd74c013b30 tx=0x7fd74c00a370 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd739ffb640 1 -- 172.21.15.42:0/1029778391 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440bcdb0 con 0x55d3168f1b70 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd739ffb640 1 -- 172.21.15.42:0/1029778391 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7440bc6b0 con 0x55d3168f1b70 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.729+0000 7fd739ffb640 1 -- 172.21.15.42:0/1029778391 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440dc0c0 con 0x55d3168f1b70 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 -- 172.21.15.42:0/1029778391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1b70 msgr2=0x55d3168fa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1b70 0x55d3168fa210 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd74402c4e0 tx=0x7fd74402bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 -- 172.21.15.42:0/1029778391 shutdown_connections 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1b70 0x55d3168fa210 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/1029778391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 -- 172.21.15.42:0/1029778391 >> 172.21.15.42:0/1029778391 conn(0x55d3168eb980 msgr2=0x55d3168ebd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 -- 172.21.15.42:0/1029778391 shutdown_connections 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 -- 172.21.15.42:0/1029778391 wait complete. 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 -- start start 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 Processor -- start 2024-03-12T04:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1b70 0x55d3168f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.730+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d3168f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d316859020 con 0x55d3168fa750 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d31652c430 con 0x55d3168f1250 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3164e8e00 con 0x55d3168f1b70 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd75c7a1740 10 client.0 _ll_register_callbacks cb 0x55d316800cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d3168f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d3168f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51490/0 (socket says 172.21.15.42:51490) 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 -- 172.21.15.42:0/831947995 learned_addr learned my addr 172.21.15.42:0/831947995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd7537fe640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd752ffd640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1b70 0x55d3168f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 -- 172.21.15.42:0/831947995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1b70 msgr2=0x55d3168f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1b70 0x55d3168f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 -- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 msgr2=0x55d3168f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 -- 172.21.15.42:0/831947995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7541a46c0 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd752ffd640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1b70 0x55d3168f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd753fff640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d3168f8150 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b9e0 tx=0x7fd74c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8f619540632bab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd732ffd640 1 -- 172.21.15.42:0/831947995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c012de0 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd732ffd640 1 -- 172.21.15.42:0/831947995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd74c004b20 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3165587e0 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd732ffd640 1 -- 172.21.15.42:0/831947995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd74c004ce0 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd75c7a1740 10 client.17745 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.731+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d3168576a0 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd732ffd640 1 -- 172.21.15.42:0/831947995 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d3165587e0 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd732ffd640 1 -- 172.21.15.42:0/831947995 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd74c00f430 con 0x55d3168fa750 2024-03-12T04:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd75c7a1740 10 client.17745 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd75c7a1740 10 client.17745 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd75c7a1740 20 client.17745 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d31690a8d0 0x55d31690ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd75c7a1740 4 client.17745 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d316595720 con 0x55d31690a8d0 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.732+0000 7fd7537fe640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d31690a8d0 0x55d31690ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.733+0000 7fd7537fe640 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d31690a8d0 0x55d31690ccf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2f71bf20ebea4e6 server_cookie=e2c85d5095aee587 in_seq=0 out_seq=0 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.733+0000 7fd732ffd640 10 client.17745 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.733+0000 7fd732ffd640 1 -- 172.21.15.42:0/831947995 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d316595720 con 0x55d31690a8d0 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.733+0000 7fd732ffd640 10 client.17745 handle_command_reply: tid=0 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.733+0000 7fd75c7a1740 1 client.17745 shutdown 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 msgr2=0x55d3168f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d3168f8150 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b9e0 tx=0x7fd74c004a40 comp rx=0 tx=0).stop 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 shutdown_connections 2024-03-12T04:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d31690a8d0 0x55d31690ccf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d3168f8150 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1b70 0x55d3168f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd732ffd640 0 client.17745 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/831947995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1250 0x55d3168f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 >> 172.21.15.42:0/831947995 conn(0x55d3168eb980 msgr2=0x55d3168ed6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 shutdown_connections 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 1 -- 172.21.15.42:0/831947995 wait complete. 2024-03-12T04:23:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.734+0000 7fd75c7a1740 20 client.17745 trim_cache size 0 max 16384 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 Processor -- start 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 -- start start 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1250 0x55d3168f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168f4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3167de130 con 0x55d3168f1250 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d316907390 con 0x55d3168f1b70 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.745+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3168f0eb0 con 0x55d3168fa750 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1250 0x55d3168f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1250 0x55d3168f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51498/0 (socket says 172.21.15.42:51498) 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 -- 172.21.15.42:0/596688995 learned_addr learned my addr 172.21.15.42:0/596688995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd752ffd640 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 -- 172.21.15.42:0/596688995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 msgr2=0x55d3168f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd753fff640 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 -- 172.21.15.42:0/596688995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 msgr2=0x55d3168f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 -- 172.21.15.42:0/596688995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3167f3c20 con 0x55d3168f1250 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd752ffd640 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd753fff640 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7537fe640 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1250 0x55d3168f9e00 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd7440bc5f0 tx=0x7fd744009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf5d7ac026f12b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7397fa640 1 -- 172.21.15.42:0/596688995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440efbb0 con 0x55d3168f1250 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7397fa640 1 -- 172.21.15.42:0/596688995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7440baab0 con 0x55d3168f1250 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.746+0000 7fd7397fa640 1 -- 172.21.15.42:0/596688995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440ed340 con 0x55d3168f1250 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- 172.21.15.42:0/596688995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1250 msgr2=0x55d3168f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1250 0x55d3168f9e00 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd7440bc5f0 tx=0x7fd744009260 comp rx=0 tx=0).stop 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- 172.21.15.42:0/596688995 shutdown_connections 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168fa750 0x55d3168f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/596688995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168f1250 0x55d3168f9e00 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- 172.21.15.42:0/596688995 >> 172.21.15.42:0/596688995 conn(0x55d3168eb980 msgr2=0x55d3168f29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- 172.21.15.42:0/596688995 shutdown_connections 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- 172.21.15.42:0/596688995 wait complete. 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- start start 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 Processor -- start 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1250 0x55d316901870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d316900290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d316900960 con 0x55d3168fa750 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d316900ad0 con 0x55d3168f1b70 2024-03-12T04:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.747+0000 7fd75c7a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d316960e60 con 0x55d3168f1250 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd75c7a1740 10 client.0 _ll_register_callbacks cb 0x7fd7440bc480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d316900290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d316900290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51506/0 (socket says 172.21.15.42:51506) 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 -- 172.21.15.42:0/3733896532 learned_addr learned my addr 172.21.15.42:0/3733896532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd7537fe640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1250 0x55d316901870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd752ffd640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 -- 172.21.15.42:0/3733896532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1250 msgr2=0x55d316901870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1250 0x55d316901870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 -- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 msgr2=0x55d3168ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 -- 172.21.15.42:0/3733896532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d31681b060 con 0x55d3168fa750 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd752ffd640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd753fff640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d316900290 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00f740 tx=0x7fd74c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c716efac494e462 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd732ffd640 1 -- 172.21.15.42:0/3733896532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c012df0 con 0x55d3168fa750 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd732ffd640 1 -- 172.21.15.42:0/3733896532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd74c030430 con 0x55d3168fa750 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3165e4890 con 0x55d3168fa750 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd732ffd640 1 -- 172.21.15.42:0/3733896532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd74c0305f0 con 0x55d3168fa750 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd75c7a1740 10 client.17751 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.748+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d3168198c0 con 0x55d3168fa750 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd732ffd640 1 -- 172.21.15.42:0/3733896532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d3165e4890 con 0x55d3168fa750 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd732ffd640 1 -- 172.21.15.42:0/3733896532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd74c0397b0 con 0x55d3168fa750 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd75c7a1740 10 client.17751 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd75c7a1740 10 client.17751 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd75c7a1740 20 client.17751 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d3169625c0 0x55d3169649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd75c7a1740 4 client.17751 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d316504700 con 0x55d3169625c0 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd7537fe640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d3169625c0 0x55d3169649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd7537fe640 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d3169625c0 0x55d3169649e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed75a888594456ab server_cookie=fe6298937c976c35 in_seq=0 out_seq=0 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.749+0000 7fd732ffd640 10 client.17751 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd732ffd640 1 -- 172.21.15.42:0/3733896532 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d316504700 con 0x55d3169625c0 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd732ffd640 10 client.17751 handle_command_reply: tid=0 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 client.17751 shutdown 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 msgr2=0x55d316900290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d316900290 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00f740 tx=0x7fd74c00fc60 comp rx=0 tx=0).stop 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 shutdown_connections 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55d3169625c0 0x55d3169649e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3168fa750 0x55d316900290 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3168f1b70 0x55d3168ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd732ffd640 0 client.17751 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 --2- 172.21.15.42:0/3733896532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3168f1250 0x55d316901870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 >> 172.21.15.42:0/3733896532 conn(0x55d3168eb980 msgr2=0x55d3168ea2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 shutdown_connections 2024-03-12T04:23:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 1 -- 172.21.15.42:0/3733896532 wait complete. 2024-03-12T04:23:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.750+0000 7fd75c7a1740 20 client.17751 trim_cache size 0 max 16384 2024-03-12T04:23:11.759 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd73c0a1ba0 msgr2=0x7fd73c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd73c0a1ba0 0x7fd73c0a4060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd74c013b30 tx=0x7fd74c00a370 comp rx=0 tx=0).stop 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 msgr2=0x7fd7541976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd7541976d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd744009230 tx=0x7fd7440042f0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 shutdown_connections 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd73c0a1ba0 0x7fd73c0a4060 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd75419bf90 0x7fd75419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd754105330 0x7fd754197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 --2- 172.21.15.42:0/3396158142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd754100260 0x7fd7541976d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 >> 172.21.15.42:0/3396158142 conn(0x7fd754077180 msgr2=0x7fd7540686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 shutdown_connections 2024-03-12T04:23:11.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.751+0000 7fd759af4640 1 -- 172.21.15.42:0/3396158142 wait complete. 2024-03-12T04:23:11.763 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:11.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:23:11.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 -- 172.21.15.42:0/4209699931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb624100cc0 msgr2=0x7fb624104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 --2- 172.21.15.42:0/4209699931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb624100cc0 0x7fb624104d00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb614009930 tx=0x7fb61402bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 -- 172.21.15.42:0/4209699931 shutdown_connections 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 --2- 172.21.15.42:0/4209699931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6241053e0 0x7fb62410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 --2- 172.21.15.42:0/4209699931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb624100cc0 0x7fb624104d00 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 --2- 172.21.15.42:0/4209699931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624100310 0x7fb6241006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 -- 172.21.15.42:0/4209699931 >> 172.21.15.42:0/4209699931 conn(0x7fb6240771e0 msgr2=0x7fb6240775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 -- 172.21.15.42:0/4209699931 shutdown_connections 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.867+0000 7fb62c04d640 1 -- 172.21.15.42:0/4209699931 wait complete. 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 Processor -- start 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 -- start start 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624100310 0x7fb624197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6241053e0 0x7fb624197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb62419bfa0 0x7fb62419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62419c990 con 0x7fb62419bfa0 2024-03-12T04:23:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62419cb00 con 0x7fb624100310 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62c04d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62419cc70 con 0x7fb6241053e0 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62a5ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb62419bfa0 0x7fb62419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62a5ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb62419bfa0 0x7fb62419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51538/0 (socket says 172.21.15.42:51538) 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb62a5ea640 1 -- 172.21.15.42:0/898564908 learned_addr learned my addr 172.21.15.42:0/898564908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb6295e8640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6241053e0 0x7fb624197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.868+0000 7fb629de9640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624100310 0x7fb624197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62a5ea640 1 -- 172.21.15.42:0/898564908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6241053e0 msgr2=0x7fb624197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6241053e0 0x7fb624197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62a5ea640 1 -- 172.21.15.42:0/898564908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624100310 msgr2=0x7fb624197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624100310 0x7fb624197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62a5ea640 1 -- 172.21.15.42:0/898564908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614009560 con 0x7fb62419bfa0 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb629de9640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624100310 0x7fb624197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb6295e8640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6241053e0 0x7fb624197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb62419bfa0 0x7fb62419c450 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00eaa0 tx=0x7fb61c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb61affd640 1 -- 172.21.15.42:0/898564908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00cdb0 con 0x7fb62419bfa0 2024-03-12T04:23:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb61affd640 1 -- 172.21.15.42:0/898564908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb61c005020 con 0x7fb62419bfa0 2024-03-12T04:23:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6241a4250 con 0x7fb62419bfa0 2024-03-12T04:23:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb61affd640 1 -- 172.21.15.42:0/898564908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0116b0 con 0x7fb62419bfa0 2024-03-12T04:23:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.869+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6241a4680 con 0x7fb62419bfa0 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.871+0000 7fb61affd640 1 -- 172.21.15.42:0/898564908 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fb61c005190 con 0x7fb62419bfa0 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.871+0000 7fb62ec1a740 1 Processor -- start 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.871+0000 7fb61affd640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb6100a1ba0 0x7fb6100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb61affd640 1 -- 172.21.15.42:0/898564908 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fb61c0cda50 con 0x7fb62419bfa0 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb6295e8640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb6100a1ba0 0x7fb6100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62ec1a740 1 -- start start 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6666f80 0x5563f6667360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6674620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f6676f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f6622960 con 0x5563f6674b60 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f65d6310 con 0x5563f6666f80 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f65d3840 con 0x5563f66678a0 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f6676f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f6676f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51544/0 (socket says 172.21.15.42:51544) 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 -- 172.21.15.42:0/500427953 learned_addr learned my addr 172.21.15.42:0/500427953 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb6295e8640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb6100a1ba0 0x7fb6100a4060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb614009900 tx=0x7fb614005fd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb629de9640 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6666f80 0x5563f6667360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb6295e8640 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6674620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 -- 172.21.15.42:0/500427953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 msgr2=0x5563f6674620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6674620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 -- 172.21.15.42:0/500427953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6666f80 msgr2=0x5563f6667360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6666f80 0x5563f6667360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 -- 172.21.15.42:0/500427953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563f6591060 con 0x5563f6674b60 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.872+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f6676f50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00efd0 tx=0x7fb61c099fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5506af9500003eb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb60ffff640 1 -- 172.21.15.42:0/500427953 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c09e7d0 con 0x5563f6674b60 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb60ffff640 1 -- 172.21.15.42:0/500427953 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb61c0a2e20 con 0x5563f6674b60 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb60ffff640 1 -- 172.21.15.42:0/500427953 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c096cf0 con 0x5563f6674b60 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 -- 172.21.15.42:0/500427953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 msgr2=0x5563f6676f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f6676f50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00efd0 tx=0x7fb61c099fe0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 -- 172.21.15.42:0/500427953 shutdown_connections 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f6676f50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6674620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/500427953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6666f80 0x5563f6667360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 -- 172.21.15.42:0/500427953 >> 172.21.15.42:0/500427953 conn(0x5563f6661980 msgr2=0x5563f6661d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 -- 172.21.15.42:0/500427953 shutdown_connections 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.873+0000 7fb62ec1a740 1 -- 172.21.15.42:0/500427953 wait complete. 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 -- start start 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 Processor -- start 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f6671bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f66678a0 0x5563f6672100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f6672640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f65cf020 con 0x5563f66678a0 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f62a2430 con 0x5563f6674b60 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f625ee00 con 0x5563f6666f80 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62ec1a740 10 client.0 _ll_register_callbacks cb 0x5563f6576cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb6295e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f66678a0 0x5563f6672100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb6295e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f66678a0 0x5563f6672100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51560/0 (socket says 172.21.15.42:51560) 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb6295e8640 1 -- 172.21.15.42:0/323744544 learned_addr learned my addr 172.21.15.42:0/323744544 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.874+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f6672640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb629de9640 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f6671bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb6295e8640 1 -- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 msgr2=0x5563f6671bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb6295e8640 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f6671bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb6295e8640 1 -- 172.21.15.42:0/323744544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 msgr2=0x5563f6672640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb6295e8640 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f6672640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb6295e8640 1 -- 172.21.15.42:0/323744544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6241a4680 con 0x5563f66678a0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb6295e8640 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f66678a0 0x5563f6672100 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb614009930 tx=0x7fb614009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5f5a8d5ee9f787 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb60d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb60d7fa640 1 -- 172.21.15.42:0/323744544 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140044d0 con 0x5563f66678a0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb60d7fa640 1 -- 172.21.15.42:0/323744544 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fb614004a70 con 0x5563f66678a0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563f62ce7e0 con 0x5563f66678a0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb60d7fa640 1 -- 172.21.15.42:0/323744544 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb614003b40 con 0x5563f66678a0 2024-03-12T04:23:11.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb62ec1a740 10 client.17763 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5563f65cd6a0 con 0x5563f66678a0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.875+0000 7fb60d7fa640 1 -- 172.21.15.42:0/323744544 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5563f62ce7e0 con 0x5563f66678a0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb60d7fa640 1 -- 172.21.15.42:0/323744544 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb61403c040 con 0x5563f66678a0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb62ec1a740 10 client.17763 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb62ec1a740 10 client.17763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb62ec1a740 20 client.17763 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f6684fe0 0x5563f6687400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb62ec1a740 4 client.17763 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563f630b720 con 0x5563f6684fe0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb629de9640 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f6684fe0 0x5563f6687400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb60d7fa640 10 client.17763 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.876+0000 7fb629de9640 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f6684fe0 0x5563f6687400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ab03a74e4dc2871 server_cookie=dadcb83936161e1c in_seq=0 out_seq=0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb60d7fa640 1 -- 172.21.15.42:0/323744544 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563f630b720 con 0x5563f6684fe0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb60d7fa640 10 client.17763 handle_command_reply: tid=0 2024-03-12T04:23:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 client.17763 shutdown 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f66678a0 msgr2=0x5563f6672100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f66678a0 0x5563f6672100 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb614009930 tx=0x7fb614009260 comp rx=0 tx=0).stop 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 shutdown_connections 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f6684fe0 0x5563f6687400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f6672640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f66678a0 0x5563f6672100 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/323744544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f6671bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb60d7fa640 0 client.17763 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 >> 172.21.15.42:0/323744544 conn(0x5563f6661980 msgr2=0x5563f66636d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 shutdown_connections 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 1 -- 172.21.15.42:0/323744544 wait complete. 2024-03-12T04:23:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.877+0000 7fb62ec1a740 20 client.17763 trim_cache size 0 max 16384 2024-03-12T04:23:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 Processor -- start 2024-03-12T04:23:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 -- start start 2024-03-12T04:23:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6666f80 0x5563f66742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6672460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f666ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f6554130 con 0x5563f6666f80 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f665d5f0 con 0x5563f6674b60 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f668c690 con 0x5563f66678a0 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb629de9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6666f80 0x5563f66742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb629de9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6666f80 0x5563f66742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51564/0 (socket says 172.21.15.42:51564) 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb629de9640 1 -- 172.21.15.42:0/2076173170 learned_addr learned my addr 172.21.15.42:0/2076173170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f666ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb6295e8640 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6672460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb629de9640 1 -- 172.21.15.42:0/2076173170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 msgr2=0x5563f6672460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.889+0000 7fb629de9640 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6672460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb629de9640 1 -- 172.21.15.42:0/2076173170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 msgr2=0x5563f666ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb629de9640 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f666ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb629de9640 1 -- 172.21.15.42:0/2076173170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563f6569c20 con 0x5563f6666f80 2024-03-12T04:23:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f666ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb6295e8640 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6672460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb629de9640 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6666f80 0x5563f66742f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb62000dc70 tx=0x7fb62000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221c6b05e6fb6660 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb60f7fe640 1 -- 172.21.15.42:0/2076173170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620016070 con 0x5563f6666f80 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb60f7fe640 1 -- 172.21.15.42:0/2076173170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb620017040 con 0x5563f6666f80 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb60f7fe640 1 -- 172.21.15.42:0/2076173170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200208b0 con 0x5563f6666f80 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 -- 172.21.15.42:0/2076173170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6666f80 msgr2=0x5563f66742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6666f80 0x5563f66742f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb62000dc70 tx=0x7fb62000d6b0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 -- 172.21.15.42:0/2076173170 shutdown_connections 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f6674b60 0x5563f666ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f66678a0 0x5563f6672460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/2076173170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6666f80 0x5563f66742f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 -- 172.21.15.42:0/2076173170 >> 172.21.15.42:0/2076173170 conn(0x5563f6661980 msgr2=0x5563f666cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 -- 172.21.15.42:0/2076173170 shutdown_connections 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.890+0000 7fb62ec1a740 1 -- 172.21.15.42:0/2076173170 wait complete. 2024-03-12T04:23:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 -- start start 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 Processor -- start 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f667a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f66678a0 0x5563f667ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f66db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f667b650 con 0x5563f6674b60 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f66dba30 con 0x5563f66678a0 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f66dbb70 con 0x5563f6666f80 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62ec1a740 10 client.0 _ll_register_callbacks cb 0x7fb61c00cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62a5ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f66db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62a5ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f66db4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51578/0 (socket says 172.21.15.42:51578) 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb62a5ea640 1 -- 172.21.15.42:0/3209371947 learned_addr learned my addr 172.21.15.42:0/3209371947 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb629de9640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f667a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.891+0000 7fb6295e8640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f66678a0 0x5563f667ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62a5ea640 1 -- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 msgr2=0x5563f667a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f667a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62a5ea640 1 -- 172.21.15.42:0/3209371947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f66678a0 msgr2=0x5563f667ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f66678a0 0x5563f667ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62a5ea640 1 -- 172.21.15.42:0/3209371947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62000d8c0 con 0x5563f6674b60 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb6295e8640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f66678a0 0x5563f667ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62a5ea640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f66db4f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00efd0 tx=0x7fb61c09a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfa8681dd3fc7fb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb60d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb60d7fa640 1 -- 172.21.15.42:0/3209371947 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c005630 con 0x5563f6674b60 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb60d7fa640 1 -- 172.21.15.42:0/3209371947 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fb61c09e7d0 con 0x5563f6674b60 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61c0a2e20 con 0x5563f6674b60 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb60d7fa640 1 -- 172.21.15.42:0/3209371947 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb61c0ba0c0 con 0x5563f6674b60 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 10 client.17769 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5563f658f8c0 con 0x5563f6674b60 2024-03-12T04:23:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb60d7fa640 1 -- 172.21.15.42:0/3209371947 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb61c0a2e20 con 0x5563f6674b60 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb60d7fa640 1 -- 172.21.15.42:0/3209371947 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb61c018030 con 0x5563f6674b60 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 10 client.17769 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 20 client.17769 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f66dcee0 0x5563f66df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 4 client.17769 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.892+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5563f635a890 con 0x5563f66dcee0 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb629de9640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f66dcee0 0x5563f66df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb629de9640 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f66dcee0 0x5563f66df300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8f63119324dace9 server_cookie=5aef4442b919683e in_seq=0 out_seq=0 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb60d7fa640 10 client.17769 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb60d7fa640 1 -- 172.21.15.42:0/3209371947 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5563f635a890 con 0x5563f66dcee0 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb60d7fa640 10 client.17769 handle_command_reply: tid=0 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb62ec1a740 1 client.17769 shutdown 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 msgr2=0x5563f66db4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.893+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f66db4f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00efd0 tx=0x7fb61c09a110 comp rx=0 tx=0).stop 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 shutdown_connections 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x5563f66dcee0 0x5563f66df300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563f6674b60 0x5563f66db4f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563f66678a0 0x5563f667ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 --2- 172.21.15.42:0/3209371947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563f6666f80 0x5563f667a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb60d7fa640 0 client.17769 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 >> 172.21.15.42:0/3209371947 conn(0x5563f6661980 msgr2=0x5563f66602f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 shutdown_connections 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 1 -- 172.21.15.42:0/3209371947 wait complete. 2024-03-12T04:23:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62ec1a740 20 client.17769 trim_cache size 0 max 16384 2024-03-12T04:23:11.903 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb6100a1ba0 msgr2=0x7fb6100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb6100a1ba0 0x7fb6100a4060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb614009900 tx=0x7fb614005fd0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb62419bfa0 msgr2=0x7fb62419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb62419bfa0 0x7fb62419c450 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00eaa0 tx=0x7fb61c00ef60 comp rx=0 tx=0).stop 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 shutdown_connections 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb6100a1ba0 0x7fb6100a4060 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb62419bfa0 0x7fb62419c450 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6241053e0 0x7fb624197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 --2- 172.21.15.42:0/898564908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624100310 0x7fb624197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 >> 172.21.15.42:0/898564908 conn(0x7fb6240771e0 msgr2=0x7fb6240fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 shutdown_connections 2024-03-12T04:23:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:11.894+0000 7fb62c04d640 1 -- 172.21.15.42:0/898564908 wait complete. 2024-03-12T04:23:11.906 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:11.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:23:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 -- 172.21.15.42:0/2732981837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48100c50 msgr2=0x7f6b48104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/2732981837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48100c50 0x7f6b48104c90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009930 tx=0x7f6b3802bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 -- 172.21.15.42:0/2732981837 shutdown_connections 2024-03-12T04:23:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/2732981837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b48105370 0x7f6b4810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/2732981837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48100c50 0x7f6b48104c90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/2732981837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b481002a0 0x7f6b48100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 -- 172.21.15.42:0/2732981837 >> 172.21.15.42:0/2732981837 conn(0x7f6b48076400 msgr2=0x7f6b48076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 -- 172.21.15.42:0/2732981837 shutdown_connections 2024-03-12T04:23:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.009+0000 7f6b4fedb640 1 -- 172.21.15.42:0/2732981837 wait complete. 2024-03-12T04:23:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 Processor -- start 2024-03-12T04:23:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 -- start start 2024-03-12T04:23:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b481002a0 0x7f6b48101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48105370 0x7f6b48102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b481027b0 0x7f6b481a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b481a5ef0 con 0x7f6b48105370 2024-03-12T04:23:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b481a6060 con 0x7f6b481002a0 2024-03-12T04:23:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4fedb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b481a6220 con 0x7f6b481027b0 2024-03-12T04:23:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48105370 0x7f6b48102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48105370 0x7f6b48102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51592/0 (socket says 172.21.15.42:51592) 2024-03-12T04:23:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 -- 172.21.15.42:0/403512689 learned_addr learned my addr 172.21.15.42:0/403512689 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4e478640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b481027b0 0x7f6b481a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 -- 172.21.15.42:0/403512689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b481027b0 msgr2=0x7f6b481a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b481027b0 0x7f6b481a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 -- 172.21.15.42:0/403512689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b481002a0 msgr2=0x7f6b48101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b481002a0 0x7f6b48101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4d476640 1 -- 172.21.15.42:0/403512689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38009560 con 0x7f6b48105370 2024-03-12T04:23:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.010+0000 7f6b4e478640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b481027b0 0x7f6b481a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:12.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.011+0000 7f6b4d476640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48105370 0x7f6b48102270 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6b38000c00 tx=0x7f6b38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:12.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.011+0000 7f6b3effd640 1 -- 172.21.15.42:0/403512689 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3802cb70 con 0x7f6b48105370 2024-03-12T04:23:12.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.011+0000 7f6b3effd640 1 -- 172.21.15.42:0/403512689 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b3802de90 con 0x7f6b48105370 2024-03-12T04:23:12.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.011+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b481a64a0 con 0x7f6b48105370 2024-03-12T04:23:12.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.011+0000 7f6b3effd640 1 -- 172.21.15.42:0/403512689 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380053c0 con 0x7f6b48105370 2024-03-12T04:23:12.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.011+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b481a6a00 con 0x7f6b48105370 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b3effd640 1 -- 172.21.15.42:0/403512689 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f6b380035f0 con 0x7f6b48105370 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 Processor -- start 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 -- start start 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28c3250 0x55e8c28c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28cc750 0x55e8c28ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c287e960 con 0x55e8c28c3b70 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c2832310 con 0x55e8c28cc750 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.013+0000 7f6b52b83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c282f840 con 0x55e8c28c3250 2024-03-12T04:23:12.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51594/0 (socket says 172.21.15.42:51594) 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 -- 172.21.15.42:0/3878113376 learned_addr learned my addr 172.21.15.42:0/3878113376 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b3effd640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6b340a1c50 0x7f6b340a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4d476640 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28cc750 0x55e8c28ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4e478640 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28c3250 0x55e8c28c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4e478640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6b340a1c50 0x7f6b340a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 -- 172.21.15.42:0/3878113376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28c3250 msgr2=0x55e8c28c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28c3250 0x55e8c28c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 -- 172.21.15.42:0/3878113376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28cc750 msgr2=0x55e8c28ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b3effd640 1 -- 172.21.15.42:0/403512689 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6b380efac0 con 0x7f6b48105370 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28cc750 0x55e8c28ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 -- 172.21.15.42:0/3878113376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c27ed060 con 0x55e8c28c3b70 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4d476640 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28cc750 0x55e8c28ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28cc210 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009820 tx=0x7f6b44009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488463443511cea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b4e478640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6b340a1c50 0x7f6b340a4110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009270 tx=0x7f6b40005bd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b33fff640 1 -- 172.21.15.42:0/3878113376 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400bad0 con 0x55e8c28c3b70 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b33fff640 1 -- 172.21.15.42:0/3878113376 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b4400bc30 con 0x55e8c28c3b70 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.014+0000 7f6b33fff640 1 -- 172.21.15.42:0/3878113376 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440105e0 con 0x55e8c28c3b70 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 -- 172.21.15.42:0/3878113376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 msgr2=0x55e8c28cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28cc210 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009820 tx=0x7f6b44009ce0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 -- 172.21.15.42:0/3878113376 shutdown_connections 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28cc750 0x55e8c28ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28cc210 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 --2- 172.21.15.42:0/3878113376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28c3250 0x55e8c28c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 -- 172.21.15.42:0/3878113376 >> 172.21.15.42:0/3878113376 conn(0x55e8c28bd980 msgr2=0x55e8c28bdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 -- 172.21.15.42:0/3878113376 shutdown_connections 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 -- 172.21.15.42:0/3878113376 wait complete. 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.015+0000 7f6b52b83740 1 -- start start 2024-03-12T04:23:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 1 Processor -- start 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c282b020 con 0x55e8c28c3b70 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c24fe430 con 0x55e8c28c3250 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c24bae00 con 0x55e8c28cc750 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b52b83740 10 client.0 _ll_register_callbacks cb 0x55e8c27d2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51602/0 (socket says 172.21.15.42:51602) 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 -- 172.21.15.42:0/4158613815 learned_addr learned my addr 172.21.15.42:0/4158613815 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4e478640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 -- 172.21.15.42:0/4158613815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 msgr2=0x55e8c28ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 -- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 msgr2=0x55e8c28c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4d476640 1 -- 172.21.15.42:0/4158613815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b44009450 con 0x55e8c28c3b70 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.016+0000 7f6b4e478640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b4d476640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9c80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6b38004c20 tx=0x7f6b3808ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0613b86bda80cc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b317fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b317fa640 1 -- 172.21.15.42:0/4158613815 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380ed690 con 0x55e8c28c3b70 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b317fa640 1 -- 172.21.15.42:0/4158613815 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6b380bcf00 con 0x55e8c28c3b70 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8c252a7e0 con 0x55e8c28c3b70 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b317fa640 1 -- 172.21.15.42:0/4158613815 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b380bc800 con 0x55e8c28c3b70 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 10 client.17781 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8c28296a0 con 0x55e8c28c3b70 2024-03-12T04:23:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b317fa640 1 -- 172.21.15.42:0/4158613815 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e8c252a7e0 con 0x55e8c28c3b70 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b317fa640 1 -- 172.21.15.42:0/4158613815 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6b3803c070 con 0x55e8c28c3b70 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 10 client.17781 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 20 client.17781 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c28dc9a0 0x55e8c28ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 4 client.17781 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.017+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8c2567720 con 0x55e8c28dc9a0 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.018+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c28dc9a0 0x55e8c28ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.018+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c28dc9a0 0x55e8c28ded90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c087f6ad3e2139 server_cookie=97164cf095d3b58a in_seq=0 out_seq=0 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.018+0000 7f6b317fa640 10 client.17781 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b317fa640 1 -- 172.21.15.42:0/4158613815 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8c2567720 con 0x55e8c28dc9a0 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b317fa640 10 client.17781 handle_command_reply: tid=0 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 client.17781 shutdown 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 msgr2=0x55e8c28c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9c80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6b38004c20 tx=0x7f6b3808ea00 comp rx=0 tx=0).stop 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 shutdown_connections 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c28dc9a0 0x55e8c28ded90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9c80 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b317fa640 0 client.17781 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 --2- 172.21.15.42:0/4158613815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 >> 172.21.15.42:0/4158613815 conn(0x55e8c28bd980 msgr2=0x55e8c28bf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 shutdown_connections 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 1 -- 172.21.15.42:0/4158613815 wait complete. 2024-03-12T04:23:12.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.019+0000 7f6b52b83740 20 client.17781 trim_cache size 0 max 16384 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 Processor -- start 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 -- start start 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c27b0130 con 0x55e8c28c3b70 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c28c2eb0 con 0x55e8c28c3250 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b52b83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c28be980 con 0x55e8c28cc750 2024-03-12T04:23:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b4d476640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b4d476640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51608/0 (socket says 172.21.15.42:51608) 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.031+0000 7f6b4d476640 1 -- 172.21.15.42:0/1171173197 learned_addr learned my addr 172.21.15.42:0/1171173197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b4d476640 1 -- 172.21.15.42:0/1171173197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 msgr2=0x55e8c28c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b4d476640 1 --2- 172.21.15.42:0/1171173197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b4d476640 1 -- 172.21.15.42:0/1171173197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 msgr2=0x55e8c28cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b4d476640 1 --2- 172.21.15.42:0/1171173197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b4d476640 1 -- 172.21.15.42:0/1171173197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c27c5c20 con 0x55e8c28c3b70 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b4d476640 1 --2- 172.21.15.42:0/1171173197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9fe0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6b380bcb90 tx=0x7f6b38004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d96edefc97eeb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b337fe640 1 -- 172.21.15.42:0/1171173197 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380c4d10 con 0x55e8c28c3b70 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b337fe640 1 -- 172.21.15.42:0/1171173197 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b380912e0 con 0x55e8c28c3b70 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b337fe640 1 -- 172.21.15.42:0/1171173197 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380ed660 con 0x55e8c28c3b70 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b52b83740 1 -- 172.21.15.42:0/1171173197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 msgr2=0x55e8c28c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b52b83740 1 --2- 172.21.15.42:0/1171173197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9fe0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6b380bcb90 tx=0x7f6b38004dc0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b52b83740 1 -- 172.21.15.42:0/1171173197 shutdown_connections 2024-03-12T04:23:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b52b83740 1 --2- 172.21.15.42:0/1171173197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.032+0000 7f6b52b83740 1 --2- 172.21.15.42:0/1171173197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3b70 0x55e8c28c9fe0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 --2- 172.21.15.42:0/1171173197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3250 0x55e8c28cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 -- 172.21.15.42:0/1171173197 >> 172.21.15.42:0/1171173197 conn(0x55e8c28bd980 msgr2=0x55e8c28b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 -- 172.21.15.42:0/1171173197 shutdown_connections 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 -- 172.21.15.42:0/1171173197 wait complete. 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 -- start start 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 Processor -- start 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3250 0x55e8c28d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3b70 0x55e8c28d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c2932fb0 con 0x55e8c28c3250 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c2933120 con 0x55e8c28c3b70 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c29332e0 con 0x55e8c28cc750 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b52b83740 10 client.0 _ll_register_callbacks cb 0x7f6b44009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b4dc77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3250 0x55e8c28d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b4dc77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3250 0x55e8c28d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51618/0 (socket says 172.21.15.42:51618) 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.033+0000 7f6b4dc77640 1 -- 172.21.15.42:0/2811740933 learned_addr learned my addr 172.21.15.42:0/2811740933 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4d476640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3b70 0x55e8c28d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4e478640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4dc77640 1 -- 172.21.15.42:0/2811740933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 msgr2=0x55e8c28d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4dc77640 1 -- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3b70 msgr2=0x55e8c28d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3b70 0x55e8c28d27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4dc77640 1 -- 172.21.15.42:0/2811740933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8c28296a0 con 0x55e8c28c3250 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4d476640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3b70 0x55e8c28d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4e478640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b4dc77640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3250 0x55e8c28d2290 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f6b44011e20 tx=0x7f6b4400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241afb81be80a044 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b317fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b317fa640 1 -- 172.21.15.42:0/2811740933 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44011c50 con 0x55e8c28c3250 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b317fa640 1 -- 172.21.15.42:0/2811740933 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6b4401f040 con 0x55e8c28c3250 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8c25b6890 con 0x55e8c28c3250 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b317fa640 1 -- 172.21.15.42:0/2811740933 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b44022de0 con 0x55e8c28c3250 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b52b83740 10 client.17787 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8c27eb8c0 con 0x55e8c28c3250 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b317fa640 1 -- 172.21.15.42:0/2811740933 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e8c25b6890 con 0x55e8c28c3250 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.034+0000 7f6b317fa640 1 -- 172.21.15.42:0/2811740933 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6b4402b440 con 0x55e8c28c3250 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 10 client.17787 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 20 client.17787 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c2934aa0 0x55e8c2936ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 4 client.17787 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8c24d6700 con 0x55e8c2934aa0 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b4d476640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c2934aa0 0x55e8c2936ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b317fa640 10 client.17787 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b4d476640 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c2934aa0 0x55e8c2936ec0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f12ebb40a1695bd server_cookie=c230b26d8d64f3d5 in_seq=0 out_seq=0 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b317fa640 1 -- 172.21.15.42:0/2811740933 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8c24d6700 con 0x55e8c2934aa0 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b317fa640 10 client.17787 handle_command_reply: tid=0 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 1 client.17787 shutdown 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3250 msgr2=0x55e8c28d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.035+0000 7f6b52b83740 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3250 0x55e8c28d2290 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f6b44011e20 tx=0x7f6b4400bf40 comp rx=0 tx=0).stop 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 shutdown_connections 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e8c2934aa0 0x55e8c2936ec0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8c28cc750 0x55e8c28d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8c28c3b70 0x55e8c28d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 --2- 172.21.15.42:0/2811740933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8c28c3250 0x55e8c28d2290 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b317fa640 0 client.17787 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 >> 172.21.15.42:0/2811740933 conn(0x55e8c28bd980 msgr2=0x55e8c28c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:12.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 shutdown_connections 2024-03-12T04:23:12.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 1 -- 172.21.15.42:0/2811740933 wait complete. 2024-03-12T04:23:12.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b52b83740 20 client.17787 trim_cache size 0 max 16384 2024-03-12T04:23:12.049 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6b340a1c50 msgr2=0x7f6b340a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6b340a1c50 0x7f6b340a4110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009270 tx=0x7f6b40005bd0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48105370 msgr2=0x7f6b48102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48105370 0x7f6b48102270 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6b38000c00 tx=0x7f6b38004270 comp rx=0 tx=0).stop 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 shutdown_connections 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f6b340a1c50 0x7f6b340a4110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b481027b0 0x7f6b481a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b48105370 0x7f6b48102270 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 --2- 172.21.15.42:0/403512689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b481002a0 0x7f6b48101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 >> 172.21.15.42:0/403512689 conn(0x7f6b48076400 msgr2=0x7f6b48069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 shutdown_connections 2024-03-12T04:23:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:12.036+0000 7f6b4fedb640 1 -- 172.21.15.42:0/403512689 wait complete. 2024-03-12T04:23:12.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:12 smithi119 ceph-mon[36119]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:23:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:12 smithi136 ceph-mon[35571]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:23:12.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:12 smithi042 ceph-mon[32260]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:23:13.052 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:23:13.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:23:13.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:23:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.169+0000 7f801a09d640 1 -- 172.21.15.42:0/3339118274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014100cc0 msgr2=0x7f8014104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.169+0000 7f801a09d640 1 --2- 172.21.15.42:0/3339118274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014100cc0 0x7f8014104d00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009930 tx=0x7f7ffc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.169+0000 7f801a09d640 1 -- 172.21.15.42:0/3339118274 shutdown_connections 2024-03-12T04:23:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.169+0000 7f801a09d640 1 --2- 172.21.15.42:0/3339118274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141053e0 0x7f801410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.169+0000 7f801a09d640 1 --2- 172.21.15.42:0/3339118274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014100cc0 0x7f8014104d00 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.169+0000 7f801a09d640 1 --2- 172.21.15.42:0/3339118274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8014100310 0x7f80141006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.169+0000 7f801a09d640 1 -- 172.21.15.42:0/3339118274 >> 172.21.15.42:0/3339118274 conn(0x7f80140771e0 msgr2=0x7f80140775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.170+0000 7f801a09d640 1 -- 172.21.15.42:0/3339118274 shutdown_connections 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.170+0000 7f801a09d640 1 -- 172.21.15.42:0/3339118274 wait complete. 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.170+0000 7f801a09d640 1 Processor -- start 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.170+0000 7f801a09d640 1 -- start start 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.170+0000 7f801a09d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8014100310 0x7f8014198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f801a09d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141053e0 0x7f8014198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f801a09d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8014199020 0x7f80141a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f801a09d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80141a3ca0 con 0x7f80141053e0 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f801a09d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80141a3e10 con 0x7f8014199020 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f801a09d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80141a3fd0 con 0x7f8014100310 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141053e0 0x7f8014198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141053e0 0x7f8014198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51638/0 (socket says 172.21.15.42:51638) 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f80137fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8014100310 0x7f8014198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 -- 172.21.15.42:0/4260298298 learned_addr learned my addr 172.21.15.42:0/4260298298 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8013fff640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8014199020 0x7f80141a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 -- 172.21.15.42:0/4260298298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8014100310 msgr2=0x7f8014198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8014100310 0x7f8014198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 -- 172.21.15.42:0/4260298298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8014199020 msgr2=0x7f80141a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8014199020 0x7f80141a35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 -- 172.21.15.42:0/4260298298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc009560 con 0x7f80141053e0 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8013fff640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8014199020 0x7f80141a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8012ffd640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141053e0 0x7f8014198a90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc000c00 tx=0x7f7ffc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8010ff9640 1 -- 172.21.15.42:0/4260298298 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc02cb70 con 0x7f80141053e0 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8010ff9640 1 -- 172.21.15.42:0/4260298298 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7ffc02de90 con 0x7f80141053e0 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80141a4250 con 0x7f80141053e0 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.171+0000 7f8010ff9640 1 -- 172.21.15.42:0/4260298298 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc005380 con 0x7f80141053e0 2024-03-12T04:23:13.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.172+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80141a4760 con 0x7f80141053e0 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.174+0000 7f8010ff9640 1 -- 172.21.15.42:0/4260298298 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7ffc0035f0 con 0x7f80141053e0 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.174+0000 7f801cc7c740 1 Processor -- start 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f801cc7c740 1 -- start start 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0d30 0x561aba9f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561aba9fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f801cc7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba95ef90 con 0x561aba9f0d30 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f801cc7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba95c4c0 con 0x561aba9f98b0 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f801cc7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba957ca0 con 0x561aba9f0380 2024-03-12T04:23:13.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f8010ff9640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f80000a1ba0 0x7f80000a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0d30 0x561aba9f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0d30 0x561aba9f92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51646/0 (socket says 172.21.15.42:51646) 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 -- 172.21.15.42:0/553738477 learned_addr learned my addr 172.21.15.42:0/553738477 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f8013fff640 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f8010ff9640 1 -- 172.21.15.42:0/4260298298 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f7ffc0efab0 con 0x7f80141053e0 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f8012ffd640 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561aba9fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f8013fff640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f80000a1ba0 0x7f80000a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 -- 172.21.15.42:0/553738477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 msgr2=0x561aba9f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 -- 172.21.15.42:0/553738477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 msgr2=0x561aba9fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561aba9fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 -- 172.21.15.42:0/553738477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aba91a0b0 con 0x561aba9f0d30 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.175+0000 7f80137fe640 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0d30 0x561aba9f92e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8004007f90 tx=0x7f800400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2354344a50a906 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f8009ffb640 1 -- 172.21.15.42:0/553738477 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004004d20 con 0x561aba9f0d30 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f8013fff640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f80000a1ba0 0x7f80000a4060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f800c005fd0 tx=0x7f800c005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f8009ffb640 1 -- 172.21.15.42:0/553738477 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8004004e80 con 0x561aba9f0d30 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f8009ffb640 1 -- 172.21.15.42:0/553738477 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040055e0 con 0x561aba9f0d30 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 -- 172.21.15.42:0/553738477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0d30 msgr2=0x561aba9f92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0d30 0x561aba9f92e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8004007f90 tx=0x7f800400f850 comp rx=0 tx=0).stop 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 -- 172.21.15.42:0/553738477 shutdown_connections 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561aba9fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0d30 0x561aba9f92e0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 --2- 172.21.15.42:0/553738477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 -- 172.21.15.42:0/553738477 >> 172.21.15.42:0/553738477 conn(0x561aba9ea840 msgr2=0x561aba9eac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 -- 172.21.15.42:0/553738477 shutdown_connections 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.176+0000 7f801cc7c740 1 -- 172.21.15.42:0/553738477 wait complete. 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 -- start start 2024-03-12T04:23:13.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 Processor -- start 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0d30 0x561aba9f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba8fcf40 con 0x561aba9f98b0 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba62b4b0 con 0x561aba9f0d30 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba8dd670 con 0x561aba9f0380 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f801cc7c740 10 client.0 _ll_register_callbacks cb 0x561aba94af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f8013fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f8013fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51648/0 (socket says 172.21.15.42:51648) 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f8013fff640 1 -- 172.21.15.42:0/4276267361 learned_addr learned my addr 172.21.15.42:0/4276267361 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.177+0000 7f8012ffd640 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0d30 0x561aba9f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f80137fe640 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f8013fff640 1 -- 172.21.15.42:0/4276267361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 msgr2=0x561aba9f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f8013fff640 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f8013fff640 1 -- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0d30 msgr2=0x561aba9f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f8013fff640 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0d30 0x561aba9f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f8013fff640 1 -- 172.21.15.42:0/4276267361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8004007bc0 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f8013fff640 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9f72b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f800c00efd0 tx=0x7f800c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20f733f407612d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f7ff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/4276267361 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c010ee0 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/4276267361 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f800c023070 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561aba931f40 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/4276267361 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f800c01ce30 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f801cc7c740 10 client.17799 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561aba956320 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.178+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/4276267361 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561aba931f40 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/4276267361 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f800c031850 con 0x561aba9f98b0 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f801cc7c740 10 client.17799 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:13.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f801cc7c740 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f801cc7c740 20 client.17799 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f801cc7c740 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa0c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f801cc7c740 4 client.17799 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aba94e310 con 0x561abaa09e90 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f80137fe640 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa0c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f80137fe640 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa0c2b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a806c6d809d15a4 server_cookie=25c0debe1bbc46f2 in_seq=0 out_seq=0 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.179+0000 7f7ff2ffd640 10 client.17799 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/4276267361 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561aba94e310 con 0x561abaa09e90 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f7ff2ffd640 10 client.17799 handle_command_reply: tid=0 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 client.17799 shutdown 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 msgr2=0x561aba9f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9f72b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f800c00efd0 tx=0x7f800c0046c0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 shutdown_connections 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa0c2b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f7ff2ffd640 0 client.17799 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9f72b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0d30 0x561aba9f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 --2- 172.21.15.42:0/4276267361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0380 0x561aba9f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 >> 172.21.15.42:0/4276267361 conn(0x561aba9ea840 msgr2=0x561aba9ec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 shutdown_connections 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 1 -- 172.21.15.42:0/4276267361 wait complete. 2024-03-12T04:23:13.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.180+0000 7f801cc7c740 20 client.17799 trim_cache size 0 max 16384 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 Processor -- start 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 -- start start 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0380 0x561aba9f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa00910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561abaa00e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba9fc160 con 0x561aba9f0380 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba9e6110 con 0x561aba9f98b0 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f801cc7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aba9f4d80 con 0x561aba9f0d30 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0380 0x561aba9f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0380 0x561aba9f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51662/0 (socket says 172.21.15.42:51662) 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 -- 172.21.15.42:0/1375677890 learned_addr learned my addr 172.21.15.42:0/1375677890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f8013fff640 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561abaa00e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f8012ffd640 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa00910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 -- 172.21.15.42:0/1375677890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 msgr2=0x561abaa00910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa00910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 -- 172.21.15.42:0/1375677890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 msgr2=0x561abaa00e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561abaa00e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 -- 172.21.15.42:0/1375677890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aba8f2c00 con 0x561aba9f0380 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f8012ffd640 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa00910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.196+0000 7f80137fe640 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0380 0x561aba9f70d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f800400fd50 tx=0x7f8004005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29de3416cf0f9d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f80097fa640 1 -- 172.21.15.42:0/1375677890 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004007a60 con 0x561aba9f0380 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f80097fa640 1 -- 172.21.15.42:0/1375677890 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f80040057a0 con 0x561aba9f0380 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f80097fa640 1 -- 172.21.15.42:0/1375677890 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004010e50 con 0x561aba9f0380 2024-03-12T04:23:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 -- 172.21.15.42:0/1375677890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0380 msgr2=0x561aba9f70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0380 0x561aba9f70d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f800400fd50 tx=0x7f8004005280 comp rx=0 tx=0).stop 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 -- 172.21.15.42:0/1375677890 shutdown_connections 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f98b0 0x561abaa00e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa00910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1375677890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f0380 0x561aba9f70d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 -- 172.21.15.42:0/1375677890 >> 172.21.15.42:0/1375677890 conn(0x561aba9ea840 msgr2=0x561aba9e9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 -- 172.21.15.42:0/1375677890 shutdown_connections 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.197+0000 7f801cc7c740 1 -- 172.21.15.42:0/1375677890 wait complete. 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 -- start start 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 Processor -- start 2024-03-12T04:23:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0380 0x561abaa5ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa5f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561abaa00460 con 0x561aba9f98b0 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561abaa005d0 con 0x561aba9f0380 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561abaa00790 con 0x561aba9f0d30 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f801cc7c740 10 client.0 _ll_register_callbacks cb 0x7f80040079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51664/0 (socket says 172.21.15.42:51664) 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 -- 172.21.15.42:0/1571010931 learned_addr learned my addr 172.21.15.42:0/1571010931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8012ffd640 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa5f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f80137fe640 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0380 0x561abaa5ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 -- 172.21.15.42:0/1571010931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 msgr2=0x561abaa5f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa5f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 -- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0380 msgr2=0x561abaa5ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0380 0x561abaa5ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.198+0000 7f8013fff640 1 -- 172.21.15.42:0/1571010931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800401c040 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f8013fff640 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9ffea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f800c031be0 tx=0x7f800c031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c8104e562defbf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f7ff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/1571010931 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c010ee0 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/1571010931 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f800c023040 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561aba97d900 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/1571010931 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f800c01cbf0 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 10 client.17805 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561aba918910 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/1571010931 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561aba97d900 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/1571010931 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f800c004d00 con 0x561aba9f98b0 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 10 client.17805 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 20 client.17805 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa68cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 4 client.17805 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:23:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.199+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561aba657860 con 0x561abaa09e90 2024-03-12T04:23:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.200+0000 7f80137fe640 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa68cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.200+0000 7f80137fe640 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa68cd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39fd7af779690cb server_cookie=8d34f7ab6c0eb676 in_seq=0 out_seq=0 2024-03-12T04:23:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.200+0000 7f7ff2ffd640 10 client.17805 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f7ff2ffd640 1 -- 172.21.15.42:0/1571010931 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561aba657860 con 0x561abaa09e90 2024-03-12T04:23:13.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f7ff2ffd640 10 client.17805 handle_command_reply: tid=0 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 client.17805 shutdown 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 msgr2=0x561aba9ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9ffea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f800c031be0 tx=0x7f800c031a30 comp rx=0 tx=0).stop 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 shutdown_connections 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x561abaa09e90 0x561abaa68cd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561aba9f98b0 0x561aba9ffea0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561aba9f0d30 0x561abaa5f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 --2- 172.21.15.42:0/1571010931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aba9f0380 0x561abaa5ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 >> 172.21.15.42:0/1571010931 conn(0x561aba9ea840 msgr2=0x561aba9f5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f7ff2ffd640 0 client.17805 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 shutdown_connections 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.201+0000 7f801cc7c740 1 -- 172.21.15.42:0/1571010931 wait complete. 2024-03-12T04:23:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801cc7c740 20 client.17805 trim_cache size 0 max 16384 2024-03-12T04:23:13.210 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:23:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:23:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "cd08503b-19a3-49ae-8685-184c09e2db8d", 2024-03-12T04:23:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:23:13.210 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f80000a1ba0 msgr2=0x7f80000a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f80000a1ba0 0x7f80000a4060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f800c005fd0 tx=0x7f800c005850 comp rx=0 tx=0).stop 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141053e0 msgr2=0x7f8014198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141053e0 0x7f8014198a90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc000c00 tx=0x7f7ffc004270 comp rx=0 tx=0).stop 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 shutdown_connections 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f80000a1ba0 0x7f80000a4060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8014199020 0x7f80141a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141053e0 0x7f8014198a90 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 --2- 172.21.15.42:0/4260298298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8014100310 0x7f8014198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 >> 172.21.15.42:0/4260298298 conn(0x7f80140771e0 msgr2=0x7f80140fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 shutdown_connections 2024-03-12T04:23:13.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.202+0000 7f801a09d640 1 -- 172.21.15.42:0/4260298298 wait complete. 2024-03-12T04:23:13.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:23:13.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:23:13.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.318+0000 7f8b980d4640 1 -- 172.21.15.42:0/3195049335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b90106110 msgr2=0x7f8b901064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.318+0000 7f8b980d4640 1 --2- 172.21.15.42:0/3195049335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b90106110 0x7f8b901064f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009900 tx=0x7f8b8c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.318+0000 7f8b980d4640 1 -- 172.21.15.42:0/3195049335 shutdown_connections 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.318+0000 7f8b980d4640 1 --2- 172.21.15.42:0/3195049335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90100c90 0x7f8b9010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.318+0000 7f8b980d4640 1 --2- 172.21.15.42:0/3195049335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b901002d0 0x7f8b90100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.318+0000 7f8b980d4640 1 --2- 172.21.15.42:0/3195049335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b90106110 0x7f8b901064f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 -- 172.21.15.42:0/3195049335 >> 172.21.15.42:0/3195049335 conn(0x7f8b90076e20 msgr2=0x7f8b90077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 -- 172.21.15.42:0/3195049335 shutdown_connections 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 -- 172.21.15.42:0/3195049335 wait complete. 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 Processor -- start 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 -- start start 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b901002d0 0x7f8b901976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90100c90 0x7f8b90197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90106110 0x7f8b9019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9019c560 con 0x7f8b901002d0 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9019c6d0 con 0x7f8b90100c90 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b980d4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9019c890 con 0x7f8b90106110 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b95e70640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b901002d0 0x7f8b901976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b95e70640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b901002d0 0x7f8b901976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51686/0 (socket says 172.21.15.42:51686) 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.319+0000 7f8b95e70640 1 -- 172.21.15.42:0/4155135378 learned_addr learned my addr 172.21.15.42:0/4155135378 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b9566f640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90100c90 0x7f8b90197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b95e70640 1 -- 172.21.15.42:0/4155135378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90106110 msgr2=0x7f8b9019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b96671640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90106110 0x7f8b9019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b95e70640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90106110 0x7f8b9019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b95e70640 1 -- 172.21.15.42:0/4155135378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90100c90 msgr2=0x7f8b90197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b95e70640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90100c90 0x7f8b90197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b95e70640 1 -- 172.21.15.42:0/4155135378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009560 con 0x7f8b901002d0 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b96671640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90106110 0x7f8b9019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b95e70640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b901002d0 0x7f8b901976a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006010 tx=0x7f8b8c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b86ffd640 1 -- 172.21.15.42:0/4155135378 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c004440 con 0x7f8b901002d0 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b86ffd640 1 -- 172.21.15.42:0/4155135378 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c02de90 con 0x7f8b901002d0 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9019cb10 con 0x7f8b901002d0 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b86ffd640 1 -- 172.21.15.42:0/4155135378 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0052e0 con 0x7f8b901002d0 2024-03-12T04:23:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.320+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901a43f0 con 0x7f8b901002d0 2024-03-12T04:23:13.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.322+0000 7f8b86ffd640 1 -- 172.21.15.42:0/4155135378 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8b8c003cc0 con 0x7f8b901002d0 2024-03-12T04:23:13.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 Processor -- start 2024-03-12T04:23:13.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 -- start start 2024-03-12T04:23:13.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 0x562bfaae03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b86ffd640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8b7c0a1ba0 0x7f8b7c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaae0900 0x562bfaaed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaaedbc0 0x562bfaaeffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfaa4f280 con 0x562bfaae0900 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfaa4c7b0 con 0x562bfaaedbc0 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfaa47f90 con 0x562bfaadffe0 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaae0900 0x562bfaaed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaae0900 0x562bfaaed680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51700/0 (socket says 172.21.15.42:51700) 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b86ffd640 1 -- 172.21.15.42:0/4155135378 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f8b8c0ef7d0 con 0x7f8b901002d0 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 -- 172.21.15.42:0/248970334 learned_addr learned my addr 172.21.15.42:0/248970334 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b95e70640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8b7c0a1ba0 0x7f8b7c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b96671640 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 0x562bfaae03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 -- 172.21.15.42:0/248970334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 msgr2=0x562bfaae03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 0x562bfaae03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b96671640 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaaedbc0 0x562bfaaeffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 -- 172.21.15.42:0/248970334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaaedbc0 msgr2=0x562bfaaeffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaaedbc0 0x562bfaaeffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b9566f640 1 -- 172.21.15.42:0/248970334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bfaa06570 con 0x562bfaae0900 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.323+0000 7f8b96671640 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaaedbc0 0x562bfaaeffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.324+0000 7f8b95e70640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8b7c0a1ba0 0x7f8b7c0a4060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c0dc040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.324+0000 7f8b9566f640 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaae0900 0x562bfaaed680 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000abe0 tx=0x7f8b8000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3e2c7a8ebf4103 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.324+0000 7f8b7bfff640 1 -- 172.21.15.42:0/248970334 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80018640 con 0x562bfaae0900 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.324+0000 7f8b7bfff640 1 -- 172.21.15.42:0/248970334 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b80018c50 con 0x562bfaae0900 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.324+0000 7f8b7bfff640 1 -- 172.21.15.42:0/248970334 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80010b20 con 0x562bfaae0900 2024-03-12T04:23:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.324+0000 7f8b9ad89740 1 -- 172.21.15.42:0/248970334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaae0900 msgr2=0x562bfaaed680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.324+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaae0900 0x562bfaaed680 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000abe0 tx=0x7f8b8000f790 comp rx=0 tx=0).stop 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- 172.21.15.42:0/248970334 shutdown_connections 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaaedbc0 0x562bfaaeffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaae0900 0x562bfaaed680 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/248970334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 0x562bfaae03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- 172.21.15.42:0/248970334 >> 172.21.15.42:0/248970334 conn(0x562bfaada990 msgr2=0x562bfaadad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- 172.21.15.42:0/248970334 shutdown_connections 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- 172.21.15.42:0/248970334 wait complete. 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- start start 2024-03-12T04:23:13.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 Processor -- start 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaadffe0 0x562bfaaed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaae0900 0x562bfaaea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaeb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfa71b430 con 0x562bfaaedbc0 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfa6d7e00 con 0x562bfaadffe0 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.325+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfa9ccfd0 con 0x562bfaae0900 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b9ad89740 10 client.0 _ll_register_callbacks cb 0x562bfaa22fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaeb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaeb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51712/0 (socket says 172.21.15.42:51712) 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 -- 172.21.15.42:0/3284693776 learned_addr learned my addr 172.21.15.42:0/3284693776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaadffe0 0x562bfaaed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b9566f640 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaae0900 0x562bfaaea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 -- 172.21.15.42:0/3284693776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaae0900 msgr2=0x562bfaaea9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaae0900 0x562bfaaea9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 -- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaadffe0 msgr2=0x562bfaaed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaadffe0 0x562bfaaed640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 -- 172.21.15.42:0/3284693776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8000a840 con 0x562bfaaedbc0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b96671640 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaeb110 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8b880049d0 tx=0x7f8b88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc59990b8b4fdb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b797fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b797fa640 1 -- 172.21.15.42:0/3284693776 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88020440 con 0x562bfaaedbc0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b797fa640 1 -- 172.21.15.42:0/3284693776 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f8b880205a0 con 0x562bfaaedbc0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bfa7477e0 con 0x562bfaaedbc0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b797fa640 1 -- 172.21.15.42:0/3284693776 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b88020760 con 0x562bfaaedbc0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b9ad89740 10 client.17817 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.326+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562bfaa46610 con 0x562bfaaedbc0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b797fa640 1 -- 172.21.15.42:0/3284693776 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x562bfa7477e0 con 0x562bfaaedbc0 2024-03-12T04:23:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b797fa640 1 -- 172.21.15.42:0/3284693776 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8b880103d0 con 0x562bfaaedbc0 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9ad89740 10 client.17817 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9ad89740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9ad89740 20 client.17817 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfaafdd50 0x562bfab00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9ad89740 4 client.17817 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bfa784720 con 0x562bfaafdd50 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9566f640 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfaafdd50 0x562bfab00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b797fa640 10 client.17817 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.327+0000 7f8b9566f640 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfaafdd50 0x562bfab00170 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2144831bad2703 server_cookie=c1addc2573ebe8d9 in_seq=0 out_seq=0 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b797fa640 1 -- 172.21.15.42:0/3284693776 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562bfa784720 con 0x562bfaafdd50 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b797fa640 10 client.17817 handle_command_reply: tid=0 2024-03-12T04:23:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 client.17817 shutdown 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 msgr2=0x562bfaaeb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaeb110 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8b880049d0 tx=0x7f8b88004a40 comp rx=0 tx=0).stop 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 shutdown_connections 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfaafdd50 0x562bfab00170 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaeb110 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaae0900 0x562bfaaea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b797fa640 0 client.17817 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3284693776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaadffe0 0x562bfaaed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 >> 172.21.15.42:0/3284693776 conn(0x562bfaada990 msgr2=0x562bfaadc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 shutdown_connections 2024-03-12T04:23:13.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.328+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3284693776 wait complete. 2024-03-12T04:23:13.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.329+0000 7f8b9ad89740 20 client.17817 trim_cache size 0 max 16384 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 Processor -- start 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 -- start start 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 0x562bfaaed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 0x562bfaaed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaf4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfa7477e0 con 0x562bfaaedbc0 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfaad60f0 con 0x562bfaae0900 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfaadbb30 con 0x562bfaadffe0 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaf4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaf4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51718/0 (socket says 172.21.15.42:51718) 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 -- 172.21.15.42:0/4040829190 learned_addr learned my addr 172.21.15.42:0/4040829190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 -- 172.21.15.42:0/4040829190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 msgr2=0x562bfaaed100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 --2- 172.21.15.42:0/4040829190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 0x562bfaaed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 -- 172.21.15.42:0/4040829190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 msgr2=0x562bfaaed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 --2- 172.21.15.42:0/4040829190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 0x562bfaaed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 -- 172.21.15.42:0/4040829190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bfa9e2ac0 con 0x562bfaaedbc0 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b96671640 1 --2- 172.21.15.42:0/4040829190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaf4a80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009f30 tx=0x7f8b88012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92167fa3bdd4748 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.344+0000 7f8b7b7fe640 1 -- 172.21.15.42:0/4040829190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88005b70 con 0x562bfaaedbc0 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b7b7fe640 1 -- 172.21.15.42:0/4040829190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b88010d80 con 0x562bfaaedbc0 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b7b7fe640 1 -- 172.21.15.42:0/4040829190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88005070 con 0x562bfaaedbc0 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 -- 172.21.15.42:0/4040829190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 msgr2=0x562bfaaf4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/4040829190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaf4a80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009f30 tx=0x7f8b88012f40 comp rx=0 tx=0).stop 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 -- 172.21.15.42:0/4040829190 shutdown_connections 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/4040829190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaaedbc0 0x562bfaaf4a80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/4040829190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 0x562bfaaed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/4040829190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaadffe0 0x562bfaaed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 -- 172.21.15.42:0/4040829190 >> 172.21.15.42:0/4040829190 conn(0x562bfaada990 msgr2=0x562bfaae5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 -- 172.21.15.42:0/4040829190 shutdown_connections 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 -- 172.21.15.42:0/4040829190 wait complete. 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.345+0000 7f8b9ad89740 1 -- start start 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 1 Processor -- start 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaadffe0 0x562bfaaf3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 0x562bfaaf3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaaedbc0 0x562bfaaf40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfab54280 con 0x562bfaadffe0 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfab543f0 con 0x562bfaae0900 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfab545b0 con 0x562bfaaedbc0 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9ad89740 10 client.0 _ll_register_callbacks cb 0x7f8b8000a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaadffe0 0x562bfaaf3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaadffe0 0x562bfaaf3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51730/0 (socket says 172.21.15.42:51730) 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 -- 172.21.15.42:0/3818718079 learned_addr learned my addr 172.21.15.42:0/3818718079 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaaedbc0 0x562bfaaf40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b9566f640 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 0x562bfaaf3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 -- 172.21.15.42:0/3818718079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaaedbc0 msgr2=0x562bfaaf40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaaedbc0 0x562bfaaf40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 -- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 msgr2=0x562bfaaf3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 0x562bfaaf3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.346+0000 7f8b96671640 1 -- 172.21.15.42:0/3818718079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bfaa46610 con 0x562bfaadffe0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b96671640 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaadffe0 0x562bfaaf3630 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009600 tx=0x7f8b88009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac14415c9b4ff7f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b797fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b797fa640 1 -- 172.21.15.42:0/3818718079 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88010bd0 con 0x562bfaadffe0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b797fa640 1 -- 172.21.15.42:0/3818718079 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f8b88020440 con 0x562bfaadffe0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bfaaea800 con 0x562bfaadffe0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b797fa640 1 -- 172.21.15.42:0/3818718079 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b88020600 con 0x562bfaadffe0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 10 client.17823 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562bfaa086a0 con 0x562bfaadffe0 2024-03-12T04:23:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b797fa640 1 -- 172.21.15.42:0/3818718079 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x562bfaaea800 con 0x562bfaadffe0 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b797fa640 1 -- 172.21.15.42:0/3818718079 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8b880296f0 con 0x562bfaadffe0 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 10 client.17823 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 20 client.17823 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfab55ab0 0x562bfab57ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 4 client.17823 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f8b80018c50 con 0x562bfab55ab0 2024-03-12T04:23:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.347+0000 7f8b9566f640 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfab55ab0 0x562bfab57ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9566f640 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfab55ab0 0x562bfab57ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e9277ba29b500e server_cookie=2cbf9eaefd981cc9 in_seq=0 out_seq=0 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b797fa640 10 client.17823 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b797fa640 1 -- 172.21.15.42:0/3818718079 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f8b80018c50 con 0x562bfab55ab0 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b797fa640 10 client.17823 handle_command_reply: tid=0 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 client.17823 shutdown 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaadffe0 msgr2=0x562bfaaf3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaadffe0 0x562bfaaf3630 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009600 tx=0x7f8b88009a80 comp rx=0 tx=0).stop 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 shutdown_connections 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x562bfab55ab0 0x562bfab57ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562bfaaedbc0 0x562bfaaf40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bfaae0900 0x562bfaaf3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b797fa640 0 client.17823 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 --2- 172.21.15.42:0/3818718079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562bfaadffe0 0x562bfaaf3630 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 >> 172.21.15.42:0/3818718079 conn(0x562bfaada990 msgr2=0x562bfaae92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 shutdown_connections 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 1 -- 172.21.15.42:0/3818718079 wait complete. 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.348+0000 7f8b9ad89740 20 client.17823 trim_cache size 0 max 16384 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stdout: "cd08503b-19a3-49ae-8685-184c09e2db8d": { 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "cd08503b-19a3-49ae-8685-184c09e2db8d", 2024-03-12T04:23:13.358 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:23:13.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:23:13.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:23:13.359 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:23:13.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8b7c0a1ba0 msgr2=0x7f8b7c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8b7c0a1ba0 0x7f8b7c0a4060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c0dc040 comp rx=0 tx=0).stop 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b901002d0 msgr2=0x7f8b901976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b901002d0 0x7f8b901976a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006010 tx=0x7f8b8c004040 comp rx=0 tx=0).stop 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 shutdown_connections 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8b7c0a1ba0 0x7f8b7c0a4060 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90106110 0x7f8b9019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90100c90 0x7f8b90197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 --2- 172.21.15.42:0/4155135378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b901002d0 0x7f8b901976a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 >> 172.21.15.42:0/4155135378 conn(0x7f8b90076e20 msgr2=0x7f8b900ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 shutdown_connections 2024-03-12T04:23:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:13.349+0000 7f8b980d4640 1 -- 172.21.15.42:0/4155135378 wait complete. 2024-03-12T04:23:13.361 INFO:tasks.cephfs.filesystem:scrub status for tag:cd08503b-19a3-49ae-8685-184c09e2db8d - {'path': '/', 'tag': 'cd08503b-19a3-49ae-8685-184c09e2db8d', 'options': 'recursive,force'} 2024-03-12T04:23:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:14 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:23:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:14 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:14 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:23:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:14 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:14.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:14 smithi119 ceph-mon[36119]: pgmap v247: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:23:14.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:14 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:23:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:14 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:23:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:14 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:14 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:23:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:14 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:14.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:14 smithi136 ceph-mon[35571]: pgmap v247: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:23:14.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:14 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:23:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:14 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:23:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:14 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:14 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:23:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:14 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:14 smithi042 ceph-mon[32260]: pgmap v247: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:23:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:14 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:23:16.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:16 smithi119 ceph-mon[36119]: osdmap e94: 12 total, 12 up, 12 in 2024-03-12T04:23:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:16 smithi136 ceph-mon[35571]: osdmap e94: 12 total, 12 up, 12 in 2024-03-12T04:23:16.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:16 smithi042 ceph-mon[32260]: osdmap e94: 12 total, 12 up, 12 in 2024-03-12T04:23:17.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:17 smithi119 ceph-mon[36119]: pgmap v249: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:23:17.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:17 smithi136 ceph-mon[35571]: pgmap v249: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:23:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:17 smithi042 ceph-mon[32260]: pgmap v249: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:23:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:19 smithi119 ceph-mon[36119]: pgmap v250: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-12T04:23:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:19 smithi136 ceph-mon[35571]: pgmap v250: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-12T04:23:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:19 smithi042 ceph-mon[32260]: pgmap v250: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-12T04:23:21.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:21 smithi119 ceph-mon[36119]: pgmap v251: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 206 B/s wr, 10 op/s 2024-03-12T04:23:21.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:21 smithi136 ceph-mon[35571]: pgmap v251: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 206 B/s wr, 10 op/s 2024-03-12T04:23:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:21 smithi042 ceph-mon[32260]: pgmap v251: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 206 B/s wr, 10 op/s 2024-03-12T04:23:23.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:23 smithi119 ceph-mon[36119]: pgmap v252: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:23:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:23 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:23:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:23 smithi136 ceph-mon[35571]: pgmap v252: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:23:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:23 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:23:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:23 smithi042 ceph-mon[32260]: pgmap v252: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:23:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:23 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:23:24.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:24 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:23:24.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:24 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:23:24.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:24 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:23:24.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:24 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:23:24.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:24 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:23:24.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:24 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:23:24.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:24 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:23:24.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:24 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:23:24.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:24 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:23:25.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:25 smithi119 ceph-mon[36119]: pgmap v253: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:23:25.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:25 smithi136 ceph-mon[35571]: pgmap v253: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:23:25.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:25 smithi042 ceph-mon[32260]: pgmap v253: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:23:26.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:26 smithi042 ceph-mon[32260]: pgmap v254: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 100 B/s wr, 10 op/s 2024-03-12T04:23:26.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:26 smithi119 ceph-mon[36119]: pgmap v254: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 100 B/s wr, 10 op/s 2024-03-12T04:23:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:26 smithi136 ceph-mon[35571]: pgmap v254: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 100 B/s wr, 10 op/s 2024-03-12T04:23:28.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:28 smithi119 ceph-mon[36119]: pgmap v255: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:23:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:28 smithi136 ceph-mon[35571]: pgmap v255: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:23:28.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:28 smithi042 ceph-mon[32260]: pgmap v255: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:23:30.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:30 smithi119 ceph-mon[36119]: pgmap v256: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:30.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:30 smithi136 ceph-mon[35571]: pgmap v256: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:30.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:30 smithi042 ceph-mon[32260]: pgmap v256: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:32.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:32 smithi119 ceph-mon[36119]: pgmap v257: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:32.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:32 smithi136 ceph-mon[35571]: pgmap v257: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:32.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:32 smithi042 ceph-mon[32260]: pgmap v257: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:34.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:34 smithi119 ceph-mon[36119]: pgmap v258: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:34 smithi136 ceph-mon[35571]: pgmap v258: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:34.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:34 smithi042 ceph-mon[32260]: pgmap v258: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:36 smithi119 ceph-mon[36119]: pgmap v259: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:36 smithi136 ceph-mon[35571]: pgmap v259: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:36.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:36 smithi042 ceph-mon[32260]: pgmap v259: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:38.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:38 smithi119 ceph-mon[36119]: pgmap v260: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:38.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:38 smithi136 ceph-mon[35571]: pgmap v260: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:38.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:38 smithi042 ceph-mon[32260]: pgmap v260: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:40.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:40 smithi119 ceph-mon[36119]: pgmap v261: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:40.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:40 smithi136 ceph-mon[35571]: pgmap v261: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:40.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:40 smithi042 ceph-mon[32260]: pgmap v261: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:42.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:42 smithi042 ceph-mon[32260]: pgmap v262: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:42.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:42 smithi119 ceph-mon[36119]: pgmap v262: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:42 smithi136 ceph-mon[35571]: pgmap v262: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:23:43.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:23:43.363 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:23:43.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 -- 172.21.15.42:0/1238257228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff42406a170 msgr2=0x7ff42410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/1238257228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff42406a170 0x7ff42410e000 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009cd0 tx=0x7ff41c02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 -- 172.21.15.42:0/1238257228 shutdown_connections 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/1238257228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff42406a170 0x7ff42410e000 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/1238257228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4240697b0 0x7ff424069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/1238257228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff424106130 0x7ff424106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 -- 172.21.15.42:0/1238257228 >> 172.21.15.42:0/1238257228 conn(0x7ff424076e20 msgr2=0x7ff424077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 -- 172.21.15.42:0/1238257228 shutdown_connections 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 -- 172.21.15.42:0/1238257228 wait complete. 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 Processor -- start 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.477+0000 7ff42a0f2640 1 -- start start 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff42a0f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4240697b0 0x7ff424197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff42a0f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff42406a170 0x7ff424197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff42a0f2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff424106130 0x7ff42419be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff42a0f2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419c470 con 0x7ff4240697b0 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff42a0f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419c5e0 con 0x7ff42406a170 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff42a0f2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419c7a0 con 0x7ff424106130 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4240697b0 0x7ff424197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4240697b0 0x7ff424197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41968/0 (socket says 172.21.15.42:41968) 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 -- 172.21.15.42:0/835776775 learned_addr learned my addr 172.21.15.42:0/835776775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff423fff640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff424106130 0x7ff42419be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff422ffd640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff42406a170 0x7ff424197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 -- 172.21.15.42:0/835776775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff424106130 msgr2=0x7ff42419be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff424106130 0x7ff42419be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 -- 172.21.15.42:0/835776775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff42406a170 msgr2=0x7ff424197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff42406a170 0x7ff424197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 -- 172.21.15.42:0/835776775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c009900 con 0x7ff4240697b0 2024-03-12T04:23:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff423fff640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff424106130 0x7ff42419be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff4237fe640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4240697b0 0x7ff424197600 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e9f0 tx=0x7ff41400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.478+0000 7ff420ff9640 1 -- 172.21.15.42:0/835776775 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414009700 con 0x7ff4240697b0 2024-03-12T04:23:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.479+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42419ca80 con 0x7ff4240697b0 2024-03-12T04:23:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.479+0000 7ff420ff9640 1 -- 172.21.15.42:0/835776775 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff41400cd80 con 0x7ff4240697b0 2024-03-12T04:23:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.479+0000 7ff420ff9640 1 -- 172.21.15.42:0/835776775 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414018840 con 0x7ff4240697b0 2024-03-12T04:23:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.479+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a4300 con 0x7ff4240697b0 2024-03-12T04:23:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff420ff9640 1 -- 172.21.15.42:0/835776775 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7ff4140189a0 con 0x7ff4240697b0 2024-03-12T04:23:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 Processor -- start 2024-03-12T04:23:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 -- start start 2024-03-12T04:23:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb36872b0 0x55edb3687690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb3687bd0 0x55edb3690270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb3692ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb35f6280 con 0x55edb36907b0 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb35f37b0 con 0x55edb3687bd0 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb35eef90 con 0x55edb36872b0 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff422ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb3692ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff422ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb3692ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41974/0 (socket says 172.21.15.42:41974) 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff422ffd640 1 -- 172.21.15.42:0/1904855424 learned_addr learned my addr 172.21.15.42:0/1904855424 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff420ff9640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff40c0a1ba0 0x7ff40c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.481+0000 7ff4237fe640 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb3687bd0 0x55edb3690270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff423fff640 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb36872b0 0x55edb3687690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff423fff640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff40c0a1ba0 0x7ff40c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff422ffd640 1 -- 172.21.15.42:0/1904855424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb36872b0 msgr2=0x55edb3687690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff422ffd640 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb36872b0 0x55edb3687690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff422ffd640 1 -- 172.21.15.42:0/1904855424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb3687bd0 msgr2=0x55edb3690270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff422ffd640 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb3687bd0 0x55edb3690270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff422ffd640 1 -- 172.21.15.42:0/1904855424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edb35ad570 con 0x55edb36907b0 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff420ff9640 1 -- 172.21.15.42:0/835776775 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff414014070 con 0x7ff4240697b0 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff422ffd640 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb3692ba0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff418009b60 tx=0x7ff41800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d674f4a2d7018c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff423fff640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff40c0a1ba0 0x7ff40c0a4060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff41c02c630 tx=0x7ff41c0095c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff409ffb640 1 -- 172.21.15.42:0/1904855424 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418009e50 con 0x55edb36907b0 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff409ffb640 1 -- 172.21.15.42:0/1904855424 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff418011920 con 0x55edb36907b0 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.482+0000 7ff409ffb640 1 -- 172.21.15.42:0/1904855424 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418010860 con 0x55edb36907b0 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1904855424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 msgr2=0x55edb3692ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb3692ba0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff418009b60 tx=0x7ff41800e6c0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1904855424 shutdown_connections 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb3692ba0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb3687bd0 0x55edb3690270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1904855424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb36872b0 0x55edb3687690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1904855424 >> 172.21.15.42:0/1904855424 conn(0x55edb3681990 msgr2=0x55edb3681d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1904855424 shutdown_connections 2024-03-12T04:23:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1904855424 wait complete. 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 -- start start 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.483+0000 7ff42cd9d740 1 Processor -- start 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb3690180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb32c2430 con 0x55edb36872b0 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb327ee00 con 0x55edb36907b0 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb3573fd0 con 0x55edb3687bd0 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff42cd9d740 10 client.0 _ll_register_callbacks cb 0x55edb35c9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb3690180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb3690180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41976/0 (socket says 172.21.15.42:41976) 2024-03-12T04:23:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 -- 172.21.15.42:0/1831470052 learned_addr learned my addr 172.21.15.42:0/1831470052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff422ffd640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff423fff640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 -- 172.21.15.42:0/1831470052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 msgr2=0x55edb368d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 -- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 msgr2=0x55edb368db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 -- 172.21.15.42:0/1831470052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff418009790 con 0x55edb36872b0 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff423fff640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff422ffd640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff4237fe640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb3690180 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff4140cb410 tx=0x7ff41409b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45923ff3f301a67e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.484+0000 7ff402ffd640 1 -- 172.21.15.42:0/1831470052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140a3030 con 0x55edb36872b0 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff402ffd640 1 -- 172.21.15.42:0/1831470052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff414096f00 con 0x55edb36872b0 2024-03-12T04:23:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edb32ee7e0 con 0x55edb36872b0 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff402ffd640 1 -- 172.21.15.42:0/1831470052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4140970c0 con 0x55edb36872b0 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 10 client.17835 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55edb35ed610 con 0x55edb36872b0 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff402ffd640 1 -- 172.21.15.42:0/1831470052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55edb32ee7e0 con 0x55edb36872b0 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff402ffd640 1 -- 172.21.15.42:0/1831470052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4140965e0 con 0x55edb36872b0 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 10 client.17835 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 20 client.17835 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36a0820 0x55edb36a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 4 client.17835 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.485+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edb332b720 con 0x55edb36a0820 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.486+0000 7ff422ffd640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36a0820 0x55edb36a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.486+0000 7ff422ffd640 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36a0820 0x55edb36a2c10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a7b6b4a0f9d58c server_cookie=873e71580c0f21b5 in_seq=0 out_seq=0 2024-03-12T04:23:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.486+0000 7ff402ffd640 10 client.17835 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff402ffd640 1 -- 172.21.15.42:0/1831470052 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55edb332b720 con 0x55edb36a0820 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff402ffd640 10 client.17835 handle_command_reply: tid=0 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 client.17835 shutdown 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 msgr2=0x55edb3690180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb3690180 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff4140cb410 tx=0x7ff41409b010 comp rx=0 tx=0).stop 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 shutdown_connections 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36a0820 0x55edb36a2c10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff402ffd640 0 client.17835 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/1831470052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb3690180 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 >> 172.21.15.42:0/1831470052 conn(0x55edb3681990 msgr2=0x55edb3683730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 shutdown_connections 2024-03-12T04:23:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 1 -- 172.21.15.42:0/1831470052 wait complete. 2024-03-12T04:23:43.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.487+0000 7ff42cd9d740 20 client.17835 trim_cache size 0 max 16384 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 Processor -- start 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 -- start start 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb368fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb3690180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb367d0f0 con 0x55edb36872b0 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb36811d0 con 0x55edb36907b0 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb3686f10 con 0x55edb3687bd0 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.502+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb368fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb368fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41984/0 (socket says 172.21.15.42:41984) 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 -- 172.21.15.42:0/3002041323 learned_addr learned my addr 172.21.15.42:0/3002041323 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff423fff640 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 -- 172.21.15.42:0/3002041323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 msgr2=0x55edb3690180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb3690180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 -- 172.21.15.42:0/3002041323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 msgr2=0x55edb368cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 -- 172.21.15.42:0/3002041323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edb3589ac0 con 0x55edb36872b0 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff423fff640 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4237fe640 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb368fc40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff414098b40 tx=0x7ff414000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb92bedc503e813 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4097fa640 1 -- 172.21.15.42:0/3002041323 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41409a3d0 con 0x55edb36872b0 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4097fa640 1 -- 172.21.15.42:0/3002041323 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff4140a3030 con 0x55edb36872b0 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.503+0000 7ff4097fa640 1 -- 172.21.15.42:0/3002041323 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41409bbc0 con 0x55edb36872b0 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- 172.21.15.42:0/3002041323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 msgr2=0x55edb368fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb368fc40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff414098b40 tx=0x7ff414000bc0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- 172.21.15.42:0/3002041323 shutdown_connections 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36907b0 0x55edb368cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb3690180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/3002041323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36872b0 0x55edb368fc40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- 172.21.15.42:0/3002041323 >> 172.21.15.42:0/3002041323 conn(0x55edb3681990 msgr2=0x55edb368bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- 172.21.15.42:0/3002041323 shutdown_connections 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- 172.21.15.42:0/3002041323 wait complete. 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- start start 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 Processor -- start 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36872b0 0x55edb368a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb368af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb36f6ed0 con 0x55edb36907b0 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb36f7040 con 0x55edb36872b0 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.504+0000 7ff42cd9d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb36f7200 con 0x55edb3687bd0 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff42cd9d740 10 client.0 _ll_register_callbacks cb 0x7ff418009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb368af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb368af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41988/0 (socket says 172.21.15.42:41988) 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 -- 172.21.15.42:0/2331732440 learned_addr learned my addr 172.21.15.42:0/2331732440 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff4237fe640 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36872b0 0x55edb368a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff422ffd640 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 -- 172.21.15.42:0/2331732440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 msgr2=0x55edb368a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 -- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36872b0 msgr2=0x55edb368a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36872b0 0x55edb368a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 -- 172.21.15.42:0/2331732440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edb35ed610 con 0x55edb36907b0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff423fff640 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb368af40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff41c036000 tx=0x7ff41c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b791647e78293c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff402ffd640 1 -- 172.21.15.42:0/2331732440 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0327d0 con 0x55edb36907b0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff402ffd640 1 -- 172.21.15.42:0/2331732440 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff41c032930 con 0x55edb36907b0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff418011920 con 0x55edb36907b0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff402ffd640 1 -- 172.21.15.42:0/2331732440 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff41c003970 con 0x55edb36907b0 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff42cd9d740 10 client.17841 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:43.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.505+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55edb35af6a0 con 0x55edb36907b0 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff402ffd640 1 -- 172.21.15.42:0/2331732440 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff418011920 con 0x55edb36907b0 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff402ffd640 1 -- 172.21.15.42:0/2331732440 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff41c004680 con 0x55edb36907b0 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff42cd9d740 10 client.17841 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff42cd9d740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff42cd9d740 20 client.17841 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36f8a60 0x55edb36fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff42cd9d740 4 client.17841 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55edb337a890 con 0x55edb36f8a60 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff4237fe640 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36f8a60 0x55edb36fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff4237fe640 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36f8a60 0x55edb36fae80 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de03a60d8b0a9e45 server_cookie=16b0f77b9dcc1520 in_seq=0 out_seq=0 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.506+0000 7ff402ffd640 10 client.17841 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff402ffd640 1 -- 172.21.15.42:0/2331732440 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55edb337a890 con 0x55edb36f8a60 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff402ffd640 10 client.17841 handle_command_reply: tid=0 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 client.17841 shutdown 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 msgr2=0x55edb368af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb368af40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff41c036000 tx=0x7ff41c009800 comp rx=0 tx=0).stop 2024-03-12T04:23:43.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 shutdown_connections 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55edb36f8a60 0x55edb36fae80 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55edb36907b0 0x55edb368af40 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edb3687bd0 0x55edb368a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 --2- 172.21.15.42:0/2331732440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edb36872b0 0x55edb368a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff402ffd640 0 client.17841 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 >> 172.21.15.42:0/2331732440 conn(0x55edb3681990 msgr2=0x55edb368cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 shutdown_connections 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 1 -- 172.21.15.42:0/2331732440 wait complete. 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42cd9d740 20 client.17841 trim_cache size 0 max 16384 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:23:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:23:43.517 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:23:43.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff40c0a1ba0 msgr2=0x7ff40c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff40c0a1ba0 0x7ff40c0a4060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff41c02c630 tx=0x7ff41c0095c0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4240697b0 msgr2=0x7ff424197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4240697b0 0x7ff424197600 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e9f0 tx=0x7ff41400eeb0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 shutdown_connections 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff40c0a1ba0 0x7ff40c0a4060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff424106130 0x7ff42419be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff42406a170 0x7ff424197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 --2- 172.21.15.42:0/835776775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4240697b0 0x7ff424197600 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.507+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 >> 172.21.15.42:0/835776775 conn(0x7ff424076e20 msgr2=0x7ff4240fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.508+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 shutdown_connections 2024-03-12T04:23:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.508+0000 7ff42a0f2640 1 -- 172.21.15.42:0/835776775 wait complete. 2024-03-12T04:23:43.521 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:23:43.521 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 -- 172.21.15.42:0/2662046218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4100d90 msgr2=0x7f62c410ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2662046218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4100d90 0x7f62c410ce60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009cd0 tx=0x7f62bc02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 -- 172.21.15.42:0/2662046218 shutdown_connections 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2662046218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4100d90 0x7f62c410ce60 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2662046218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62c41003d0 0x7f62c4100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2662046218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c4105060 0x7f62c4105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 -- 172.21.15.42:0/2662046218 >> 172.21.15.42:0/2662046218 conn(0x7f62c4075e80 msgr2=0x7f62c4076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 -- 172.21.15.42:0/2662046218 shutdown_connections 2024-03-12T04:23:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.621+0000 7f62cb10d640 1 -- 172.21.15.42:0/2662046218 wait complete. 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 Processor -- start 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 -- start start 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62c41003d0 0x7f62c41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c4100d90 0x7f62c4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4105060 0x7f62c419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c419c5b0 con 0x7f62c4105060 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c419c720 con 0x7f62c4100d90 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62cb10d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c419c8e0 con 0x7f62c41003d0 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4105060 0x7f62c419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4105060 0x7f62c419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42008/0 (socket says 172.21.15.42:42008) 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 -- 172.21.15.42:0/2111285021 learned_addr learned my addr 172.21.15.42:0/2111285021 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c8ea9640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62c41003d0 0x7f62c41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 -- 172.21.15.42:0/2111285021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62c41003d0 msgr2=0x7f62c41976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c3fff640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c4100d90 0x7f62c4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62c41003d0 0x7f62c41976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 -- 172.21.15.42:0/2111285021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c4100d90 msgr2=0x7f62c4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c4100d90 0x7f62c4197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c96aa640 1 -- 172.21.15.42:0/2111285021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc009900 con 0x7f62c4105060 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.622+0000 7f62c8ea9640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62c41003d0 0x7f62c41976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.623+0000 7f62c96aa640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4105060 0x7f62c419bfb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f62bc004af0 tx=0x7f62bc034c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.623+0000 7f62c1ffb640 1 -- 172.21.15.42:0/2111285021 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc032a30 con 0x7f62c4105060 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.623+0000 7f62c1ffb640 1 -- 172.21.15.42:0/2111285021 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62bc003e20 con 0x7f62c4105060 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.623+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c419cb60 con 0x7f62c4105060 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.623+0000 7f62c1ffb640 1 -- 172.21.15.42:0/2111285021 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc004790 con 0x7f62c4105060 2024-03-12T04:23:43.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.623+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c41a4440 con 0x7f62c4105060 2024-03-12T04:23:43.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.624+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c41017a0 con 0x7f62c4105060 2024-03-12T04:23:43.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.625+0000 7f62c1ffb640 1 -- 172.21.15.42:0/2111285021 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f62bc00a040 con 0x7f62c4105060 2024-03-12T04:23:43.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.625+0000 7f62c1ffb640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62b40a1c20 0x7f62b40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.626+0000 7f62c8ea9640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62b40a1c20 0x7f62b40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.626+0000 7f62c1ffb640 1 -- 172.21.15.42:0/2111285021 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f62bc0f04d0 con 0x7f62c4105060 2024-03-12T04:23:43.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.626+0000 7f62c8ea9640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62b40a1c20 0x7f62b40a40e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f62b8005fd0 tx=0x7f62b800b5f0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.630+0000 7f62c1ffb640 1 -- 172.21.15.42:0/2111285021 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62bc0b9470 con 0x7f62c4105060 2024-03-12T04:23:43.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.849+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62c4109510 con 0x7f62c4105060 2024-03-12T04:23:43.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.850+0000 7f62c1ffb640 1 -- 172.21.15.42:0/2111285021 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f62bc0bcb30 con 0x7f62c4105060 2024-03-12T04:23:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:23:43.853 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:23:43.853 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:23:43.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62b40a1c20 msgr2=0x7f62b40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62b40a1c20 0x7f62b40a40e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f62b8005fd0 tx=0x7f62b800b5f0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4105060 msgr2=0x7f62c419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4105060 0x7f62c419bfb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f62bc004af0 tx=0x7f62bc034c90 comp rx=0 tx=0).stop 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 shutdown_connections 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f62b40a1c20 0x7f62b40a40e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62c4105060 0x7f62c419bfb0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c4100d90 0x7f62c4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 --2- 172.21.15.42:0/2111285021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62c41003d0 0x7f62c41976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 >> 172.21.15.42:0/2111285021 conn(0x7f62c4075e80 msgr2=0x7f62c4069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 shutdown_connections 2024-03-12T04:23:43.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.852+0000 7f62cb10d640 1 -- 172.21.15.42:0/2111285021 wait complete. 2024-03-12T04:23:43.865 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:43.865 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:23:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 -- 172.21.15.42:0/2533848748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 msgr2=0x7f4e200689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 --2- 172.21.15.42:0/2533848748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e200689d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009930 tx=0x7f4e1802bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 -- 172.21.15.42:0/2533848748 shutdown_connections 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 --2- 172.21.15.42:0/2533848748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e201064c0 0x7f4e2010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 --2- 172.21.15.42:0/2533848748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e20068fa0 0x7f4e20105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 --2- 172.21.15.42:0/2533848748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e200689d0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 -- 172.21.15.42:0/2533848748 >> 172.21.15.42:0/2533848748 conn(0x7f4e20077180 msgr2=0x7f4e20077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 -- 172.21.15.42:0/2533848748 shutdown_connections 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.967+0000 7f4e26c81640 1 -- 172.21.15.42:0/2533848748 wait complete. 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 Processor -- start 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 -- start start 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e2019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e201064c0 0x7f4e2019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e20199700 0x7f4e20199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2019a1d0 con 0x7f4e200685f0 2024-03-12T04:23:43.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2019a340 con 0x7f4e201064c0 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e26c81640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2019a500 con 0x7f4e20199700 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e24a1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e2019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e24a1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e2019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42020/0 (socket says 172.21.15.42:42020) 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e24a1d640 1 -- 172.21.15.42:0/102106502 learned_addr learned my addr 172.21.15.42:0/102106502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.968+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e201064c0 0x7f4e2019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e24a1d640 1 -- 172.21.15.42:0/102106502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e20199700 msgr2=0x7f4e20199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e2521e640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e20199700 0x7f4e20199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e20199700 0x7f4e20199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e24a1d640 1 -- 172.21.15.42:0/102106502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e201064c0 msgr2=0x7f4e2019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e201064c0 0x7f4e2019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e24a1d640 1 -- 172.21.15.42:0/102106502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e18009560 con 0x7f4e200685f0 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e201064c0 0x7f4e2019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e2521e640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e20199700 0x7f4e20199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e2019f640 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009230 tx=0x7f4e180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e1dffb640 1 -- 172.21.15.42:0/102106502 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e180044c0 con 0x7f4e200685f0 2024-03-12T04:23:43.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e1dffb640 1 -- 172.21.15.42:0/102106502 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e1802de90 con 0x7f4e200685f0 2024-03-12T04:23:43.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e1dffb640 1 -- 172.21.15.42:0/102106502 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18005400 con 0x7f4e200685f0 2024-03-12T04:23:43.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e201a6440 con 0x7f4e200685f0 2024-03-12T04:23:43.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.969+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e201a6920 con 0x7f4e200685f0 2024-03-12T04:23:43.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.971+0000 7f4e1dffb640 1 -- 172.21.15.42:0/102106502 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f4e18005560 con 0x7f4e200685f0 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 Processor -- start 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 -- start start 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf759c960 con 0x55fbf75e1250 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf7550310 con 0x55fbf75ea750 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e29853740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf754d840 con 0x55fbf75e1b70 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e1dffb640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4e0c0a1ba0 0x7f4e0c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42022/0 (socket says 172.21.15.42:42022) 2024-03-12T04:23:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 -- 172.21.15.42:0/2266099612 learned_addr learned my addr 172.21.15.42:0/2266099612 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4e0c0a1ba0 0x7f4e0c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e1dffb640 1 -- 172.21.15.42:0/102106502 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4e180ef890 con 0x7f4e200685f0 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 -- 172.21.15.42:0/2266099612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 msgr2=0x55fbf75ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 -- 172.21.15.42:0/2266099612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 msgr2=0x55fbf75ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e2521e640 1 -- 172.21.15.42:0/2266099612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbf750b060 con 0x55fbf75e1250 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.972+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e2521e640 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e1630 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4e08013b20 tx=0x7f4e08010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d74f51f5d3e5979 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e2521e640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4e0c0a1ba0 0x7f4e0c0a4060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4e08010870 tx=0x7f4e0801b000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e16ffd640 1 -- 172.21.15.42:0/2266099612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0801a9d0 con 0x55fbf75e1250 2024-03-12T04:23:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e16ffd640 1 -- 172.21.15.42:0/2266099612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e08019040 con 0x55fbf75e1250 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e16ffd640 1 -- 172.21.15.42:0/2266099612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0801f930 con 0x55fbf75e1250 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e29853740 1 -- 172.21.15.42:0/2266099612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 msgr2=0x55fbf75e1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e29853740 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e1630 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4e08013b20 tx=0x7f4e08010370 comp rx=0 tx=0).stop 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e29853740 1 -- 172.21.15.42:0/2266099612 shutdown_connections 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e29853740 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e29853740 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e29853740 1 --2- 172.21.15.42:0/2266099612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e1630 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.973+0000 7f4e29853740 1 -- 172.21.15.42:0/2266099612 >> 172.21.15.42:0/2266099612 conn(0x55fbf75db980 msgr2=0x55fbf75dbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 -- 172.21.15.42:0/2266099612 shutdown_connections 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 -- 172.21.15.42:0/2266099612 wait complete. 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 -- start start 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 Processor -- start 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf7549020 con 0x55fbf75e1250 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf721c430 con 0x55fbf75ea750 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf71d8e00 con 0x55fbf75e1b70 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e29853740 10 client.0 _ll_register_callbacks cb 0x55fbf74f0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e24a1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e24a1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42024/0 (socket says 172.21.15.42:42024) 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.974+0000 7f4e24a1d640 1 -- 172.21.15.42:0/2417183860 learned_addr learned my addr 172.21.15.42:0/2417183860 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e2521e640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e24a1d640 1 -- 172.21.15.42:0/2417183860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 msgr2=0x55fbf75e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e24a1d640 1 -- 172.21.15.42:0/2417183860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 msgr2=0x55fbf75e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e24a1d640 1 -- 172.21.15.42:0/2417183860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e08013700 con 0x55fbf75e1250 2024-03-12T04:23:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e7670 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4e180ed170 tx=0x7f4e180b89c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3b1f6f342f918e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4dfffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4dfffff640 1 -- 172.21.15.42:0/2417183860 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e180bccf0 con 0x55fbf75e1250 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4dfffff640 1 -- 172.21.15.42:0/2417183860 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4e180bc5f0 con 0x55fbf75e1250 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbf72487e0 con 0x55fbf75e1250 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4dfffff640 1 -- 172.21.15.42:0/2417183860 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e180c0740 con 0x55fbf75e1250 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e29853740 10 client.17859 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fbf75476a0 con 0x55fbf75e1250 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4dfffff640 1 -- 172.21.15.42:0/2417183860 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55fbf72487e0 con 0x55fbf75e1250 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4dfffff640 1 -- 172.21.15.42:0/2417183860 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4e1803c070 con 0x55fbf75e1250 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e29853740 10 client.17859 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e29853740 10 client.17859 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.975+0000 7f4e29853740 20 client.17859 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:43.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4e29853740 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf75fa7f0 0x55fbf75fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4e29853740 4 client.17859 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbf7285720 con 0x55fbf75fa7f0 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf75fa7f0 0x55fbf75fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4dfffff640 10 client.17859 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf75fa7f0 0x55fbf75fcc00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34f82661cb5ff7f6 server_cookie=2accf2593f4247ed in_seq=0 out_seq=0 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4dfffff640 1 -- 172.21.15.42:0/2417183860 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fbf7285720 con 0x55fbf75fa7f0 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4dfffff640 10 client.17859 handle_command_reply: tid=0 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.976+0000 7f4e29853740 1 client.17859 shutdown 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 msgr2=0x55fbf75e7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e7670 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4e180ed170 tx=0x7f4e180b89c0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 shutdown_connections 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf75fa7f0 0x55fbf75fcc00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 --2- 172.21.15.42:0/2417183860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1250 0x55fbf75e7670 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4dfffff640 0 client.17859 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:43.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 >> 172.21.15.42:0/2417183860 conn(0x55fbf75db980 msgr2=0x55fbf75dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 shutdown_connections 2024-03-12T04:23:43.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 1 -- 172.21.15.42:0/2417183860 wait complete. 2024-03-12T04:23:43.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.977+0000 7f4e29853740 20 client.17859 trim_cache size 0 max 16384 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.988+0000 7f4e29853740 1 Processor -- start 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.988+0000 7f4e29853740 1 -- start start 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e29853740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75e1250 0x55fbf75e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e29853740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e29853740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75ea750 0x55fbf75e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e29853740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf74ce130 con 0x55fbf75e1b70 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e29853740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf75cda40 con 0x55fbf75e1250 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e29853740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf75ecb40 con 0x55fbf75ea750 2024-03-12T04:23:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42040/0 (socket says 172.21.15.42:42040) 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 -- 172.21.15.42:0/2208204919 learned_addr learned my addr 172.21.15.42:0/2208204919 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 -- 172.21.15.42:0/2208204919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75ea750 msgr2=0x55fbf75e48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75e1250 0x55fbf75e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75ea750 0x55fbf75e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 -- 172.21.15.42:0/2208204919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75e1250 msgr2=0x55fbf75e9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75e1250 0x55fbf75e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 -- 172.21.15.42:0/2208204919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbf74e3c20 con 0x55fbf75e1b70 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7f10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4e2006a8e0 tx=0x7f4e1000a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387969e7744ad0a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e167fc640 1 -- 172.21.15.42:0/2208204919 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10016040 con 0x55fbf75e1b70 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e167fc640 1 -- 172.21.15.42:0/2208204919 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e10003e20 con 0x55fbf75e1b70 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.989+0000 7f4e167fc640 1 -- 172.21.15.42:0/2208204919 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10018820 con 0x55fbf75e1b70 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 -- 172.21.15.42:0/2208204919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 msgr2=0x55fbf75e7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7f10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4e2006a8e0 tx=0x7f4e1000a520 comp rx=0 tx=0).stop 2024-03-12T04:23:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 -- 172.21.15.42:0/2208204919 shutdown_connections 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75ea750 0x55fbf75e48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75e7f10 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 --2- 172.21.15.42:0/2208204919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75e1250 0x55fbf75e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 -- 172.21.15.42:0/2208204919 >> 172.21.15.42:0/2208204919 conn(0x55fbf75db980 msgr2=0x55fbf75d6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 -- 172.21.15.42:0/2208204919 shutdown_connections 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 -- 172.21.15.42:0/2208204919 wait complete. 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 -- start start 2024-03-12T04:23:43.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.990+0000 7f4e29853740 1 Processor -- start 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e29853740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1250 0x55fbf75f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e29853740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75efc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e29853740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75f0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e29853740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf75f0750 con 0x55fbf75e1b70 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e29853740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf75f08c0 con 0x55fbf75ea750 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e29853740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf75f0a80 con 0x55fbf75e1250 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e29853740 10 client.0 _ll_register_callbacks cb 0x7f4e0800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:43.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75efc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42046/0 (socket says 172.21.15.42:42046) 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 -- 172.21.15.42:0/3827764202 learned_addr learned my addr 172.21.15.42:0/3827764202 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1250 0x55fbf75f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 -- 172.21.15.42:0/3827764202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1250 msgr2=0x55fbf75f1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e2521e640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1250 0x55fbf75f1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 -- 172.21.15.42:0/3827764202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 msgr2=0x55fbf75f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75f0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 -- 172.21.15.42:0/3827764202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e10018b70 con 0x55fbf75e1b70 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1250 0x55fbf75f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4e1ffff640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75efc00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000a620 tx=0x7f4e1000ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8db50012958afbe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.991+0000 7f4dfffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4dfffff640 1 -- 172.21.15.42:0/3827764202 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10026070 con 0x55fbf75e1b70 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4dfffff640 1 -- 172.21.15.42:0/3827764202 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4e10005570 con 0x55fbf75e1b70 2024-03-12T04:23:44.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e08019040 con 0x55fbf75e1b70 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4dfffff640 1 -- 172.21.15.42:0/3827764202 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e10003ca0 con 0x55fbf75e1b70 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 10 client.17865 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fbf75098c0 con 0x55fbf75e1b70 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4dfffff640 1 -- 172.21.15.42:0/3827764202 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e08019040 con 0x55fbf75e1b70 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4dfffff640 1 -- 172.21.15.42:0/3827764202 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4e100043b0 con 0x55fbf75e1b70 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 10 client.17865 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 10 client.17865 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 20 client.17865 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf76522b0 0x55fbf76546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 4 client.17865 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fbf72d4890 con 0x55fbf76522b0 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.992+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf76522b0 0x55fbf76546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4dfffff640 10 client.17865 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e24a1d640 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf76522b0 0x55fbf76546d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8048a823c30d13e4 server_cookie=df58e83e0e8e9328 in_seq=0 out_seq=0 2024-03-12T04:23:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4dfffff640 1 -- 172.21.15.42:0/3827764202 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fbf72d4890 con 0x55fbf76522b0 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4dfffff640 10 client.17865 handle_command_reply: tid=0 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 client.17865 shutdown 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 msgr2=0x55fbf75efc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75efc00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000a620 tx=0x7f4e1000ad10 comp rx=0 tx=0).stop 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 shutdown_connections 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fbf75ea750 0x55fbf75f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55fbf76522b0 0x55fbf76546d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbf75e1b70 0x55fbf75efc00 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 --2- 172.21.15.42:0/3827764202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fbf75e1250 0x55fbf75f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4dfffff640 0 client.17865 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 >> 172.21.15.42:0/3827764202 conn(0x55fbf75db980 msgr2=0x55fbf75da2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 shutdown_connections 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 1 -- 172.21.15.42:0/3827764202 wait complete. 2024-03-12T04:23:44.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.993+0000 7f4e29853740 20 client.17865 trim_cache size 0 max 16384 2024-03-12T04:23:44.003 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4e0c0a1ba0 msgr2=0x7f4e0c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4e0c0a1ba0 0x7f4e0c0a4060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4e08010870 tx=0x7f4e0801b000 comp rx=0 tx=0).stop 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 msgr2=0x7f4e2019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e2019f640 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009230 tx=0x7f4e180040c0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 shutdown_connections 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4e0c0a1ba0 0x7f4e0c0a4060 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e20199700 0x7f4e20199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e201064c0 0x7f4e2019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 --2- 172.21.15.42:0/102106502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e200685f0 0x7f4e2019f640 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 >> 172.21.15.42:0/102106502 conn(0x7f4e20077180 msgr2=0x7f4e2010c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 shutdown_connections 2024-03-12T04:23:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:43.994+0000 7f4e26c81640 1 -- 172.21.15.42:0/102106502 wait complete. 2024-03-12T04:23:44.007 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:44.007 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:23:44.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 -- 172.21.15.42:0/1859769073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 msgr2=0x7ff82c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 --2- 172.21.15.42:0/1859769073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c10cfb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff824009cd0 tx=0x7ff82402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 -- 172.21.15.42:0/1859769073 shutdown_connections 2024-03-12T04:23:44.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 --2- 172.21.15.42:0/1859769073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c10cfb0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 --2- 172.21.15.42:0/1859769073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c100cc0 0x7ff82c104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 --2- 172.21.15.42:0/1859769073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff82c100310 0x7ff82c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 -- 172.21.15.42:0/1859769073 >> 172.21.15.42:0/1859769073 conn(0x7ff82c0771e0 msgr2=0x7ff82c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 -- 172.21.15.42:0/1859769073 shutdown_connections 2024-03-12T04:23:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.109+0000 7ff831177640 1 -- 172.21.15.42:0/1859769073 wait complete. 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 Processor -- start 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 -- start start 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff82c100310 0x7ff82c101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c1022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c102820 0x7ff82c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1a5f20 con 0x7ff82c1053e0 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1a6060 con 0x7ff82c102820 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff831177640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1a61f0 con 0x7ff82c100310 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff823fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c1022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff823fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c1022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42078/0 (socket says 172.21.15.42:42078) 2024-03-12T04:23:44.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff823fff640 1 -- 172.21.15.42:0/2268752167 learned_addr learned my addr 172.21.15.42:0/2268752167 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff82b59e640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c102820 0x7ff82c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff82c100310 0x7ff82c101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff823fff640 1 -- 172.21.15.42:0/2268752167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff82c100310 msgr2=0x7ff82c101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff823fff640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff82c100310 0x7ff82c101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.110+0000 7ff823fff640 1 -- 172.21.15.42:0/2268752167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c102820 msgr2=0x7ff82c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff823fff640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c102820 0x7ff82c1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff823fff640 1 -- 172.21.15.42:0/2268752167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff824009900 con 0x7ff82c1053e0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff82b59e640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c102820 0x7ff82c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff823fff640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c1022e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c9a0 tx=0x7ff81400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff828d99640 1 -- 172.21.15.42:0/2268752167 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814007bb0 con 0x7ff82c1053e0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c1a64d0 con 0x7ff82c1053e0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff828d99640 1 -- 172.21.15.42:0/2268752167 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff814007d10 con 0x7ff82c1053e0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff828d99640 1 -- 172.21.15.42:0/2268752167 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400f450 con 0x7ff82c1053e0 2024-03-12T04:23:44.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.111+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c1a6a30 con 0x7ff82c1053e0 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.113+0000 7ff828d99640 1 -- 172.21.15.42:0/2268752167 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7ff814004eb0 con 0x7ff82c1053e0 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.113+0000 7ff833e26740 1 Processor -- start 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.113+0000 7ff833e26740 1 -- start start 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff833e26740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640f5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff833e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5b70 0x5572640fe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff833e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640fe750 0x557264100b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff833e26740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572640b0960 con 0x5572640f5250 2024-03-12T04:23:44.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff833e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557264064310 con 0x5572640fe750 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff833e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557264061840 con 0x5572640f5b70 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff828d99640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff8180a1ba0 0x7ff8180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640f5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42086/0 (socket says 172.21.15.42:42086) 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82b59e640 1 -- 172.21.15.42:0/3895488891 learned_addr learned my addr 172.21.15.42:0/3895488891 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5b70 0x5572640fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82b59e640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff8180a1ba0 0x7ff8180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff828d99640 1 -- 172.21.15.42:0/2268752167 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff8140cd8b0 con 0x7ff82c1053e0 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff823fff640 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640fe750 0x557264100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82ad9d640 1 -- 172.21.15.42:0/3895488891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640fe750 msgr2=0x557264100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640fe750 0x557264100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82ad9d640 1 -- 172.21.15.42:0/3895488891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 msgr2=0x5572640f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82ad9d640 1 -- 172.21.15.42:0/3895488891 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55726401f060 con 0x5572640f5b70 2024-03-12T04:23:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.114+0000 7ff82b59e640 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.115+0000 7ff82b59e640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff8180a1ba0 0x7ff8180a4060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff824009ca0 tx=0x7ff8240095c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.115+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5b70 0x5572640fe210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00cb10 tx=0x7ff81c00e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7f0ff461e6f51 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.115+0000 7ff8217fa640 1 -- 172.21.15.42:0/3895488891 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004020 con 0x5572640f5b70 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.115+0000 7ff8217fa640 1 -- 172.21.15.42:0/3895488891 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff81c0035f0 con 0x5572640f5b70 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.115+0000 7ff8217fa640 1 -- 172.21.15.42:0/3895488891 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c010690 con 0x5572640f5b70 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- 172.21.15.42:0/3895488891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5b70 msgr2=0x5572640fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5b70 0x5572640fe210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00cb10 tx=0x7ff81c00e350 comp rx=0 tx=0).stop 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- 172.21.15.42:0/3895488891 shutdown_connections 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640fe750 0x557264100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5b70 0x5572640fe210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 --2- 172.21.15.42:0/3895488891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- 172.21.15.42:0/3895488891 >> 172.21.15.42:0/3895488891 conn(0x5572640ef980 msgr2=0x5572640efd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- 172.21.15.42:0/3895488891 shutdown_connections 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- 172.21.15.42:0/3895488891 wait complete. 2024-03-12T04:23:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- start start 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 Processor -- start 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5250 0x5572640fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5b70 0x5572640fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 0x5572640fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726405d020 con 0x5572640f5b70 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557263d30430 con 0x5572640f5250 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557263cece00 con 0x5572640fe750 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff833e26740 10 client.0 _ll_register_callbacks cb 0x557264004cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff823fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5b70 0x5572640fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff823fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5b70 0x5572640fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42090/0 (socket says 172.21.15.42:42090) 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff823fff640 1 -- 172.21.15.42:0/1796061319 learned_addr learned my addr 172.21.15.42:0/1796061319 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.116+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5250 0x5572640fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff82b59e640 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 0x5572640fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff823fff640 1 -- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 msgr2=0x5572640fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff823fff640 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 0x5572640fc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff823fff640 1 -- 172.21.15.42:0/1796061319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5250 msgr2=0x5572640fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff823fff640 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5250 0x5572640fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff823fff640 1 -- 172.21.15.42:0/1796061319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c00c770 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff823fff640 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5b70 0x5572640fbca0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff8140968b0 tx=0x7ff81409a3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbab3c1b5f5a4df server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff8127fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff8127fc640 1 -- 172.21.15.42:0/1796061319 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140cb450 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff8127fc640 1 -- 172.21.15.42:0/1796061319 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff81409aba0 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557263d5c7e0 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff8127fc640 1 -- 172.21.15.42:0/1796061319 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff81409a4a0 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff833e26740 10 client.17877 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55726405b6a0 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff8127fc640 1 -- 172.21.15.42:0/1796061319 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557263d5c7e0 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.117+0000 7ff8127fc640 1 -- 172.21.15.42:0/1796061319 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff814020070 con 0x5572640f5b70 2024-03-12T04:23:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff833e26740 10 client.17877 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff833e26740 10 client.17877 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff833e26740 20 client.17877 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff833e26740 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55726410eb90 0x557264110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff833e26740 4 client.17877 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557263d99720 con 0x55726410eb90 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55726410eb90 0x557264110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55726410eb90 0x557264110fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8e6f44501c4a563 server_cookie=132067c81c79ad15 in_seq=0 out_seq=0 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.118+0000 7ff8127fc640 10 client.17877 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff8127fc640 1 -- 172.21.15.42:0/1796061319 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557263d99720 con 0x55726410eb90 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff8127fc640 10 client.17877 handle_command_reply: tid=0 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 client.17877 shutdown 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5b70 msgr2=0x5572640fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5b70 0x5572640fbca0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff8140968b0 tx=0x7ff81409a3a0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 shutdown_connections 2024-03-12T04:23:44.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55726410eb90 0x557264110fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 0x5572640fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5b70 0x5572640fbca0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 --2- 172.21.15.42:0/1796061319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5250 0x5572640fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 >> 172.21.15.42:0/1796061319 conn(0x5572640ef980 msgr2=0x5572640f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff8127fc640 0 client.17877 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 shutdown_connections 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 1 -- 172.21.15.42:0/1796061319 wait complete. 2024-03-12T04:23:44.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.119+0000 7ff833e26740 20 client.17877 trim_cache size 0 max 16384 2024-03-12T04:23:44.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 Processor -- start 2024-03-12T04:23:44.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 -- start start 2024-03-12T04:23:44.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640fde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 0x5572640fc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 0x5572640f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557263fe2130 con 0x5572640f5250 2024-03-12T04:23:44.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572640f45d0 con 0x5572640f5b70 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff833e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572640f7380 con 0x5572640fe750 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff82ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff82ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640fde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42102/0 (socket says 172.21.15.42:42102) 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.131+0000 7ff82ad9d640 1 -- 172.21.15.42:0/1867078194 learned_addr learned my addr 172.21.15.42:0/1867078194 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff82ad9d640 1 -- 172.21.15.42:0/1867078194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 msgr2=0x5572640f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/1867078194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 0x5572640f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff82ad9d640 1 -- 172.21.15.42:0/1867078194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 msgr2=0x5572640fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/1867078194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 0x5572640fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff82ad9d640 1 -- 172.21.15.42:0/1867078194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557263ff7c20 con 0x5572640f5250 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/1867078194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640fde90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00cb10 tx=0x7ff81c00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2693f756e13934 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff820ff9640 1 -- 172.21.15.42:0/1867078194 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c020040 con 0x5572640f5250 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff820ff9640 1 -- 172.21.15.42:0/1867078194 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff81c021040 con 0x5572640f5250 2024-03-12T04:23:44.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff820ff9640 1 -- 172.21.15.42:0/1867078194 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0052a0 con 0x5572640f5250 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff833e26740 1 -- 172.21.15.42:0/1867078194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 msgr2=0x5572640fde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.132+0000 7ff833e26740 1 --2- 172.21.15.42:0/1867078194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640fde90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00cb10 tx=0x7ff81c00c430 comp rx=0 tx=0).stop 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- 172.21.15.42:0/1867078194 shutdown_connections 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 --2- 172.21.15.42:0/1867078194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640fe750 0x5572640f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 --2- 172.21.15.42:0/1867078194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 0x5572640fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 --2- 172.21.15.42:0/1867078194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640f5250 0x5572640fde90 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- 172.21.15.42:0/1867078194 >> 172.21.15.42:0/1867078194 conn(0x5572640ef980 msgr2=0x557264115970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- 172.21.15.42:0/1867078194 shutdown_connections 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- 172.21.15.42:0/1867078194 wait complete. 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- start start 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 Processor -- start 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5250 0x5572641049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 0x557264104f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640fe750 0x55726415f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726415f790 con 0x5572640fe750 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726415f900 con 0x5572640f5b70 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726415fac0 con 0x5572640f5250 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff833e26740 10 client.0 _ll_register_callbacks cb 0x7ff81c00c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff82b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640fe750 0x55726415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff82b59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640fe750 0x55726415f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42112/0 (socket says 172.21.15.42:42112) 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff82b59e640 1 -- 172.21.15.42:0/262443492 learned_addr learned my addr 172.21.15.42:0/262443492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff823fff640 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 0x557264104f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5250 0x5572641049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff82b59e640 1 -- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5250 msgr2=0x5572641049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.133+0000 7ff82b59e640 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5250 0x5572641049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff82b59e640 1 -- 172.21.15.42:0/262443492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 msgr2=0x557264104f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff82b59e640 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 0x557264104f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff82b59e640 1 -- 172.21.15.42:0/262443492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55726401f060 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff82b59e640 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640fe750 0x55726415f220 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff82402c700 tx=0x7ff824032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f786a4ecf354c38b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff8127fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff8127fc640 1 -- 172.21.15.42:0/262443492 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824003b70 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff8127fc640 1 -- 172.21.15.42:0/262443492 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff824003cd0 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff81c0035f0 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff8127fc640 1 -- 172.21.15.42:0/262443492 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff824003e90 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 10 client.17883 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55726401d8c0 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff8127fc640 1 -- 172.21.15.42:0/262443492 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff81c0035f0 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff8127fc640 1 -- 172.21.15.42:0/262443492 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff82404d070 con 0x5572640fe750 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 10 client.17883 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 10 client.17883 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 20 client.17883 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557264167380 0x5572641697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 4 client.17883 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.134+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557263de8890 con 0x557264167380 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557264167380 0x5572641697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff82ad9d640 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557264167380 0x5572641697a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5526706948de927 server_cookie=aaf9eeee5185975d in_seq=0 out_seq=0 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff8127fc640 10 client.17883 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff8127fc640 1 -- 172.21.15.42:0/262443492 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557263de8890 con 0x557264167380 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff8127fc640 10 client.17883 handle_command_reply: tid=0 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 client.17883 shutdown 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640fe750 msgr2=0x55726415f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640fe750 0x55726415f220 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff82402c700 tx=0x7ff824032340 comp rx=0 tx=0).stop 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 shutdown_connections 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557264167380 0x5572641697a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572640fe750 0x55726415f220 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572640f5b70 0x557264104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff8127fc640 0 client.17883 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:23:44.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 --2- 172.21.15.42:0/262443492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572640f5250 0x5572641049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 >> 172.21.15.42:0/262443492 conn(0x5572640ef980 msgr2=0x5572640ee2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 shutdown_connections 2024-03-12T04:23:44.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.135+0000 7ff833e26740 1 -- 172.21.15.42:0/262443492 wait complete. 2024-03-12T04:23:44.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff833e26740 20 client.17883 trim_cache size 0 max 16384 2024-03-12T04:23:44.146 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff8180a1ba0 msgr2=0x7ff8180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff8180a1ba0 0x7ff8180a4060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff824009ca0 tx=0x7ff8240095c0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 msgr2=0x7ff82c1022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c1022e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c9a0 tx=0x7ff81400ce60 comp rx=0 tx=0).stop 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 shutdown_connections 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff8180a1ba0 0x7ff8180a4060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c102820 0x7ff82c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff82c1053e0 0x7ff82c1022e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 --2- 172.21.15.42:0/2268752167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff82c100310 0x7ff82c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 >> 172.21.15.42:0/2268752167 conn(0x7ff82c0771e0 msgr2=0x7ff82c069b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 shutdown_connections 2024-03-12T04:23:44.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.136+0000 7ff831177640 1 -- 172.21.15.42:0/2268752167 wait complete. 2024-03-12T04:23:44.149 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:44.149 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:23:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 -- 172.21.15.42:0/2269404782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440101390 msgr2=0x7f6440101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 --2- 172.21.15.42:0/2269404782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440101390 0x7f6440101770 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6438009930 tx=0x7f643802bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 -- 172.21.15.42:0/2269404782 shutdown_connections 2024-03-12T04:23:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 --2- 172.21.15.42:0/2269404782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6440106460 0x7f644010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 --2- 172.21.15.42:0/2269404782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6440101d40 0x7f6440105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 --2- 172.21.15.42:0/2269404782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440101390 0x7f6440101770 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 -- 172.21.15.42:0/2269404782 >> 172.21.15.42:0/2269404782 conn(0x7f64400fcbb0 msgr2=0x7f64400fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 -- 172.21.15.42:0/2269404782 shutdown_connections 2024-03-12T04:23:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.252+0000 7f6444e88640 1 -- 172.21.15.42:0/2269404782 wait complete. 2024-03-12T04:23:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 Processor -- start 2024-03-12T04:23:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 -- start start 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6440101390 0x7f6440073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6440106460 0x7f64400739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440073f00 0x7f64401a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401a5f30 con 0x7f6440073f00 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401a60a0 con 0x7f6440106460 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f6444e88640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401a6260 con 0x7f6440101390 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440073f00 0x7f64401a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440073f00 0x7f64401a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42130/0 (socket says 172.21.15.42:42130) 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 -- 172.21.15.42:0/2587335306 learned_addr learned my addr 172.21.15.42:0/2587335306 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643e59c640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6440101390 0x7f6440073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643dd9b640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6440106460 0x7f64400739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 -- 172.21.15.42:0/2587335306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6440101390 msgr2=0x7f6440073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6440101390 0x7f6440073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 -- 172.21.15.42:0/2587335306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6440106460 msgr2=0x7f64400739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6440106460 0x7f64400739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643ed9d640 1 -- 172.21.15.42:0/2587335306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6438009560 con 0x7f6440073f00 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.253+0000 7f643dd9b640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6440106460 0x7f64400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.254+0000 7f643ed9d640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440073f00 0x7f64401a5960 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f643000b970 tx=0x7f643000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.254+0000 7f64377fe640 1 -- 172.21.15.42:0/2587335306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000cb40 con 0x7f6440073f00 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.254+0000 7f64377fe640 1 -- 172.21.15.42:0/2587335306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f643000cca0 con 0x7f6440073f00 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.254+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64401a6540 con 0x7f6440073f00 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.254+0000 7f64377fe640 1 -- 172.21.15.42:0/2587335306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300076e0 con 0x7f6440073f00 2024-03-12T04:23:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.254+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64401a6a00 con 0x7f6440073f00 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f64377fe640 1 -- 172.21.15.42:0/2587335306 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f6430014440 con 0x7f6440073f00 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 Processor -- start 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 -- start start 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9c250 0x559782c9c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9cb70 0x559782ca5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782ca7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782c57960 con 0x559782c9cb70 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782c0b310 con 0x559782c9c250 2024-03-12T04:23:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f6447a68740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782c08840 con 0x559782ca5750 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f643e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9cb70 0x559782ca5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f643e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9cb70 0x559782ca5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42146/0 (socket says 172.21.15.42:42146) 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f643e59c640 1 -- 172.21.15.42:0/4173884164 learned_addr learned my addr 172.21.15.42:0/4173884164 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f64377fe640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f642c0a1bd0 0x7f642c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.257+0000 7f643dd9b640 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9c250 0x559782c9c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643dd9b640 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782ca7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643ed9d640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f642c0a1bd0 0x7f642c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643e59c640 1 -- 172.21.15.42:0/4173884164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 msgr2=0x559782ca7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643e59c640 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782ca7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643e59c640 1 -- 172.21.15.42:0/4173884164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9c250 msgr2=0x559782c9c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f64377fe640 1 -- 172.21.15.42:0/2587335306 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f64300ccf70 con 0x7f6440073f00 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643e59c640 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9c250 0x559782c9c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643e59c640 1 -- 172.21.15.42:0/4173884164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559782bc6060 con 0x559782c9cb70 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643e59c640 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9cb70 0x559782ca5210 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6438036040 tx=0x7f64380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e6215a5799bb6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f643ed9d640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f642c0a1bd0 0x7f642c0a4090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64300cace0 tx=0x7f643006d030 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f6434ff9640 1 -- 172.21.15.42:0/4173884164 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64380048f0 con 0x559782c9cb70 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f6434ff9640 1 -- 172.21.15.42:0/4173884164 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f643802de90 con 0x559782c9cb70 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.258+0000 7f6434ff9640 1 -- 172.21.15.42:0/4173884164 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438003850 con 0x559782c9cb70 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 -- 172.21.15.42:0/4173884164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9cb70 msgr2=0x559782ca5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9cb70 0x559782ca5210 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6438036040 tx=0x7f64380046c0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 -- 172.21.15.42:0/4173884164 shutdown_connections 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782ca7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9cb70 0x559782ca5210 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 --2- 172.21.15.42:0/4173884164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9c250 0x559782c9c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 -- 172.21.15.42:0/4173884164 >> 172.21.15.42:0/4173884164 conn(0x559782c96980 msgr2=0x559782c96d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 -- 172.21.15.42:0/4173884164 shutdown_connections 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 -- 172.21.15.42:0/4173884164 wait complete. 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 -- start start 2024-03-12T04:23:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 Processor -- start 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.259+0000 7f6447a68740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9c250 0x559782ca2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782ca2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782ca5750 0x559782ca3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782c04020 con 0x559782ca5750 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597828d7430 con 0x559782c9cb70 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782893e00 con 0x559782c9c250 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 10 client.0 _ll_register_callbacks cb 0x559782babcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782ca5750 0x559782ca3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782ca5750 0x559782ca3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42160/0 (socket says 172.21.15.42:42160) 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 -- 172.21.15.42:0/2694511414 learned_addr learned my addr 172.21.15.42:0/2694511414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643e59c640 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782ca2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 -- 172.21.15.42:0/2694511414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9c250 msgr2=0x559782ca2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9c250 0x559782ca2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 -- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 msgr2=0x559782ca2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782ca2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 -- 172.21.15.42:0/2694511414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644010e840 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f643dd9b640 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782ca5750 0x559782ca3200 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6428014c50 tx=0x7f642800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff33522d87e72335 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6425ffb640 1 -- 172.21.15.42:0/2694511414 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428019070 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6425ffb640 1 -- 172.21.15.42:0/2694511414 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6428014e00 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597829037e0 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6425ffb640 1 -- 172.21.15.42:0/2694511414 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64280140d0 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 10 client.17895 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.260+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559782c026a0 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6425ffb640 1 -- 172.21.15.42:0/2694511414 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5597829037e0 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6425ffb640 1 -- 172.21.15.42:0/2694511414 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6428003a00 con 0x559782ca5750 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6447a68740 10 client.17895 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6447a68740 10 client.17895 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6447a68740 20 client.17895 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6447a68740 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782cb5b50 0x559782cb7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6447a68740 4 client.17895 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559782940720 con 0x559782cb5b50 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f643e59c640 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782cb5b50 0x559782cb7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f6425ffb640 10 client.17895 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.261+0000 7f643e59c640 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782cb5b50 0x559782cb7f40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25860898ef257104 server_cookie=55b0209c659ae13b in_seq=0 out_seq=0 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6425ffb640 1 -- 172.21.15.42:0/2694511414 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559782940720 con 0x559782cb5b50 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6425ffb640 10 client.17895 handle_command_reply: tid=0 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 client.17895 shutdown 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782ca5750 msgr2=0x559782ca3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782ca5750 0x559782ca3200 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6428014c50 tx=0x7f642800f790 comp rx=0 tx=0).stop 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 shutdown_connections 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782cb5b50 0x559782cb7f40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782ca5750 0x559782ca3200 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6425ffb640 0 client.17895 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782ca2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 --2- 172.21.15.42:0/2694511414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9c250 0x559782ca2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 >> 172.21.15.42:0/2694511414 conn(0x559782c96980 msgr2=0x559782c986d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 shutdown_connections 2024-03-12T04:23:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.262+0000 7f6447a68740 1 -- 172.21.15.42:0/2694511414 wait complete. 2024-03-12T04:23:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.263+0000 7f6447a68740 20 client.17895 trim_cache size 0 max 16384 2024-03-12T04:23:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 Processor -- start 2024-03-12T04:23:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 -- start start 2024-03-12T04:23:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782ca4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782c9f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782c9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782b89130 con 0x559782c9c250 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782cb2600 con 0x559782c9cb70 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f6447a68740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782ca2570 con 0x559782ca5750 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f643dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782ca4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f643dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782ca4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42168/0 (socket says 172.21.15.42:42168) 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.274+0000 7f643dd9b640 1 -- 172.21.15.42:0/3960877497 learned_addr learned my addr 172.21.15.42:0/3960877497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643e59c640 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782c9f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643dd9b640 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782c9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643dd9b640 1 -- 172.21.15.42:0/3960877497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 msgr2=0x559782c9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643dd9b640 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782c9ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643dd9b640 1 -- 172.21.15.42:0/3960877497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 msgr2=0x559782c9f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643dd9b640 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782c9f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643dd9b640 1 -- 172.21.15.42:0/3960877497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559782b9ec20 con 0x559782c9c250 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643e59c640 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782c9f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f643dd9b640 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782ca4eb0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6428014940 tx=0x7f6428009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51fff8bdb079b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f6427fff640 1 -- 172.21.15.42:0/3960877497 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428019070 con 0x559782c9c250 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f6427fff640 1 -- 172.21.15.42:0/3960877497 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f642801a040 con 0x559782c9c250 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f6427fff640 1 -- 172.21.15.42:0/3960877497 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280035f0 con 0x559782c9c250 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f6447a68740 1 -- 172.21.15.42:0/3960877497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 msgr2=0x559782ca4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.275+0000 7f6447a68740 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782ca4eb0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6428014940 tx=0x7f6428009b30 comp rx=0 tx=0).stop 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- 172.21.15.42:0/3960877497 shutdown_connections 2024-03-12T04:23:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782ca5750 0x559782c9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782c9cb70 0x559782c9f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 --2- 172.21.15.42:0/3960877497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782ca4eb0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- 172.21.15.42:0/3960877497 >> 172.21.15.42:0/3960877497 conn(0x559782c96980 msgr2=0x559782ca1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- 172.21.15.42:0/3960877497 shutdown_connections 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- 172.21.15.42:0/3960877497 wait complete. 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- start start 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 Processor -- start 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782cab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9cb70 0x559782cabb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782ca5750 0x559782d0c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782cac240 con 0x559782c9c250 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782d0c590 con 0x559782ca5750 2024-03-12T04:23:44.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559782d0c700 con 0x559782c9cb70 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f6447a68740 10 client.0 _ll_register_callbacks cb 0x7f6438009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f643dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782cab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f643dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782cab5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42170/0 (socket says 172.21.15.42:42170) 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f643dd9b640 1 -- 172.21.15.42:0/490652531 learned_addr learned my addr 172.21.15.42:0/490652531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f643dd9b640 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782ca5750 0x559782d0c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.276+0000 7f643dd9b640 1 -- 172.21.15.42:0/490652531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9cb70 msgr2=0x559782cabb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f643e59c640 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9cb70 0x559782cabb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f643dd9b640 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9cb70 0x559782cabb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f643dd9b640 1 -- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782ca5750 msgr2=0x559782d0c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f643dd9b640 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782ca5750 0x559782d0c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f643dd9b640 1 -- 172.21.15.42:0/490652531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559782c026a0 con 0x559782c9c250 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f643dd9b640 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782cab5e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f64280126f0 tx=0x7f6428009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72d44331cff9f6b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6425ffb640 1 -- 172.21.15.42:0/490652531 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428019040 con 0x559782c9c250 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6425ffb640 1 -- 172.21.15.42:0/490652531 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6428000e40 con 0x559782c9c250 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f643802de90 con 0x559782c9c250 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6425ffb640 1 -- 172.21.15.42:0/490652531 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6428005500 con 0x559782c9c250 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 10 client.17901 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559782bc48c0 con 0x559782c9c250 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6425ffb640 1 -- 172.21.15.42:0/490652531 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f643802de90 con 0x559782c9c250 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6425ffb640 1 -- 172.21.15.42:0/490652531 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6428030070 con 0x559782c9c250 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 10 client.17901 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 10 client.17901 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 20 client.17901 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782d0dab0 0x559782d0fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 4 client.17901 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:44.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.277+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55978298f890 con 0x559782d0dab0 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f643e59c640 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782d0dab0 0x559782d0fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f643e59c640 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782d0dab0 0x559782d0fea0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ed52148f9a45b61 server_cookie=3a42b254f889caa1 in_seq=0 out_seq=0 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6425ffb640 10 client.17901 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6425ffb640 1 -- 172.21.15.42:0/490652531 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55978298f890 con 0x559782d0dab0 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6425ffb640 10 client.17901 handle_command_reply: tid=0 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 client.17901 shutdown 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 msgr2=0x559782cab5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782cab5e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f64280126f0 tx=0x7f6428009530 comp rx=0 tx=0).stop 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 shutdown_connections 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x559782d0dab0 0x559782d0fea0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559782ca5750 0x559782d0c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6425ffb640 0 client.17901 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:23:44.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559782c9cb70 0x559782cabb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 --2- 172.21.15.42:0/490652531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559782c9c250 0x559782cab5e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 >> 172.21.15.42:0/490652531 conn(0x559782c96980 msgr2=0x559782c952f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 shutdown_connections 2024-03-12T04:23:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 1 -- 172.21.15.42:0/490652531 wait complete. 2024-03-12T04:23:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.278+0000 7f6447a68740 20 client.17901 trim_cache size 0 max 16384 2024-03-12T04:23:44.289 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f642c0a1bd0 msgr2=0x7f642c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f642c0a1bd0 0x7f642c0a4090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64300cace0 tx=0x7f643006d030 comp rx=0 tx=0).stop 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440073f00 msgr2=0x7f64401a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440073f00 0x7f64401a5960 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f643000b970 tx=0x7f643000be30 comp rx=0 tx=0).stop 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 shutdown_connections 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f642c0a1bd0 0x7f642c0a4090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6440073f00 0x7f64401a5960 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6440106460 0x7f64400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 --2- 172.21.15.42:0/2587335306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6440101390 0x7f6440073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 >> 172.21.15.42:0/2587335306 conn(0x7f64400fcbb0 msgr2=0x7f64400fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 shutdown_connections 2024-03-12T04:23:44.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.279+0000 7f6444e88640 1 -- 172.21.15.42:0/2587335306 wait complete. 2024-03-12T04:23:44.292 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:44.292 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 -- 172.21.15.42:0/2047211291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d90101d40 msgr2=0x7f1d90105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 --2- 172.21.15.42:0/2047211291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d90101d40 0x7f1d90105d80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009930 tx=0x7f1d8402bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 -- 172.21.15.42:0/2047211291 shutdown_connections 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 --2- 172.21.15.42:0/2047211291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d90106460 0x7f1d9010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 --2- 172.21.15.42:0/2047211291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d90101d40 0x7f1d90105d80 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 --2- 172.21.15.42:0/2047211291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d90101390 0x7f1d90101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 -- 172.21.15.42:0/2047211291 >> 172.21.15.42:0/2047211291 conn(0x7f1d900fcbb0 msgr2=0x7f1d900fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 -- 172.21.15.42:0/2047211291 shutdown_connections 2024-03-12T04:23:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.395+0000 7f1d9690f640 1 -- 172.21.15.42:0/2047211291 wait complete. 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 Processor -- start 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 -- start start 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d90101390 0x7f1d901976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d90106460 0x7f1d90197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d9019bf90 0x7f1d9019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9019c980 con 0x7f1d9019bf90 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9019caf0 con 0x7f1d90106460 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d9690f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9019cc60 con 0x7f1d90101390 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d9019bf90 0x7f1d9019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d9019bf90 0x7f1d9019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42198/0 (socket says 172.21.15.42:42198) 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 -- 172.21.15.42:0/1725041610 learned_addr learned my addr 172.21.15.42:0/1725041610 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d90106460 0x7f1d90197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d90101390 0x7f1d901976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 -- 172.21.15.42:0/1725041610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d90101390 msgr2=0x7f1d901976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d90101390 0x7f1d901976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 -- 172.21.15.42:0/1725041610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d90106460 msgr2=0x7f1d90197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d90106460 0x7f1d90197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.396+0000 7f1d94eac640 1 -- 172.21.15.42:0/1725041610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009560 con 0x7f1d9019bf90 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.397+0000 7f1d94eac640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d9019bf90 0x7f1d9019c440 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800cce0 tx=0x7f1d8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.397+0000 7f1d8d7fa640 1 -- 172.21.15.42:0/1725041610 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800f450 con 0x7f1d9019bf90 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.397+0000 7f1d8d7fa640 1 -- 172.21.15.42:0/1725041610 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1d8800fa60 con 0x7f1d9019bf90 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.397+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d901a4240 con 0x7f1d9019bf90 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.397+0000 7f1d8d7fa640 1 -- 172.21.15.42:0/1725041610 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88018b10 con 0x7f1d9019bf90 2024-03-12T04:23:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.397+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d901a4670 con 0x7f1d9019bf90 2024-03-12T04:23:44.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.399+0000 7f1d8d7fa640 1 -- 172.21.15.42:0/1725041610 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f1d8800fbd0 con 0x7f1d9019bf90 2024-03-12T04:23:44.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.399+0000 7f1d995b6740 1 Processor -- start 2024-03-12T04:23:44.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.399+0000 7f1d995b6740 1 -- start start 2024-03-12T04:23:44.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8d7fa640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1d780a1ba0 0x7f1d780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a16620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d995b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f469c4960 con 0x555f46a08f00 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d995b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46978310 con 0x555f46a16b60 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d995b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46975840 con 0x555f46a09820 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1d780a1ba0 0x7f1d780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8d7fa640 1 -- 172.21.15.42:0/1725041610 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1d880cdbb0 con 0x7f1d9019bf90 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42204/0 (socket says 172.21.15.42:42204) 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2179039525 learned_addr learned my addr 172.21.15.42:0/2179039525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d94eac640 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2179039525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 msgr2=0x555f46a16620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a16620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2179039525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 msgr2=0x555f46a18f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a18f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2179039525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f46933060 con 0x555f46a08f00 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1d780a1ba0 0x7f1d780a4060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d84000c00 tx=0x7f1d84005d50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a092e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000cb70 tx=0x7f1d8000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655ad9ca49b70299 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d767fc640 1 -- 172.21.15.42:0/2179039525 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010630 con 0x555f46a08f00 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.400+0000 7f1d767fc640 1 -- 172.21.15.42:0/2179039525 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1d8000ce70 con 0x555f46a08f00 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d767fc640 1 -- 172.21.15.42:0/2179039525 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80019a50 con 0x555f46a08f00 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 -- 172.21.15.42:0/2179039525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 msgr2=0x555f46a092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a092e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000cb70 tx=0x7f1d8000e720 comp rx=0 tx=0).stop 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 -- 172.21.15.42:0/2179039525 shutdown_connections 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a18f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a16620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2179039525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a092e0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 -- 172.21.15.42:0/2179039525 >> 172.21.15.42:0/2179039525 conn(0x555f46a03980 msgr2=0x555f46a03d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 -- 172.21.15.42:0/2179039525 shutdown_connections 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 -- 172.21.15.42:0/2179039525 wait complete. 2024-03-12T04:23:44.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.401+0000 7f1d995b6740 1 -- start start 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 1 Processor -- start 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a13b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a14040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46971020 con 0x555f46a08f00 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46644430 con 0x555f46a16b60 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46600e00 con 0x555f46a09820 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d995b6740 10 client.0 _ll_register_callbacks cb 0x555f46918cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a13b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a13b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42214/0 (socket says 172.21.15.42:42214) 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 -- 172.21.15.42:0/3929262067 learned_addr learned my addr 172.21.15.42:0/3929262067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a14040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d94eac640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 -- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 msgr2=0x555f46a14040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a14040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 -- 172.21.15.42:0/3929262067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 msgr2=0x555f46a14580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a14580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8ffff640 1 -- 172.21.15.42:0/3929262067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8000c850 con 0x555f46a08f00 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.402+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a14040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a13b00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005710 tx=0x7f1d80005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6e99e386ba513d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/3929262067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80020070 con 0x555f46a08f00 2024-03-12T04:23:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/3929262067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1d80019a20 con 0x555f46a08f00 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f466707e0 con 0x555f46a08f00 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/3929262067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d80019be0 con 0x555f46a08f00 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 10 client.17913 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f4696f6a0 con 0x555f46a08f00 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/3929262067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x555f466707e0 con 0x555f46a08f00 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/3929262067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d80010550 con 0x555f46a08f00 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 10 client.17913 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 10 client.17913 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 20 client.17913 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a26f90 0x555f46a293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 4 client.17913 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.403+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f466ad720 con 0x555f46a26f90 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.404+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a26f90 0x555f46a293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.404+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a26f90 0x555f46a293a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7490429b8402fdf server_cookie=ccf56c3ed558e9f0 in_seq=0 out_seq=0 2024-03-12T04:23:44.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.404+0000 7f1d6f7fe640 10 client.17913 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/3929262067 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f466ad720 con 0x555f46a26f90 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d6f7fe640 10 client.17913 handle_command_reply: tid=0 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 client.17913 shutdown 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 msgr2=0x555f46a13b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a13b00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005710 tx=0x7f1d80005740 comp rx=0 tx=0).stop 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 shutdown_connections 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a26f90 0x555f46a293a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a14040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d6f7fe640 0 client.17913 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 --2- 172.21.15.42:0/3929262067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a13b00 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 >> 172.21.15.42:0/3929262067 conn(0x555f46a03980 msgr2=0x555f46a056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 shutdown_connections 2024-03-12T04:23:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 1 -- 172.21.15.42:0/3929262067 wait complete. 2024-03-12T04:23:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.405+0000 7f1d995b6740 20 client.17913 trim_cache size 0 max 16384 2024-03-12T04:23:44.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 Processor -- start 2024-03-12T04:23:44.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 -- start start 2024-03-12T04:23:44.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a16230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a0e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f468f6130 con 0x555f46a08f00 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46a03810 con 0x555f46a16b60 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d995b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46a120a0 con 0x555f46a09820 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a16230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a16230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42228/0 (socket says 172.21.15.42:42228) 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2269875986 learned_addr learned my addr 172.21.15.42:0/2269875986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a0e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2269875986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 msgr2=0x555f46a0e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a0e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2269875986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 msgr2=0x555f46a143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d94eac640 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d8ffff640 1 -- 172.21.15.42:0/2269875986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f4690bc20 con 0x555f46a08f00 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.417+0000 7f1d94eac640 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a16230 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1d80000c00 tx=0x7f1d8000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee976f8675ffbe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d75ffb640 1 -- 172.21.15.42:0/2269875986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80020040 con 0x555f46a08f00 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d75ffb640 1 -- 172.21.15.42:0/2269875986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1d80019590 con 0x555f46a08f00 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d75ffb640 1 -- 172.21.15.42:0/2269875986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010700 con 0x555f46a08f00 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 -- 172.21.15.42:0/2269875986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 msgr2=0x555f46a16230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a16230 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1d80000c00 tx=0x7f1d8000e720 comp rx=0 tx=0).stop 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 -- 172.21.15.42:0/2269875986 shutdown_connections 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a16b60 0x555f46a143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a09820 0x555f46a0e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 --2- 172.21.15.42:0/2269875986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a08f00 0x555f46a16230 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 -- 172.21.15.42:0/2269875986 >> 172.21.15.42:0/2269875986 conn(0x555f46a03980 msgr2=0x555f46a03d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 -- 172.21.15.42:0/2269875986 shutdown_connections 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.418+0000 7f1d995b6740 1 -- 172.21.15.42:0/2269875986 wait complete. 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 -- start start 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 Processor -- start 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a08f00 0x555f46a1df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a09820 0x555f46a1c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a16b60 0x555f46a1c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46a1cff0 con 0x555f46a09820 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46a1d160 con 0x555f46a08f00 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f46a7d500 con 0x555f46a16b60 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d995b6740 10 client.0 _ll_register_callbacks cb 0x7f1d8000c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d8f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a09820 0x555f46a1c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d8f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a09820 0x555f46a1c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42238/0 (socket says 172.21.15.42:42238) 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d8f7fe640 1 -- 172.21.15.42:0/1007178616 learned_addr learned my addr 172.21.15.42:0/1007178616 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a08f00 0x555f46a1df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.419+0000 7f1d94eac640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a16b60 0x555f46a1c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d8f7fe640 1 -- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a16b60 msgr2=0x555f46a1c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a16b60 0x555f46a1c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d8f7fe640 1 -- 172.21.15.42:0/1007178616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a08f00 msgr2=0x555f46a1df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a08f00 0x555f46a1df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d8f7fe640 1 -- 172.21.15.42:0/1007178616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f4696f6a0 con 0x555f46a09820 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d94eac640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a16b60 0x555f46a1c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a08f00 0x555f46a1df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d8f7fe640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a09820 0x555f46a1c2f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009930 tx=0x7f1d84009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce59a65a060d882b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/1007178616 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840532f0 con 0x555f46a09820 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/1007178616 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1d84053450 con 0x555f46a09820 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d8000ce70 con 0x555f46a09820 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/1007178616 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d84053610 con 0x555f46a09820 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 10 client.17919 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f469318c0 con 0x555f46a09820 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/1007178616 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1d8000ce70 con 0x555f46a09820 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/1007178616 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d8403c030 con 0x555f46a09820 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 10 client.17919 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 10 client.17919 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 20 client.17919 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a7ecd0 0x555f46a810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 4 client.17919 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.420+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555f466fc890 con 0x555f46a7ecd0 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a7ecd0 0x555f46a810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d6f7fe640 10 client.17919 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d8ffff640 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a7ecd0 0x555f46a810f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=160e4d3831a6941c server_cookie=e040a27765d5938f in_seq=0 out_seq=0 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d6f7fe640 1 -- 172.21.15.42:0/1007178616 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555f466fc890 con 0x555f46a7ecd0 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d6f7fe640 10 client.17919 handle_command_reply: tid=0 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 client.17919 shutdown 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a09820 msgr2=0x555f46a1c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a09820 0x555f46a1c2f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009930 tx=0x7f1d84009260 comp rx=0 tx=0).stop 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 shutdown_connections 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x555f46a7ecd0 0x555f46a810f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f46a16b60 0x555f46a1c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f46a09820 0x555f46a1c2f0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d6f7fe640 0 client.17919 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:23:44.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 --2- 172.21.15.42:0/1007178616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f46a08f00 0x555f46a1df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 >> 172.21.15.42:0/1007178616 conn(0x555f46a03980 msgr2=0x555f46a2dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.421+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 shutdown_connections 2024-03-12T04:23:44.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d995b6740 1 -- 172.21.15.42:0/1007178616 wait complete. 2024-03-12T04:23:44.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d995b6740 20 client.17919 trim_cache size 0 max 16384 2024-03-12T04:23:44.432 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:44.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1d780a1ba0 msgr2=0x7f1d780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1d780a1ba0 0x7f1d780a4060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d84000c00 tx=0x7f1d84005d50 comp rx=0 tx=0).stop 2024-03-12T04:23:44.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d9019bf90 msgr2=0x7f1d9019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d9019bf90 0x7f1d9019c440 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800cce0 tx=0x7f1d8800a620 comp rx=0 tx=0).stop 2024-03-12T04:23:44.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 shutdown_connections 2024-03-12T04:23:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f1d780a1ba0 0x7f1d780a4060 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d9019bf90 0x7f1d9019c440 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d90106460 0x7f1d90197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 --2- 172.21.15.42:0/1725041610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d90101390 0x7f1d901976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 >> 172.21.15.42:0/1725041610 conn(0x7f1d900fcbb0 msgr2=0x7f1d900fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 shutdown_connections 2024-03-12T04:23:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.422+0000 7f1d9690f640 1 -- 172.21.15.42:0/1725041610 wait complete. 2024-03-12T04:23:44.434 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:44 smithi042 ceph-mon[32260]: pgmap v263: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:44.435 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:44 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2111285021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:23:44.435 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:23:44.436 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:23:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.537+0000 7f87e4f3e640 1 -- 172.21.15.42:0/1703802426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0100260 msgr2=0x7f87e0100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.537+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/1703802426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0100260 0x7f87e0100640 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009930 tx=0x7f87d802bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.537+0000 7f87e4f3e640 1 -- 172.21.15.42:0/1703802426 shutdown_connections 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.537+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/1703802426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e0105330 0x7f87e010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.537+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/1703802426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e0100c10 0x7f87e0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.537+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/1703802426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0100260 0x7f87e0100640 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.537+0000 7f87e4f3e640 1 -- 172.21.15.42:0/1703802426 >> 172.21.15.42:0/1703802426 conn(0x7f87e0077180 msgr2=0x7f87e0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 -- 172.21.15.42:0/1703802426 shutdown_connections 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 -- 172.21.15.42:0/1703802426 wait complete. 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 Processor -- start 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 -- start start 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e0100260 0x7f87e0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e0105330 0x7f87e0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0199060 0x7f87e01a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01a3cb0 con 0x7f87e0199060 2024-03-12T04:23:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01a3e20 con 0x7f87e0100260 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.538+0000 7f87e4f3e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01a3fe0 con 0x7f87e0105330 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0199060 0x7f87e01a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0199060 0x7f87e01a3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42258/0 (socket says 172.21.15.42:42258) 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 -- 172.21.15.42:0/3694379322 learned_addr learned my addr 172.21.15.42:0/3694379322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87de59c640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e0100260 0x7f87e0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e0105330 0x7f87e0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 -- 172.21.15.42:0/3694379322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e0105330 msgr2=0x7f87e0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e0105330 0x7f87e0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 -- 172.21.15.42:0/3694379322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e0100260 msgr2=0x7f87e0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e0100260 0x7f87e0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 -- 172.21.15.42:0/3694379322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009560 con 0x7f87e0199060 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e0105330 0x7f87e0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87ded9d640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0199060 0x7f87e01a3610 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f87d000cad0 tx=0x7f87d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87cf7fe640 1 -- 172.21.15.42:0/3694379322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000ae20 con 0x7f87e0199060 2024-03-12T04:23:44.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e01a42c0 con 0x7f87e0199060 2024-03-12T04:23:44.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87cf7fe640 1 -- 172.21.15.42:0/3694379322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87d0005020 con 0x7f87e0199060 2024-03-12T04:23:44.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87cf7fe640 1 -- 172.21.15.42:0/3694379322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000f690 con 0x7f87e0199060 2024-03-12T04:23:44.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.539+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01a4820 con 0x7f87e0199060 2024-03-12T04:23:44.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.541+0000 7f87cf7fe640 1 -- 172.21.15.42:0/3694379322 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f87d0005190 con 0x7f87e0199060 2024-03-12T04:23:44.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 Processor -- start 2024-03-12T04:23:44.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 -- start start 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87cf7fe640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f87c80a1ba0 0x7f87c80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b39f80 0x55ad21b3a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b47620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b47b60 0x55ad21b49f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21af5960 con 0x55ad21b47b60 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21aa9310 con 0x55ad21b3a8a0 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87e7beb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21aa6840 con 0x55ad21b39f80 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f87c80a1ba0 0x7f87c80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b47b60 0x55ad21b49f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b47b60 0x55ad21b49f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42272/0 (socket says 172.21.15.42:42272) 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87ded9d640 1 -- 172.21.15.42:0/2651060409 learned_addr learned my addr 172.21.15.42:0/2651060409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.542+0000 7f87cf7fe640 1 -- 172.21.15.42:0/3694379322 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f87d00cd850 con 0x7f87e0199060 2024-03-12T04:23:44.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87de59c640 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b39f80 0x55ad21b3a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b47620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ded9d640 1 -- 172.21.15.42:0/2651060409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b39f80 msgr2=0x55ad21b3a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ded9d640 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b39f80 0x55ad21b3a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ded9d640 1 -- 172.21.15.42:0/2651060409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 msgr2=0x55ad21b47620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ded9d640 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b47620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ded9d640 1 -- 172.21.15.42:0/2651060409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad21a64060 con 0x55ad21b47b60 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f87c80a1ba0 0x7f87c80a4060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005d00 tx=0x7f87d4012760 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ded9d640 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b47b60 0x55ad21b49f50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f87d0096900 tx=0x7f87d00504b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9120629305d017e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ccff9640 1 -- 172.21.15.42:0/2651060409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0077ee0 con 0x55ad21b47b60 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ccff9640 1 -- 172.21.15.42:0/2651060409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87d0077d00 con 0x55ad21b47b60 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.543+0000 7f87ccff9640 1 -- 172.21.15.42:0/2651060409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d009a4a0 con 0x55ad21b47b60 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- 172.21.15.42:0/2651060409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b47b60 msgr2=0x55ad21b49f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b47b60 0x55ad21b49f50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f87d0096900 tx=0x7f87d00504b0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- 172.21.15.42:0/2651060409 shutdown_connections 2024-03-12T04:23:44.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b47b60 0x55ad21b49f50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b47620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 --2- 172.21.15.42:0/2651060409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b39f80 0x55ad21b3a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- 172.21.15.42:0/2651060409 >> 172.21.15.42:0/2651060409 conn(0x55ad21b34980 msgr2=0x55ad21b34d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- 172.21.15.42:0/2651060409 shutdown_connections 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- 172.21.15.42:0/2651060409 wait complete. 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- start start 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 Processor -- start 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21b44c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b45150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b45690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21aa2020 con 0x55ad21b39f80 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21775430 con 0x55ad21b3a8a0 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.544+0000 7f87e7beb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21731e00 con 0x55ad21b47b60 2024-03-12T04:23:44.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87e7beb740 10 client.0 _ll_register_callbacks cb 0x55ad21a49cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21b44c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21b44c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42286/0 (socket says 172.21.15.42:42286) 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 -- 172.21.15.42:0/358332850 learned_addr learned my addr 172.21.15.42:0/358332850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b45150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 -- 172.21.15.42:0/358332850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 msgr2=0x55ad21b45690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b45690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 -- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 msgr2=0x55ad21b45150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b45150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 -- 172.21.15.42:0/358332850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01a4820 con 0x55ad21b39f80 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b45150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87de59c640 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21b44c10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f87d8036040 tx=0x7f87d802ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41097dd72824ae75 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87c5ffb640 1 -- 172.21.15.42:0/358332850 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8004450 con 0x55ad21b39f80 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87c5ffb640 1 -- 172.21.15.42:0/358332850 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f87d80035f0 con 0x55ad21b39f80 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad217a17e0 con 0x55ad21b39f80 2024-03-12T04:23:44.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87c5ffb640 1 -- 172.21.15.42:0/358332850 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d80037b0 con 0x55ad21b39f80 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87e7beb740 10 client.17931 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.545+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ad21aa06a0 con 0x55ad21b39f80 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87c5ffb640 1 -- 172.21.15.42:0/358332850 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ad217a17e0 con 0x55ad21b39f80 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87c5ffb640 1 -- 172.21.15.42:0/358332850 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f87d80052e0 con 0x55ad21b39f80 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87e7beb740 10 client.17931 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87e7beb740 10 client.17931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87e7beb740 20 client.17931 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87e7beb740 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21b57ff0 0x55ad21b5a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87e7beb740 4 client.17931 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad217de720 con 0x55ad21b57ff0 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21b57ff0 0x55ad21b5a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87c5ffb640 10 client.17931 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.546+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21b57ff0 0x55ad21b5a430 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608ab339e1cfe664 server_cookie=5d12615c4f23e04e in_seq=0 out_seq=0 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87c5ffb640 1 -- 172.21.15.42:0/358332850 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad217de720 con 0x55ad21b57ff0 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87c5ffb640 10 client.17931 handle_command_reply: tid=0 2024-03-12T04:23:44.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 client.17931 shutdown 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 msgr2=0x55ad21b44c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21b44c10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f87d8036040 tx=0x7f87d802ca20 comp rx=0 tx=0).stop 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 shutdown_connections 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21b57ff0 0x55ad21b5a430 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b45690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87c5ffb640 0 client.17931 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21b45150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 --2- 172.21.15.42:0/358332850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21b44c10 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 >> 172.21.15.42:0/358332850 conn(0x55ad21b34980 msgr2=0x55ad21b366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 shutdown_connections 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 1 -- 172.21.15.42:0/358332850 wait complete. 2024-03-12T04:23:44.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.547+0000 7f87e7beb740 20 client.17931 trim_cache size 0 max 16384 2024-03-12T04:23:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 Processor -- start 2024-03-12T04:23:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 -- start start 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b39f80 0x55ad21b47340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b3a8a0 0x55ad21b454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b41e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21a27130 con 0x55ad21b3a8a0 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21b39390 con 0x55ad21b39f80 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87e7beb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21b305c0 con 0x55ad21b47b60 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b3a8a0 0x55ad21b454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b3a8a0 0x55ad21b454b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42300/0 (socket says 172.21.15.42:42300) 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 -- 172.21.15.42:0/1668308351 learned_addr learned my addr 172.21.15.42:0/1668308351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ded9d640 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b41e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87de59c640 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b39f80 0x55ad21b47340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 -- 172.21.15.42:0/1668308351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 msgr2=0x55ad21b41e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b41e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 -- 172.21.15.42:0/1668308351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b39f80 msgr2=0x55ad21b47340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b39f80 0x55ad21b47340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.559+0000 7f87ddd9b640 1 -- 172.21.15.42:0/1668308351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad21a3cc20 con 0x55ad21b3a8a0 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87de59c640 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b39f80 0x55ad21b47340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b3a8a0 0x55ad21b454b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f87e006a8e0 tx=0x7f87d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a188a1939c2fe205 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87c7fff640 1 -- 172.21.15.42:0/1668308351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4026070 con 0x55ad21b3a8a0 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87c7fff640 1 -- 172.21.15.42:0/1668308351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87d4013df0 con 0x55ad21b3a8a0 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87c7fff640 1 -- 172.21.15.42:0/1668308351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d40261d0 con 0x55ad21b3a8a0 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 -- 172.21.15.42:0/1668308351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b3a8a0 msgr2=0x55ad21b454b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b3a8a0 0x55ad21b454b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f87e006a8e0 tx=0x7f87d4015000 comp rx=0 tx=0).stop 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 -- 172.21.15.42:0/1668308351 shutdown_connections 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b41e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b3a8a0 0x55ad21b454b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 --2- 172.21.15.42:0/1668308351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b39f80 0x55ad21b47340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 -- 172.21.15.42:0/1668308351 >> 172.21.15.42:0/1668308351 conn(0x55ad21b34980 msgr2=0x55ad21b3ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 -- 172.21.15.42:0/1668308351 shutdown_connections 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.560+0000 7f87e7beb740 1 -- 172.21.15.42:0/1668308351 wait complete. 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 -- start start 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 Processor -- start 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21ba8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21ba8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b4dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21b4e160 con 0x55ad21b39f80 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21b4e2d0 con 0x55ad21b3a8a0 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21b4e490 con 0x55ad21b47b60 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87e7beb740 10 client.0 _ll_register_callbacks cb 0x7f87d0007780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:44.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21ba8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21ba8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42304/0 (socket says 172.21.15.42:42304) 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 -- 172.21.15.42:0/644230013 learned_addr learned my addr 172.21.15.42:0/644230013 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87ded9d640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b4dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 -- 172.21.15.42:0/644230013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 msgr2=0x55ad21b4dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21ba8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b4dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 -- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 msgr2=0x55ad21ba8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21ba8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87de59c640 1 -- 172.21.15.42:0/644230013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d4009840 con 0x55ad21b39f80 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.561+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21ba8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87de59c640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21ba8900 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009850 tx=0x7f87d802ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e8aaa45c96f944 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87c5ffb640 1 -- 172.21.15.42:0/644230013 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8005d50 con 0x55ad21b39f80 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87c5ffb640 1 -- 172.21.15.42:0/644230013 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f87d8000e40 con 0x55ad21b39f80 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad21b35300 con 0x55ad21b39f80 2024-03-12T04:23:44.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87c5ffb640 1 -- 172.21.15.42:0/644230013 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d8004a60 con 0x55ad21b39f80 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 10 client.17937 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ad21a628c0 con 0x55ad21b39f80 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87c5ffb640 1 -- 172.21.15.42:0/644230013 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ad21b35300 con 0x55ad21b39f80 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87c5ffb640 1 -- 172.21.15.42:0/644230013 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f87d80051a0 con 0x55ad21b39f80 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 10 client.17937 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 20 client.17937 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21bb03e0 0x55ad21bb2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 4 client.17937 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.562+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f87d0077d00 con 0x55ad21bb03e0 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21bb03e0 0x55ad21bb2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87c5ffb640 10 client.17937 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87ddd9b640 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21bb03e0 0x55ad21bb2800 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6bd880fef6ce0f9 server_cookie=8ee0de12dea64898 in_seq=0 out_seq=0 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87c5ffb640 1 -- 172.21.15.42:0/644230013 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f87d0077d00 con 0x55ad21bb03e0 2024-03-12T04:23:44.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87c5ffb640 10 client.17937 handle_command_reply: tid=0 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 client.17937 shutdown 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 msgr2=0x55ad21ba8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21ba8900 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009850 tx=0x7f87d802ccc0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 shutdown_connections 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55ad21bb03e0 0x55ad21bb2800 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad21b47b60 0x55ad21b4dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad21b3a8a0 0x55ad21ba8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 --2- 172.21.15.42:0/644230013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ad21b39f80 0x55ad21ba8900 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87c5ffb640 0 client.17937 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 >> 172.21.15.42:0/644230013 conn(0x55ad21b34980 msgr2=0x55ad21b332f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 shutdown_connections 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 1 -- 172.21.15.42:0/644230013 wait complete. 2024-03-12T04:23:44.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.563+0000 7f87e7beb740 20 client.17937 trim_cache size 0 max 16384 2024-03-12T04:23:44.574 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:23:44.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f87c80a1ba0 msgr2=0x7f87c80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f87c80a1ba0 0x7f87c80a4060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005d00 tx=0x7f87d4012760 comp rx=0 tx=0).stop 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0199060 msgr2=0x7f87e01a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0199060 0x7f87e01a3610 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f87d000cad0 tx=0x7f87d000cf90 comp rx=0 tx=0).stop 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 shutdown_connections 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f87c80a1ba0 0x7f87c80a4060 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87e0199060 0x7f87e01a3610 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e0105330 0x7f87e0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 --2- 172.21.15.42:0/3694379322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e0100260 0x7f87e0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 >> 172.21.15.42:0/3694379322 conn(0x7f87e0077180 msgr2=0x7f87e00fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 shutdown_connections 2024-03-12T04:23:44.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:44.564+0000 7f87e4f3e640 1 -- 172.21.15.42:0/3694379322 wait complete. 2024-03-12T04:23:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:44 smithi119 ceph-mon[36119]: pgmap v263: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:44 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2111285021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:23:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:44 smithi136 ceph-mon[35571]: pgmap v263: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:23:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:44 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2111285021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:23:45.578 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:23:45.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:23:45.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:23:45.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 -- 172.21.15.42:0/3668315243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74101d30 msgr2=0x7f4b74105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3668315243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74101d30 0x7f4b74105d70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009930 tx=0x7f4b5c02bc50 comp rx=0 tx=0).stop 2024-03-12T04:23:45.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 -- 172.21.15.42:0/3668315243 shutdown_connections 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3668315243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74106450 0x7f4b7410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3668315243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74101d30 0x7f4b74105d70 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3668315243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b74101380 0x7f4b74101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 -- 172.21.15.42:0/3668315243 >> 172.21.15.42:0/3668315243 conn(0x7f4b74078580 msgr2=0x7f4b74078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 -- 172.21.15.42:0/3668315243 shutdown_connections 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.694+0000 7f4b791ac640 1 -- 172.21.15.42:0/3668315243 wait complete. 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 Processor -- start 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 -- start start 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b74101380 0x7f4b74073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74106450 0x7f4b740739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74073f00 0x7f4b741a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a5f40 con 0x7f4b74073f00 2024-03-12T04:23:45.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a60b0 con 0x7f4b74106450 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b791ac640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a6270 con 0x7f4b74101380 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b7359e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74073f00 0x7f4b741a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b7359e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74073f00 0x7f4b741a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42332/0 (socket says 172.21.15.42:42332) 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b7359e640 1 -- 172.21.15.42:0/3792212984 learned_addr learned my addr 172.21.15.42:0/3792212984 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.695+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74106450 0x7f4b740739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b7359e640 1 -- 172.21.15.42:0/3792212984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b74101380 msgr2=0x7f4b74073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b74101380 0x7f4b74073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b7359e640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b74101380 0x7f4b74073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b7359e640 1 -- 172.21.15.42:0/3792212984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74106450 msgr2=0x7f4b740739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b7359e640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74106450 0x7f4b740739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b7359e640 1 -- 172.21.15.42:0/3792212984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c009560 con 0x7f4b74073f00 2024-03-12T04:23:45.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74106450 0x7f4b740739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:23:45.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b7359e640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74073f00 0x7f4b741a5970 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00eaa0 tx=0x7f4b6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b70d99640 1 -- 172.21.15.42:0/3792212984 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00cdb0 con 0x7f4b74073f00 2024-03-12T04:23:45.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b70d99640 1 -- 172.21.15.42:0/3792212984 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4b6c005020 con 0x7f4b74073f00 2024-03-12T04:23:45.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b741a6550 con 0x7f4b74073f00 2024-03-12T04:23:45.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b70d99640 1 -- 172.21.15.42:0/3792212984 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0116b0 con 0x7f4b74073f00 2024-03-12T04:23:45.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.696+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b741a69e0 con 0x7f4b74073f00 2024-03-12T04:23:45.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b70d99640 1 -- 172.21.15.42:0/3792212984 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f4b6c005190 con 0x7f4b74073f00 2024-03-12T04:23:45.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 Processor -- start 2024-03-12T04:23:45.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 -- start start 2024-03-12T04:23:45.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd501e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd50272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd50278b0 0x55bdd5029d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd4f8cf90 con 0x55bdd50278b0 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd4f8a4c0 con 0x55bdd501ed30 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd4f85ca0 con 0x55bdd501e380 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.699+0000 7f4b70d99640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4b600a1bd0 0x7f4b600a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd50278b0 0x55bdd5029d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b7359e640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4b600a1bd0 0x7f4b600a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd50278b0 0x55bdd5029d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42342/0 (socket says 172.21.15.42:42342) 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3353151941 learned_addr learned my addr 172.21.15.42:0/3353151941 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b70d99640 1 -- 172.21.15.42:0/3792212984 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4b6c0cd9d0 con 0x7f4b74073f00 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd50272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd501e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3353151941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 msgr2=0x55bdd501e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd501e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3353151941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 msgr2=0x55bdd50272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd50272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3353151941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd4f480b0 con 0x55bdd50278b0 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b7359e640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4b600a1bd0 0x7f4b600a4090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0a2890 tx=0x7f4b6c08fcf0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd50278b0 0x55bdd5029d00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009930 tx=0x7f4b5c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630849ef5e3c59e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b697fa640 1 -- 172.21.15.42:0/3353151941 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0040f0 con 0x55bdd50278b0 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b697fa640 1 -- 172.21.15.42:0/3353151941 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4b5c004250 con 0x55bdd50278b0 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.700+0000 7f4b697fa640 1 -- 172.21.15.42:0/3353151941 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c005020 con 0x55bdd50278b0 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3353151941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd50278b0 msgr2=0x55bdd5029d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd50278b0 0x55bdd5029d00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009930 tx=0x7f4b5c005fd0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3353151941 shutdown_connections 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd50278b0 0x55bdd5029d00 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd50272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3353151941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd501e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3353151941 >> 172.21.15.42:0/3353151941 conn(0x55bdd5018840 msgr2=0x55bdd5018c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3353151941 shutdown_connections 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.701+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3353151941 wait complete. 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 -- start start 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 Processor -- start 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd5024800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501ed30 0x55bdd5024d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd50278b0 0x55bdd5025280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd4f2af40 con 0x55bdd501ed30 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd4c594b0 con 0x55bdd50278b0 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd4f0b670 con 0x55bdd501e380 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b7bd7f740 10 client.0 _ll_register_callbacks cb 0x55bdd4f78f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501ed30 0x55bdd5024d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501ed30 0x55bdd5024d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42350/0 (socket says 172.21.15.42:42350) 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 -- 172.21.15.42:0/2078088028 learned_addr learned my addr 172.21.15.42:0/2078088028 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd50278b0 0x55bdd5025280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 -- 172.21.15.42:0/2078088028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 msgr2=0x55bdd5024800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd5024800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 -- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd50278b0 msgr2=0x55bdd5025280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd50278b0 0x55bdd5025280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.702+0000 7f4b72d9d640 1 -- 172.21.15.42:0/2078088028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7410e830 con 0x55bdd501ed30 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501ed30 0x55bdd5024d40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400f4f0 tx=0x7f4b6400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb640ad43c7960de server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/2078088028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400a520 con 0x55bdd501ed30 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/2078088028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4b64013850 con 0x55bdd501ed30 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdd4f5ff40 con 0x55bdd501ed30 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/2078088028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b64013a30 con 0x55bdd501ed30 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b7bd7f740 10 client.17949 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bdd4f84320 con 0x55bdd501ed30 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.703+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/2078088028 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bdd4f5ff40 con 0x55bdd501ed30 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/2078088028 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b64019020 con 0x55bdd501ed30 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b7bd7f740 10 client.17949 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b7bd7f740 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b7bd7f740 20 client.17949 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd5037e80 0x55bdd503a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b7bd7f740 4 client.17949 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdd4f7c310 con 0x55bdd5037e80 2024-03-12T04:23:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd5037e80 0x55bdd503a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b5a7fc640 10 client.17949 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.704+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd5037e80 0x55bdd503a2a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1615215a322d4bf7 server_cookie=4fde8c0f7801c381 in_seq=0 out_seq=0 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/2078088028 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdd4f7c310 con 0x55bdd5037e80 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b5a7fc640 10 client.17949 handle_command_reply: tid=0 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 client.17949 shutdown 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501ed30 msgr2=0x55bdd5024d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501ed30 0x55bdd5024d40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400f4f0 tx=0x7f4b6400f9b0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 shutdown_connections 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd5037e80 0x55bdd503a2a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd50278b0 0x55bdd5025280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501ed30 0x55bdd5024d40 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b5a7fc640 0 client.17949 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/2078088028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd5024800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 >> 172.21.15.42:0/2078088028 conn(0x55bdd5018840 msgr2=0x55bdd501a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 shutdown_connections 2024-03-12T04:23:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/2078088028 wait complete. 2024-03-12T04:23:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.705+0000 7f4b7bd7f740 20 client.17949 trim_cache size 0 max 16384 2024-03-12T04:23:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 Processor -- start 2024-03-12T04:23:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 -- start start 2024-03-12T04:23:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd5018840 0x55bdd50250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501e380 0x55bdd502e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd502ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd5019640 con 0x55bdd501e380 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd50348f0 con 0x55bdd501ed30 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd5022d50 con 0x55bdd5018840 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501e380 0x55bdd502e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501e380 0x55bdd502e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42358/0 (socket says 172.21.15.42:42358) 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 -- 172.21.15.42:0/1315336745 learned_addr learned my addr 172.21.15.42:0/1315336745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd5018840 0x55bdd50250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 -- 172.21.15.42:0/1315336745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd5018840 msgr2=0x55bdd50250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd5018840 0x55bdd50250a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 -- 172.21.15.42:0/1315336745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 msgr2=0x55bdd502ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd502ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.721+0000 7f4b72d9d640 1 -- 172.21.15.42:0/1315336745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd4f20c00 con 0x55bdd501e380 2024-03-12T04:23:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.722+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501e380 0x55bdd502e8e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400a1f0 tx=0x7f4b6400add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96092e4e115bb1fd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.722+0000 7f4b68ff9640 1 -- 172.21.15.42:0/1315336745 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400feb0 con 0x55bdd501e380 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.722+0000 7f4b68ff9640 1 -- 172.21.15.42:0/1315336745 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4b6400c040 con 0x55bdd501e380 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.722+0000 7f4b68ff9640 1 -- 172.21.15.42:0/1315336745 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64013df0 con 0x55bdd501e380 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.722+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/1315336745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501e380 msgr2=0x55bdd502e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.722+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501e380 0x55bdd502e8e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400a1f0 tx=0x7f4b6400add0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/1315336745 shutdown_connections 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd502ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd501e380 0x55bdd502e8e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/1315336745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd5018840 0x55bdd50250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/1315336745 >> 172.21.15.42:0/1315336745 conn(0x55bdd501f480 msgr2=0x55bdd5017050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/1315336745 shutdown_connections 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/1315336745 wait complete. 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- start start 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 Processor -- start 2024-03-12T04:23:45.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd5018840 0x55bdd5088b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd5089040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd502df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd502e5a0 con 0x55bdd5018840 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd502e710 con 0x55bdd501ed30 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd502e8d0 con 0x55bdd501e380 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b7bd7f740 10 client.0 _ll_register_callbacks cb 0x7f4b5c004ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd5018840 0x55bdd5088b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd5018840 0x55bdd5088b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42360/0 (socket says 172.21.15.42:42360) 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.723+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3102336565 learned_addr learned my addr 172.21.15.42:0/3102336565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3102336565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 msgr2=0x55bdd5089040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd5089040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3102336565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 msgr2=0x55bdd502df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd502df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b6bfff640 1 -- 172.21.15.42:0/3102336565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdd4f84320 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b6bfff640 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd5018840 0x55bdd5088b00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009900 tx=0x7f4b5c02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe15b1848c5b730 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/3102336565 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c004520 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/3102336565 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4b5c000e40 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdd4fab900 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/3102336565 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5c02c790 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 10 client.17955 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bdd4f46910 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/3102336565 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55bdd4fab900 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/3102336565 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b5c04b020 con 0x55bdd5018840 2024-03-12T04:23:45.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 10 client.17955 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 10 client.17955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 20 client.17955 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd50278b0 0x55bdd5092750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 4 client.17955 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.724+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bdd4c85860 con 0x55bdd50278b0 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.725+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd50278b0 0x55bdd5092750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.725+0000 7f4b72d9d640 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd50278b0 0x55bdd5092750 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144e6f1b818786ee server_cookie=669c81e6bfb831a0 in_seq=0 out_seq=0 2024-03-12T04:23:45.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.725+0000 7f4b5a7fc640 10 client.17955 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.726+0000 7f4b5a7fc640 1 -- 172.21.15.42:0/3102336565 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bdd4c85860 con 0x55bdd50278b0 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.726+0000 7f4b5a7fc640 10 client.17955 handle_command_reply: tid=0 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.726+0000 7f4b7bd7f740 1 client.17955 shutdown 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd5018840 msgr2=0x55bdd5088b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd5018840 0x55bdd5088b00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009900 tx=0x7f4b5c02bc50 comp rx=0 tx=0).stop 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 shutdown_connections 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55bdd50278b0 0x55bdd5092750 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdd501ed30 0x55bdd502df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bdd501e380 0x55bdd5089040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 --2- 172.21.15.42:0/3102336565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdd5018840 0x55bdd5088b00 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b5a7fc640 0 client.17955 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 >> 172.21.15.42:0/3102336565 conn(0x55bdd501f480 msgr2=0x55bdd50178e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 shutdown_connections 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 1 -- 172.21.15.42:0/3102336565 wait complete. 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b7bd7f740 20 client.17955 trim_cache size 0 max 16384 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "c2eb048c-4b76-4a2c-98cd-cc02e7fe4077", 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:23:45.736 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4b600a1bd0 msgr2=0x7f4b600a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4b600a1bd0 0x7f4b600a4090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0a2890 tx=0x7f4b6c08fcf0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74073f00 msgr2=0x7f4b741a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74073f00 0x7f4b741a5970 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00eaa0 tx=0x7f4b6c00ef60 comp rx=0 tx=0).stop 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 shutdown_connections 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4b600a1bd0 0x7f4b600a4090 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b74073f00 0x7f4b741a5970 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74106450 0x7f4b740739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 --2- 172.21.15.42:0/3792212984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b74101380 0x7f4b74073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 >> 172.21.15.42:0/3792212984 conn(0x7f4b74078580 msgr2=0x7f4b740ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 shutdown_connections 2024-03-12T04:23:45.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.727+0000 7f4b791ac640 1 -- 172.21.15.42:0/3792212984 wait complete. 2024-03-12T04:23:45.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:23:45.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:23:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 -- 172.21.15.42:0/3888981093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48100c60 msgr2=0x7f7c48104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3888981093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48100c60 0x7f7c48104ca0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009930 tx=0x7f7c3802bc30 comp rx=0 tx=0).stop 2024-03-12T04:23:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 -- 172.21.15.42:0/3888981093 shutdown_connections 2024-03-12T04:23:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3888981093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c48105380 0x7f7c4810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3888981093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48100c60 0x7f7c48104ca0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3888981093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c481002b0 0x7f7c48100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 -- 172.21.15.42:0/3888981093 >> 172.21.15.42:0/3888981093 conn(0x7f7c48077190 msgr2=0x7f7c480775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 -- 172.21.15.42:0/3888981093 shutdown_connections 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.843+0000 7f7c501e9640 1 -- 172.21.15.42:0/3888981093 wait complete. 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 Processor -- start 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 -- start start 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c481002b0 0x7f7c48197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48105380 0x7f7c48197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4819bfa0 0x7f7c4819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4819c990 con 0x7f7c48105380 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4819cb00 con 0x7f7c481002b0 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c501e9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4819cc70 con 0x7f7c4819bfa0 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48105380 0x7f7c48197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48105380 0x7f7c48197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42376/0 (socket says 172.21.15.42:42376) 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 -- 172.21.15.42:0/3263834801 learned_addr learned my addr 172.21.15.42:0/3263834801 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4df85640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c481002b0 0x7f7c48197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4e786640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4819bfa0 0x7f7c4819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 -- 172.21.15.42:0/3263834801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4819bfa0 msgr2=0x7f7c4819c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4819bfa0 0x7f7c4819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 -- 172.21.15.42:0/3263834801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c481002b0 msgr2=0x7f7c48197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c481002b0 0x7f7c48197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 -- 172.21.15.42:0/3263834801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38009560 con 0x7f7c48105380 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4df85640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c481002b0 0x7f7c48197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.844+0000 7f7c4d784640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48105380 0x7f7c48197c70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000c00 tx=0x7f7c38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.845+0000 7f7c3effd640 1 -- 172.21.15.42:0/3263834801 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3802cb70 con 0x7f7c48105380 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.845+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c481a4250 con 0x7f7c48105380 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.845+0000 7f7c3effd640 1 -- 172.21.15.42:0/3263834801 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c3802de90 con 0x7f7c48105380 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.845+0000 7f7c3effd640 1 -- 172.21.15.42:0/3263834801 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38005380 con 0x7f7c48105380 2024-03-12T04:23:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.845+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c481a46b0 con 0x7f7c48105380 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.847+0000 7f7c3effd640 1 -- 172.21.15.42:0/3263834801 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7c380054e0 con 0x7f7c48105380 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.847+0000 7f7c52dbf740 1 Processor -- start 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c52dbf740 1 -- start start 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd32210340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd3221ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3217f280 con 0x55cd32210880 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3217c7b0 con 0x55cd3220ff60 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd32177f90 con 0x55cd3221dbc0 2024-03-12T04:23:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42378/0 (socket says 172.21.15.42:42378) 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 -- 172.21.15.42:0/1972369237 learned_addr learned my addr 172.21.15.42:0/1972369237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4e786640 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd32210340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c3effd640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7c340a1ba0 0x7f7c340a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4d784640 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd3221ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 -- 172.21.15.42:0/1972369237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 msgr2=0x55cd3221ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd3221ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 -- 172.21.15.42:0/1972369237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 msgr2=0x55cd32210340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd32210340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4df85640 1 -- 172.21.15.42:0/1972369237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd32136570 con 0x55cd32210880 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.848+0000 7f7c4e786640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7c340a1ba0 0x7f7c340a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c3effd640 1 -- 172.21.15.42:0/3263834801 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c380ef960 con 0x7f7c48105380 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c4df85640 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d680 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009c50 tx=0x7f7c4400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e057b306b9f9cba2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c33fff640 1 -- 172.21.15.42:0/1972369237 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44005020 con 0x55cd32210880 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c33fff640 1 -- 172.21.15.42:0/1972369237 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c44005630 con 0x55cd32210880 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c33fff640 1 -- 172.21.15.42:0/1972369237 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440116b0 con 0x55cd32210880 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c52dbf740 1 -- 172.21.15.42:0/1972369237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 msgr2=0x55cd3221d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c4e786640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7c340a1ba0 0x7f7c340a4060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005fd0 tx=0x7f7c40004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.849+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d680 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009c50 tx=0x7f7c4400e6c0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- 172.21.15.42:0/1972369237 shutdown_connections 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd3221ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d680 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/1972369237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd32210340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- 172.21.15.42:0/1972369237 >> 172.21.15.42:0/1972369237 conn(0x55cd3220a990 msgr2=0x55cd3220ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- 172.21.15.42:0/1972369237 shutdown_connections 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- 172.21.15.42:0/1972369237 wait complete. 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- start start 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 Processor -- start 2024-03-12T04:23:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd3221d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd3221add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd31e4b430 con 0x55cd32210880 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd31e07e00 con 0x55cd3221dbc0 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd320fcfd0 con 0x55cd3220ff60 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.850+0000 7f7c52dbf740 10 client.0 _ll_register_callbacks cb 0x55cd32152fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42380/0 (socket says 172.21.15.42:42380) 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 -- 172.21.15.42:0/2324124026 learned_addr learned my addr 172.21.15.42:0/2324124026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4df85640 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd3221d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4e786640 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd3221add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 -- 172.21.15.42:0/2324124026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 msgr2=0x55cd3221d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd3221d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 -- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 msgr2=0x55cd3221add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd3221add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 -- 172.21.15.42:0/2324124026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c44009880 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c4d784640 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221a890 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ed240 tx=0x7f7c380bc200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b30097db0bfff07 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c317fa640 1 -- 172.21.15.42:0/2324124026 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380d51b0 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c317fa640 1 -- 172.21.15.42:0/2324124026 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c380a5e40 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd31e777e0 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c317fa640 1 -- 172.21.15.42:0/2324124026 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c380bc6c0 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c52dbf740 10 client.17967 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cd32176610 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.851+0000 7f7c317fa640 1 -- 172.21.15.42:0/2324124026 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55cd31e777e0 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c317fa640 1 -- 172.21.15.42:0/2324124026 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c3803c070 con 0x55cd32210880 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c52dbf740 10 client.17967 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c52dbf740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c52dbf740 20 client.17967 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd3222daa0 0x55cd3222fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c52dbf740 4 client.17967 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:23:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd31eb4720 con 0x55cd3222daa0 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c4df85640 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd3222daa0 0x55cd3222fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c4df85640 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd3222daa0 0x55cd3222fe90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdba87c4cb67ddc0 server_cookie=2d0a4a4148f96b49 in_seq=0 out_seq=0 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.852+0000 7f7c317fa640 10 client.17967 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c317fa640 1 -- 172.21.15.42:0/2324124026 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd31eb4720 con 0x55cd3222daa0 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c317fa640 10 client.17967 handle_command_reply: tid=0 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 client.17967 shutdown 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 msgr2=0x55cd3221a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221a890 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ed240 tx=0x7f7c380bc200 comp rx=0 tx=0).stop 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 shutdown_connections 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd3222daa0 0x55cd3222fe90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd3221add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221a890 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c317fa640 0 client.17967 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/2324124026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd3221d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 >> 172.21.15.42:0/2324124026 conn(0x55cd3220a990 msgr2=0x55cd3220c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 shutdown_connections 2024-03-12T04:23:45.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 1 -- 172.21.15.42:0/2324124026 wait complete. 2024-03-12T04:23:45.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.853+0000 7f7c52dbf740 20 client.17967 trim_cache size 0 max 16384 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 Processor -- start 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 -- start start 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd3221cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd32224af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd32229a10 con 0x55cd32210880 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3220f600 con 0x55cd3220ff60 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3220f3a0 con 0x55cd3221dbc0 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42388/0 (socket says 172.21.15.42:42388) 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.868+0000 7f7c4d784640 1 -- 172.21.15.42:0/4242863204 learned_addr learned my addr 172.21.15.42:0/4242863204 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4e786640 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd32224af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4d784640 1 -- 172.21.15.42:0/4242863204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 msgr2=0x55cd32224af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4df85640 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd3221cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4d784640 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd32224af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4d784640 1 -- 172.21.15.42:0/4242863204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 msgr2=0x55cd3221cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4d784640 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd3221cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4d784640 1 -- 172.21.15.42:0/4242863204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd32112ac0 con 0x55cd32210880 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4e786640 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd32224af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4df85640 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd3221cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c4d784640 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d500 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ed6d0 tx=0x7f7c380bc330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879af4427dcaf95d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c337fe640 1 -- 172.21.15.42:0/4242863204 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380c7020 con 0x55cd32210880 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c337fe640 1 -- 172.21.15.42:0/4242863204 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c380d5040 con 0x55cd32210880 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.869+0000 7f7c337fe640 1 -- 172.21.15.42:0/4242863204 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380d5220 con 0x55cd32210880 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- 172.21.15.42:0/4242863204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 msgr2=0x55cd3221d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d500 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ed6d0 tx=0x7f7c380bc330 comp rx=0 tx=0).stop 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- 172.21.15.42:0/4242863204 shutdown_connections 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3221dbc0 0x55cd32224af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd3221d500 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/4242863204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3220ff60 0x55cd3221cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- 172.21.15.42:0/4242863204 >> 172.21.15.42:0/4242863204 conn(0x55cd3220a990 msgr2=0x55cd3220ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- 172.21.15.42:0/4242863204 shutdown_connections 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- 172.21.15.42:0/4242863204 wait complete. 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- start start 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 Processor -- start 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd32224720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd32224c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd32220bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd32221200 con 0x55cd32210880 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd32221370 con 0x55cd3221dbc0 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd32221530 con 0x55cd3220ff60 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c52dbf740 10 client.0 _ll_register_callbacks cb 0x7f7c44009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd32224c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c4d784640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd32224c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42396/0 (socket says 172.21.15.42:42396) 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.870+0000 7f7c4d784640 1 -- 172.21.15.42:0/473750203 learned_addr learned my addr 172.21.15.42:0/473750203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4e786640 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd32220bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4df85640 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd32224720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4d784640 1 -- 172.21.15.42:0/473750203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 msgr2=0x55cd32224720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4d784640 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd32224720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4d784640 1 -- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 msgr2=0x55cd32220bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4d784640 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd32220bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4d784640 1 -- 172.21.15.42:0/473750203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd32176610 con 0x55cd32210880 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c4d784640 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd32224c60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7c380bc3c0 tx=0x7f7c38005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24679b298a59bf23 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c317fa640 1 -- 172.21.15.42:0/473750203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380f3070 con 0x55cd32210880 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c317fa640 1 -- 172.21.15.42:0/473750203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c380a5e40 con 0x55cd32210880 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c44005630 con 0x55cd32210880 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c317fa640 1 -- 172.21.15.42:0/473750203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c3803d3b0 con 0x55cd32210880 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c52dbf740 10 client.17973 fetch_fsmap learned FSMap version 27 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cd321386a0 con 0x55cd32210880 2024-03-12T04:23:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c317fa640 1 -- 172.21.15.42:0/473750203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c44005630 con 0x55cd32210880 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.871+0000 7f7c317fa640 1 -- 172.21.15.42:0/473750203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c3803c050 con 0x55cd32210880 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c52dbf740 10 client.17973 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c52dbf740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c52dbf740 20 client.17973 populate_metadata read hostname 'smithi042' 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd32284ad0 0x55cd32286ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c52dbf740 4 client.17973 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd31f03890 con 0x55cd32284ad0 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c4df85640 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd32284ad0 0x55cd32286ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c4df85640 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd32284ad0 0x55cd32286ec0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26634c19e8a5f831 server_cookie=d0efd973a19e5cb1 in_seq=0 out_seq=0 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c317fa640 10 client.17973 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c317fa640 1 -- 172.21.15.42:0/473750203 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cd31f03890 con 0x55cd32284ad0 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.872+0000 7f7c317fa640 10 client.17973 handle_command_reply: tid=0 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 client.17973 shutdown 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 msgr2=0x55cd32224c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd32224c60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7c380bc3c0 tx=0x7f7c38005790 comp rx=0 tx=0).stop 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 shutdown_connections 2024-03-12T04:23:45.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55cd32284ad0 0x55cd32286ec0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd3221dbc0 0x55cd32220bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd32210880 0x55cd32224c60 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 --2- 172.21.15.42:0/473750203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd3220ff60 0x55cd32224720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c317fa640 0 client.17973 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 >> 172.21.15.42:0/473750203 conn(0x55cd3220a990 msgr2=0x55cd32215b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 shutdown_connections 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 1 -- 172.21.15.42:0/473750203 wait complete. 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c52dbf740 20 client.17973 trim_cache size 0 max 16384 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:23:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: "c2eb048c-4b76-4a2c-98cd-cc02e7fe4077": { 2024-03-12T04:23:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:23:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "c2eb048c-4b76-4a2c-98cd-cc02e7fe4077", 2024-03-12T04:23:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:23:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:23:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:23:45.882 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7c340a1ba0 msgr2=0x7f7c340a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7c340a1ba0 0x7f7c340a4060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005fd0 tx=0x7f7c40004a40 comp rx=0 tx=0).stop 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48105380 msgr2=0x7f7c48197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48105380 0x7f7c48197c70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000c00 tx=0x7f7c38004270 comp rx=0 tx=0).stop 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 shutdown_connections 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f7c340a1ba0 0x7f7c340a4060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4819bfa0 0x7f7c4819c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48105380 0x7f7c48197c70 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 --2- 172.21.15.42:0/3263834801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c481002b0 0x7f7c48197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 >> 172.21.15.42:0/3263834801 conn(0x7f7c48077190 msgr2=0x7f7c48069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 shutdown_connections 2024-03-12T04:23:45.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:23:45.873+0000 7f7c501e9640 1 -- 172.21.15.42:0/3263834801 wait complete. 2024-03-12T04:23:45.886 INFO:tasks.cephfs.filesystem:scrub status for tag:c2eb048c-4b76-4a2c-98cd-cc02e7fe4077 - {'path': '/', 'tag': 'c2eb048c-4b76-4a2c-98cd-cc02e7fe4077', 'options': 'recursive,force'} 2024-03-12T04:23:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:46 smithi119 ceph-mon[36119]: pgmap v264: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:46 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:23:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:46 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:46 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:23:46.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:46 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:46 smithi136 ceph-mon[35571]: pgmap v264: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:46 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:23:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:46 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:46 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:23:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:46 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:46 smithi042 ceph-mon[32260]: pgmap v264: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:23:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:46 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:23:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:46 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:46 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:23:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:46 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:23:48.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:48 smithi119 ceph-mon[36119]: pgmap v265: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:48 smithi136 ceph-mon[35571]: pgmap v265: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:48.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:48 smithi042 ceph-mon[32260]: pgmap v265: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:49.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:49 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:23:49.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:49 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:23:49.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:49 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:23:50.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:50 smithi119 ceph-mon[36119]: pgmap v266: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:50.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:50 smithi136 ceph-mon[35571]: pgmap v266: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:50.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:50 smithi042 ceph-mon[32260]: pgmap v266: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:52.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:52 smithi119 ceph-mon[36119]: pgmap v267: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:52 smithi136 ceph-mon[35571]: pgmap v267: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:52.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:52 smithi042 ceph-mon[32260]: pgmap v267: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:54.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:54 smithi119 ceph-mon[36119]: pgmap v268: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:54 smithi136 ceph-mon[35571]: pgmap v268: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:54 smithi042 ceph-mon[32260]: pgmap v268: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:56.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:56 smithi119 ceph-mon[36119]: pgmap v269: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:56.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:56 smithi136 ceph-mon[35571]: pgmap v269: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:56.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:56 smithi042 ceph-mon[32260]: pgmap v269: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:23:58.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:23:58 smithi119 ceph-mon[36119]: pgmap v270: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:58.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:23:58 smithi136 ceph-mon[35571]: pgmap v270: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:23:58.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:23:58 smithi042 ceph-mon[32260]: pgmap v270: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:00.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:00 smithi119 ceph-mon[36119]: pgmap v271: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:00.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:00 smithi136 ceph-mon[35571]: pgmap v271: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:00.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:00 smithi042 ceph-mon[32260]: pgmap v271: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:02 smithi119 ceph-mon[36119]: pgmap v272: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:02.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:02 smithi136 ceph-mon[35571]: pgmap v272: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:02.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:02 smithi042 ceph-mon[32260]: pgmap v272: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:04.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:04 smithi119 ceph-mon[36119]: pgmap v273: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:24:04.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:04 smithi136 ceph-mon[35571]: pgmap v273: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:24:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:04 smithi042 ceph-mon[32260]: pgmap v273: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:24:07.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:07 smithi119 ceph-mon[36119]: pgmap v274: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:24:07.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:07 smithi136 ceph-mon[35571]: pgmap v274: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:24:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:07 smithi042 ceph-mon[32260]: pgmap v274: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:24:09.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:09 smithi119 ceph-mon[36119]: pgmap v275: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:24:09.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:09 smithi136 ceph-mon[35571]: pgmap v275: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:24:09.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:09 smithi042 ceph-mon[32260]: pgmap v275: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:24:10.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:10 smithi119 ceph-mon[36119]: osdmap e95: 12 total, 12 up, 12 in 2024-03-12T04:24:10.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:10 smithi136 ceph-mon[35571]: osdmap e95: 12 total, 12 up, 12 in 2024-03-12T04:24:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:10 smithi042 ceph-mon[32260]: osdmap e95: 12 total, 12 up, 12 in 2024-03-12T04:24:11.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:11 smithi119 ceph-mon[36119]: pgmap v277: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:24:11.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:11 smithi136 ceph-mon[35571]: pgmap v277: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:24:11.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:11 smithi042 ceph-mon[32260]: pgmap v277: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:24:12.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:12 smithi042 ceph-mon[32260]: pgmap v278: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:24:12.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:12 smithi119 ceph-mon[36119]: pgmap v278: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:24:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:12 smithi136 ceph-mon[35571]: pgmap v278: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:24:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:14 smithi119 ceph-mon[36119]: pgmap v279: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:24:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:14 smithi136 ceph-mon[35571]: pgmap v279: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:24:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:14 smithi042 ceph-mon[32260]: pgmap v279: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:24:15.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:24:15.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:24:16.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 -- 172.21.15.42:0/499206182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c100c90 msgr2=0x7f632c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 --2- 172.21.15.42:0/499206182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c100c90 0x7f632c10ce40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6324009ca0 tx=0x7f632402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 -- 172.21.15.42:0/499206182 shutdown_connections 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 --2- 172.21.15.42:0/499206182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c100c90 0x7f632c10ce40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 --2- 172.21.15.42:0/499206182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f632c1002d0 0x7f632c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 --2- 172.21.15.42:0/499206182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f632c106110 0x7f632c1064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 -- 172.21.15.42:0/499206182 >> 172.21.15.42:0/499206182 conn(0x7f632c076e20 msgr2=0x7f632c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 -- 172.21.15.42:0/499206182 shutdown_connections 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.002+0000 7f63334df640 1 -- 172.21.15.42:0/499206182 wait complete. 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 Processor -- start 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 -- start start 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f632c1002d0 0x7f632c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f632c100c90 0x7f632c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c106110 0x7f632c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c19c550 con 0x7f632c106110 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c19c6c0 con 0x7f632c100c90 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f63334df640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c19c880 con 0x7f632c1002d0 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f6331a7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c106110 0x7f632c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f6331a7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c106110 0x7f632c19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51906/0 (socket says 172.21.15.42:51906) 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f6331a7c640 1 -- 172.21.15.42:0/411059513 learned_addr learned my addr 172.21.15.42:0/411059513 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.003+0000 7f6330a7a640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f632c100c90 0x7f632c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f633127b640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f632c1002d0 0x7f632c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f6331a7c640 1 -- 172.21.15.42:0/411059513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f632c1002d0 msgr2=0x7f632c1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f6331a7c640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f632c1002d0 0x7f632c1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f6331a7c640 1 -- 172.21.15.42:0/411059513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f632c100c90 msgr2=0x7f632c197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f6331a7c640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f632c100c90 0x7f632c197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f6331a7c640 1 -- 172.21.15.42:0/411059513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324009900 con 0x7f632c106110 2024-03-12T04:24:16.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f633127b640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f632c1002d0 0x7f632c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f6331a7c640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c106110 0x7f632c19bf50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f632402c700 tx=0x7f6324035d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f63227fc640 1 -- 172.21.15.42:0/411059513 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240329e0 con 0x7f632c106110 2024-03-12T04:24:16.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f63227fc640 1 -- 172.21.15.42:0/411059513 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6324032410 con 0x7f632c106110 2024-03-12T04:24:16.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c19cb00 con 0x7f632c106110 2024-03-12T04:24:16.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f63227fc640 1 -- 172.21.15.42:0/411059513 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324004840 con 0x7f632c106110 2024-03-12T04:24:16.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.004+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c1a43e0 con 0x7f632c106110 2024-03-12T04:24:16.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.006+0000 7f63227fc640 1 -- 172.21.15.42:0/411059513 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f632403b040 con 0x7f632c106110 2024-03-12T04:24:16.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 Processor -- start 2024-03-12T04:24:16.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 -- start start 2024-03-12T04:24:16.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f63227fc640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f63180a1c20 0x7f63180a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f76109690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f76112270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f76114ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f76078280 con 0x557f76109bd0 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f760757b0 con 0x557f761092b0 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f633618d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f76070f90 con 0x557f761127b0 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f6330a7a640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f63180a1c20 0x7f63180a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f63227fc640 1 -- 172.21.15.42:0/411059513 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f63240f0130 con 0x7f632c106110 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.007+0000 7f6330a7a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f76112270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f76112270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51910/0 (socket says 172.21.15.42:51910) 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 -- 172.21.15.42:0/356525364 learned_addr learned my addr 172.21.15.42:0/356525364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f633127b640 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f76109690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6331a7c640 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f76114ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 -- 172.21.15.42:0/356525364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 msgr2=0x557f76114ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f76114ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 -- 172.21.15.42:0/356525364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 msgr2=0x557f76109690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f76109690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 -- 172.21.15.42:0/356525364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f7602f570 con 0x557f76109bd0 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f63180a1c20 0x7f63180a40e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f631c012960 tx=0x7f631c00bb70 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f6330a7a640 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f76112270 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f631c010740 tx=0x7f631c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f30973ac199caf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f63177fe640 1 -- 172.21.15.42:0/356525364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c010e00 con 0x557f76109bd0 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f63177fe640 1 -- 172.21.15.42:0/356525364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f631c00dc00 con 0x557f76109bd0 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.008+0000 7f63177fe640 1 -- 172.21.15.42:0/356525364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c020970 con 0x557f76109bd0 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 -- 172.21.15.42:0/356525364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 msgr2=0x557f76112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f76112270 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f631c010740 tx=0x7f631c01c000 comp rx=0 tx=0).stop 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 -- 172.21.15.42:0/356525364 shutdown_connections 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f76114ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f76112270 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 --2- 172.21.15.42:0/356525364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f76109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 -- 172.21.15.42:0/356525364 >> 172.21.15.42:0/356525364 conn(0x557f76103990 msgr2=0x557f76103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 -- 172.21.15.42:0/356525364 shutdown_connections 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 -- 172.21.15.42:0/356525364 wait complete. 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 -- start start 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.009+0000 7f633618d740 1 Processor -- start 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633618d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f761121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633618d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f7610f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633618d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f7610fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633618d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f75d44430 con 0x557f76109bd0 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633618d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f75d00e00 con 0x557f761092b0 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633618d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f75ff5fd0 con 0x557f761127b0 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633618d740 10 client.0 _ll_register_callbacks cb 0x557f7604bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f7610f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f7610f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51926/0 (socket says 172.21.15.42:51926) 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 -- 172.21.15.42:0/529750388 learned_addr learned my addr 172.21.15.42:0/529750388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6331a7c640 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f7610fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f633127b640 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f761121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 -- 172.21.15.42:0/529750388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 msgr2=0x557f7610fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f7610fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 -- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 msgr2=0x557f761121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f761121d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.010+0000 7f6330a7a640 1 -- 172.21.15.42:0/529750388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c01a040 con 0x557f76109bd0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f6330a7a640 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f7610f560 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f631c0058a0 tx=0x7f631c00dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465b0318e8e5fefe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f6314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f6314ff9640 1 -- 172.21.15.42:0/529750388 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c00d960 con 0x557f76109bd0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f6314ff9640 1 -- 172.21.15.42:0/529750388 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f631c005040 con 0x557f76109bd0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f75d707e0 con 0x557f76109bd0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f6314ff9640 1 -- 172.21.15.42:0/529750388 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f631c005240 con 0x557f76109bd0 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 10 client.17985 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f7606f610 con 0x557f76109bd0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f6314ff9640 1 -- 172.21.15.42:0/529750388 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557f75d707e0 con 0x557f76109bd0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f6314ff9640 1 -- 172.21.15.42:0/529750388 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f631c020c80 con 0x557f76109bd0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 10 client.17985 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 20 client.17985 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f761228e0 0x557f76124d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 4 client.17985 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.011+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f75dad720 con 0x557f761228e0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.012+0000 7f633127b640 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f761228e0 0x557f76124d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.012+0000 7f633127b640 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f761228e0 0x557f76124d00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff40351823beba8b server_cookie=f12f255f57c475a3 in_seq=0 out_seq=0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.012+0000 7f6314ff9640 10 client.17985 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f6314ff9640 1 -- 172.21.15.42:0/529750388 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f75dad720 con 0x557f761228e0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f6314ff9640 10 client.17985 handle_command_reply: tid=0 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 client.17985 shutdown 2024-03-12T04:24:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 msgr2=0x557f7610f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f7610f560 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f631c0058a0 tx=0x7f631c00dfa0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 shutdown_connections 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f761228e0 0x557f76124d00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761127b0 0x557f7610fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f76109bd0 0x557f7610f560 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f6314ff9640 0 client.17985 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 --2- 172.21.15.42:0/529750388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f761092b0 0x557f761121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 >> 172.21.15.42:0/529750388 conn(0x557f76103990 msgr2=0x557f76105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 shutdown_connections 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 1 -- 172.21.15.42:0/529750388 wait complete. 2024-03-12T04:24:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.013+0000 7f633618d740 20 client.17985 trim_cache size 0 max 16384 2024-03-12T04:24:16.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 Processor -- start 2024-03-12T04:24:16.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 -- start start 2024-03-12T04:24:16.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76111c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 0x557f761121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f7610de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7611f440 con 0x557f761127b0 2024-03-12T04:24:16.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f761153f0 con 0x557f76109bd0 2024-03-12T04:24:16.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.028+0000 7f633618d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7611e590 con 0x557f761092b0 2024-03-12T04:24:16.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f7610de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f7610de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51930/0 (socket says 172.21.15.42:51930) 2024-03-12T04:24:16.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 -- 172.21.15.42:0/2306508960 learned_addr learned my addr 172.21.15.42:0/2306508960 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f633127b640 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76111c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 -- 172.21.15.42:0/2306508960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 msgr2=0x557f76111c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76111c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 -- 172.21.15.42:0/2306508960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 msgr2=0x557f761121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 0x557f761121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 -- 172.21.15.42:0/2306508960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f7600bac0 con 0x557f761127b0 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f633127b640 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76111c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6331a7c640 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f7610de30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6324005fa0 tx=0x7f63240bc7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7609e3822e494c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6316ffd640 1 -- 172.21.15.42:0/2306508960 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240bce90 con 0x557f761127b0 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6316ffd640 1 -- 172.21.15.42:0/2306508960 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f63240c0fe0 con 0x557f761127b0 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.029+0000 7f6316ffd640 1 -- 172.21.15.42:0/2306508960 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240b9690 con 0x557f761127b0 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 -- 172.21.15.42:0/2306508960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 msgr2=0x557f7610de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f7610de30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6324005fa0 tx=0x7f63240bc7f0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 -- 172.21.15.42:0/2306508960 shutdown_connections 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f7610de30 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 0x557f761121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 --2- 172.21.15.42:0/2306508960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76111c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 -- 172.21.15.42:0/2306508960 >> 172.21.15.42:0/2306508960 conn(0x557f76103990 msgr2=0x557f7610a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 -- 172.21.15.42:0/2306508960 shutdown_connections 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 -- 172.21.15.42:0/2306508960 wait complete. 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 -- start start 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.030+0000 7f633618d740 1 Processor -- start 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76116060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 0x557f761165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f76116ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f76178e20 con 0x557f761127b0 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f76178f60 con 0x557f76109bd0 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f761790a0 con 0x557f761092b0 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 10 client.0 _ll_register_callbacks cb 0x7f631c0124b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f76116ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f76116ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51946/0 (socket says 172.21.15.42:51946) 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 -- 172.21.15.42:0/187504985 learned_addr learned my addr 172.21.15.42:0/187504985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633127b640 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6330a7a640 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 0x557f761165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 -- 172.21.15.42:0/187504985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 msgr2=0x557f76116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 -- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 msgr2=0x557f761165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 0x557f761165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 -- 172.21.15.42:0/187504985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c1a43e0 con 0x557f761127b0 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6331a7c640 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f76116ae0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f63240b92f0 tx=0x7f6324032b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6f7de059a6b274 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6314ff9640 1 -- 172.21.15.42:0/187504985 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324004250 con 0x557f761127b0 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6314ff9640 1 -- 172.21.15.42:0/187504985 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f63240d9070 con 0x557f761127b0 2024-03-12T04:24:16.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f631c00dc00 con 0x557f761127b0 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f6314ff9640 1 -- 172.21.15.42:0/187504985 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63240bce90 con 0x557f761127b0 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 10 client.17991 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.031+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f760316a0 con 0x557f761127b0 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f6314ff9640 1 -- 172.21.15.42:0/187504985 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f631c00dc00 con 0x557f761127b0 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f6314ff9640 1 -- 172.21.15.42:0/187504985 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6324001e40 con 0x557f761127b0 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633618d740 10 client.17991 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633618d740 10 client.17991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633618d740 20 client.17991 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633618d740 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f7617a2b0 0x557f7617c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633618d740 4 client.17991 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f75dfc890 con 0x557f7617a2b0 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633127b640 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f7617a2b0 0x557f7617c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f6314ff9640 10 client.17991 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.032+0000 7f633127b640 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f7617a2b0 0x557f7617c6a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5fcc252b28410fb server_cookie=ef8701ba52ecb73e in_seq=0 out_seq=0 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f6314ff9640 1 -- 172.21.15.42:0/187504985 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557f75dfc890 con 0x557f7617a2b0 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f6314ff9640 10 client.17991 handle_command_reply: tid=0 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 client.17991 shutdown 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 msgr2=0x557f76116ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f76116ae0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f63240b92f0 tx=0x7f6324032b40 comp rx=0 tx=0).stop 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 shutdown_connections 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x557f7617a2b0 0x557f7617c6a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557f761127b0 0x557f76116ae0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f6314ff9640 0 client.17991 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557f76109bd0 0x557f761165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 --2- 172.21.15.42:0/187504985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f761092b0 0x557f76116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 >> 172.21.15.42:0/187504985 conn(0x557f76103990 msgr2=0x557f7610ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 shutdown_connections 2024-03-12T04:24:16.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 1 -- 172.21.15.42:0/187504985 wait complete. 2024-03-12T04:24:16.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.033+0000 7f633618d740 20 client.17991 trim_cache size 0 max 16384 2024-03-12T04:24:16.044 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:24:16.044 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:24:16.044 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:24:16.044 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:24:16.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f63180a1c20 msgr2=0x7f63180a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f63180a1c20 0x7f63180a40e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f631c012960 tx=0x7f631c00bb70 comp rx=0 tx=0).stop 2024-03-12T04:24:16.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c106110 msgr2=0x7f632c19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c106110 0x7f632c19bf50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f632402c700 tx=0x7f6324035d20 comp rx=0 tx=0).stop 2024-03-12T04:24:16.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 shutdown_connections 2024-03-12T04:24:16.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f63180a1c20 0x7f63180a40e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632c106110 0x7f632c19bf50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f632c100c90 0x7f632c197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 --2- 172.21.15.42:0/411059513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f632c1002d0 0x7f632c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 >> 172.21.15.42:0/411059513 conn(0x7f632c076e20 msgr2=0x7f632c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 shutdown_connections 2024-03-12T04:24:16.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.034+0000 7f63334df640 1 -- 172.21.15.42:0/411059513 wait complete. 2024-03-12T04:24:16.047 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:24:16.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:24:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 -- 172.21.15.42:0/3601554452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f4100c60 msgr2=0x7fb9f4104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3601554452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f4100c60 0x7fb9f4104ca0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009930 tx=0x7fb9dc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 -- 172.21.15.42:0/3601554452 shutdown_connections 2024-03-12T04:24:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3601554452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f4105380 0x7fb9f410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3601554452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f4100c60 0x7fb9f4104ca0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3601554452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f41002b0 0x7fb9f4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 -- 172.21.15.42:0/3601554452 >> 172.21.15.42:0/3601554452 conn(0x7fb9f4077120 msgr2=0x7fb9f4077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 -- 172.21.15.42:0/3601554452 shutdown_connections 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.148+0000 7fb9f881c640 1 -- 172.21.15.42:0/3601554452 wait complete. 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 Processor -- start 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 -- start start 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f41002b0 0x7fb9f4101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f4105380 0x7fb9f4102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f41039e0 0x7fb9f41027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41aa230 con 0x7fb9f41039e0 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41aa3a0 con 0x7fb9f41002b0 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f881c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41aa560 con 0x7fb9f4105380 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f41039e0 0x7fb9f41027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f41039e0 0x7fb9f41027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51974/0 (socket says 172.21.15.42:51974) 2024-03-12T04:24:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 -- 172.21.15.42:0/3950511704 learned_addr learned my addr 172.21.15.42:0/3950511704 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9ebfff640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f4105380 0x7fb9f4102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 -- 172.21.15.42:0/3950511704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f4105380 msgr2=0x7fb9f4102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f4105380 0x7fb9f4102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 -- 172.21.15.42:0/3950511704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f41002b0 msgr2=0x7fb9f4101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f259c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f41002b0 0x7fb9f4101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f41002b0 0x7fb9f4101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9f2d9d640 1 -- 172.21.15.42:0/3950511704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc009560 con 0x7fb9f41039e0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.149+0000 7fb9ebfff640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f4105380 0x7fb9f4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.150+0000 7fb9f2d9d640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f41039e0 0x7fb9f41027c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00cce0 tx=0x7fb9ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.150+0000 7fb9eb7fe640 1 -- 172.21.15.42:0/3950511704 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00f450 con 0x7fb9f41039e0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.150+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f41aa840 con 0x7fb9f41039e0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.150+0000 7fb9eb7fe640 1 -- 172.21.15.42:0/3950511704 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9ec00fa60 con 0x7fb9f41039e0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.150+0000 7fb9eb7fe640 1 -- 172.21.15.42:0/3950511704 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec018b10 con 0x7fb9f41039e0 2024-03-12T04:24:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.150+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f41aad20 con 0x7fb9f41039e0 2024-03-12T04:24:16.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.151+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f4101780 con 0x7fb9f41039e0 2024-03-12T04:24:16.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.152+0000 7fb9eb7fe640 1 -- 172.21.15.42:0/3950511704 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fb9ec00fbd0 con 0x7fb9f41039e0 2024-03-12T04:24:16.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.153+0000 7fb9eb7fe640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb9e00a1ba0 0x7fb9e00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.153+0000 7fb9f259c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb9e00a1ba0 0x7fb9e00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.153+0000 7fb9eb7fe640 1 -- 172.21.15.42:0/3950511704 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb9ec0cdc80 con 0x7fb9f41039e0 2024-03-12T04:24:16.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.153+0000 7fb9f259c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb9e00a1ba0 0x7fb9e00a4060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005fd0 tx=0x7fb9e4005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.157+0000 7fb9eb7fe640 1 -- 172.21.15.42:0/3950511704 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9ec096b70 con 0x7fb9f41039e0 2024-03-12T04:24:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.377+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9f4109660 con 0x7fb9f41039e0 2024-03-12T04:24:16.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.378+0000 7fb9eb7fe640 1 -- 172.21.15.42:0/3950511704 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fb9ec09a230 con 0x7fb9f41039e0 2024-03-12T04:24:16.381 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:24:16.381 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:24:16.381 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb9e00a1ba0 msgr2=0x7fb9e00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb9e00a1ba0 0x7fb9e00a4060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005fd0 tx=0x7fb9e4005850 comp rx=0 tx=0).stop 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f41039e0 msgr2=0x7fb9f41027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f41039e0 0x7fb9f41027c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00cce0 tx=0x7fb9ec00a620 comp rx=0 tx=0).stop 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 shutdown_connections 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fb9e00a1ba0 0x7fb9e00a4060 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f41039e0 0x7fb9f41027c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f4105380 0x7fb9f4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 --2- 172.21.15.42:0/3950511704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f41002b0 0x7fb9f4101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 >> 172.21.15.42:0/3950511704 conn(0x7fb9f4077120 msgr2=0x7fb9f4069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 shutdown_connections 2024-03-12T04:24:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.380+0000 7fb9f881c640 1 -- 172.21.15.42:0/3950511704 wait complete. 2024-03-12T04:24:16.394 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:16.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:24:16.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:16 smithi119 ceph-mon[36119]: osdmap e96: 12 total, 12 up, 12 in 2024-03-12T04:24:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:16 smithi136 ceph-mon[35571]: osdmap e96: 12 total, 12 up, 12 in 2024-03-12T04:24:16.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- 172.21.15.42:0/2902356232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb90068fa0 msgr2=0x7fbb90105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 --2- 172.21.15.42:0/2902356232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb90068fa0 0x7fbb90105de0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009930 tx=0x7fbb8002bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- 172.21.15.42:0/2902356232 shutdown_connections 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 --2- 172.21.15.42:0/2902356232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb901064c0 0x7fbb9010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 --2- 172.21.15.42:0/2902356232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb90068fa0 0x7fbb90105de0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 --2- 172.21.15.42:0/2902356232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb900685f0 0x7fbb900689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- 172.21.15.42:0/2902356232 >> 172.21.15.42:0/2902356232 conn(0x7fbb90077180 msgr2=0x7fbb90077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- 172.21.15.42:0/2902356232 shutdown_connections 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- 172.21.15.42:0/2902356232 wait complete. 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 Processor -- start 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- start start 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb900685f0 0x7fbb90100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb901064c0 0x7fbb90100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb90100ac0 0x7fbb901a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb901a5eb0 con 0x7fbb900685f0 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb901a6020 con 0x7fbb901064c0 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.496+0000 7fbb9758a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb901a61e0 con 0x7fbb90100ac0 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb900685f0 0x7fbb90100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb900685f0 0x7fbb90100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52004/0 (socket says 172.21.15.42:52004) 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 -- 172.21.15.42:0/486723093 learned_addr learned my addr 172.21.15.42:0/486723093 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb94b25640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb901064c0 0x7fbb90100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95b27640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb90100ac0 0x7fbb901a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 -- 172.21.15.42:0/486723093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb90100ac0 msgr2=0x7fbb901a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb90100ac0 0x7fbb901a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 -- 172.21.15.42:0/486723093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb901064c0 msgr2=0x7fbb90100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb901064c0 0x7fbb90100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 -- 172.21.15.42:0/486723093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb80009560 con 0x7fbb900685f0 2024-03-12T04:24:16.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95b27640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb90100ac0 0x7fbb901a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb95326640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb900685f0 0x7fbb90100040 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00c920 tx=0x7fbb8c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb867fc640 1 -- 172.21.15.42:0/486723093 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c007c70 con 0x7fbb900685f0 2024-03-12T04:24:16.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb867fc640 1 -- 172.21.15.42:0/486723093 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb8c007dd0 con 0x7fbb900685f0 2024-03-12T04:24:16.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb901a64c0 con 0x7fbb900685f0 2024-03-12T04:24:16.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb867fc640 1 -- 172.21.15.42:0/486723093 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0187c0 con 0x7fbb900685f0 2024-03-12T04:24:16.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.497+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb901a69a0 con 0x7fbb900685f0 2024-03-12T04:24:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.499+0000 7fbb867fc640 1 -- 172.21.15.42:0/486723093 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fbb8c00f450 con 0x7fbb900685f0 2024-03-12T04:24:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 Processor -- start 2024-03-12T04:24:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 -- start start 2024-03-12T04:24:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5c70960 con 0x55e6e5cb5250 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5c24310 con 0x55e6e5cbe750 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb9a235740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5c21840 con 0x55e6e5cb5b70 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb867fc640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb7c0a1ba0 0x7fbb7c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb95b27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb95b27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52010/0 (socket says 172.21.15.42:52010) 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb95b27640 1 -- 172.21.15.42:0/2811179479 learned_addr learned my addr 172.21.15.42:0/2811179479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb867fc640 1 -- 172.21.15.42:0/486723093 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbb8c0cd9e0 con 0x7fbb900685f0 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb95b27640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb7c0a1ba0 0x7fbb7c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb95326640 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.500+0000 7fbb94b25640 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb95b27640 1 -- 172.21.15.42:0/2811179479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 msgr2=0x55e6e5cbe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb95b27640 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb95b27640 1 -- 172.21.15.42:0/2811179479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 msgr2=0x55e6e5cc0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb95b27640 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb95b27640 1 -- 172.21.15.42:0/2811179479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6e5bdf060 con 0x55e6e5cb5250 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb94b25640 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb95b27640 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb5630 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbb88010a30 tx=0x7fbb88010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a479275b8a41b50 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb7b7fe640 1 -- 172.21.15.42:0/2811179479 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8801bbd0 con 0x55e6e5cb5250 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb7b7fe640 1 -- 172.21.15.42:0/2811179479 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb8801bd30 con 0x55e6e5cb5250 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb7b7fe640 1 -- 172.21.15.42:0/2811179479 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800e5b0 con 0x55e6e5cb5250 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.501+0000 7fbb95b27640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb7c0a1ba0 0x7fbb7c0a4060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb8801d000 tx=0x7fbb88019000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- 172.21.15.42:0/2811179479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 msgr2=0x55e6e5cb5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb5630 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbb88010a30 tx=0x7fbb88010ef0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- 172.21.15.42:0/2811179479 shutdown_connections 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2811179479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb5630 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- 172.21.15.42:0/2811179479 >> 172.21.15.42:0/2811179479 conn(0x55e6e5caf980 msgr2=0x55e6e5cafd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- 172.21.15.42:0/2811179479 shutdown_connections 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- 172.21.15.42:0/2811179479 wait complete. 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- start start 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 Processor -- start 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cbb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cbc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5c1d020 con 0x55e6e5cb5250 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e58f0430 con 0x55e6e5cbe750 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.502+0000 7fbb9a235740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e58ace00 con 0x55e6e5cb5b70 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb9a235740 10 client.0 _ll_register_callbacks cb 0x55e6e5bc4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cbb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cbb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52020/0 (socket says 172.21.15.42:52020) 2024-03-12T04:24:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 -- 172.21.15.42:0/1907342351 learned_addr learned my addr 172.21.15.42:0/1907342351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95b27640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cbc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb94b25640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 -- 172.21.15.42:0/1907342351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 msgr2=0x55e6e5cbbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 -- 172.21.15.42:0/1907342351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 msgr2=0x55e6e5cbc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cbc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 -- 172.21.15.42:0/1907342351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb880105e0 con 0x55e6e5cb5250 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb94b25640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95b27640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cbc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb95326640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cbb6b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c096a90 tx=0x7fbb8c00fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eaf6800a0f6717 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb78ff9640 1 -- 172.21.15.42:0/1907342351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c09a630 con 0x55e6e5cb5250 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb78ff9640 1 -- 172.21.15.42:0/1907342351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbb8c09e780 con 0x55e6e5cb5250 2024-03-12T04:24:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6e591c7e0 con 0x55e6e5cb5250 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb78ff9640 1 -- 172.21.15.42:0/1907342351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8c0a2dd0 con 0x55e6e5cb5250 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb9a235740 10 client.18009 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.503+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6e5c1b6a0 con 0x55e6e5cb5250 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb78ff9640 1 -- 172.21.15.42:0/1907342351 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e6e591c7e0 con 0x55e6e5cb5250 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb78ff9640 1 -- 172.21.15.42:0/1907342351 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbb8c016020 con 0x55e6e5cb5250 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb9a235740 10 client.18009 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb9a235740 10 client.18009 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb9a235740 20 client.18009 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb9a235740 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5cceab0 0x55e6e5cd0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb9a235740 4 client.18009 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6e5959720 con 0x55e6e5cceab0 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb94b25640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5cceab0 0x55e6e5cd0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb78ff9640 10 client.18009 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.504+0000 7fbb94b25640 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5cceab0 0x55e6e5cd0ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c890dbc71a59babe server_cookie=8f6de81d338e4e0c in_seq=0 out_seq=0 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb78ff9640 1 -- 172.21.15.42:0/1907342351 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6e5959720 con 0x55e6e5cceab0 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb78ff9640 10 client.18009 handle_command_reply: tid=0 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 client.18009 shutdown 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 msgr2=0x55e6e5cbb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cbb6b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c096a90 tx=0x7fbb8c00fb50 comp rx=0 tx=0).stop 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 shutdown_connections 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cbe750 0x55e6e5cbc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5cceab0 0x55e6e5cd0ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 --2- 172.21.15.42:0/1907342351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5250 0x55e6e5cbb6b0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb78ff9640 0 client.18009 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 >> 172.21.15.42:0/1907342351 conn(0x55e6e5caf980 msgr2=0x55e6e5cb16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 shutdown_connections 2024-03-12T04:24:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 1 -- 172.21.15.42:0/1907342351 wait complete. 2024-03-12T04:24:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.505+0000 7fbb9a235740 20 client.18009 trim_cache size 0 max 16384 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 Processor -- start 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 -- start start 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cbe750 0x55e6e5cb8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5ba2130 con 0x55e6e5cb5b70 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5ccb590 con 0x55e6e5cb5250 2024-03-12T04:24:16.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb9a235740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5cb4eb0 con 0x55e6e5cbe750 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52022/0 (socket says 172.21.15.42:52022) 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 -- 172.21.15.42:0/3952911745 learned_addr learned my addr 172.21.15.42:0/3952911745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb95326640 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 -- 172.21.15.42:0/3952911745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cbe750 msgr2=0x55e6e5cb8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cbe750 0x55e6e5cb8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 -- 172.21.15.42:0/3952911745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 msgr2=0x55e6e5cb1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 -- 172.21.15.42:0/3952911745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6e5bb7c20 con 0x55e6e5cb5b70 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.517+0000 7fbb94b25640 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbf50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005ab0 tx=0x7fbb80004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51876e0fdeadab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb7affd640 1 -- 172.21.15.42:0/3952911745 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80004040 con 0x55e6e5cb5b70 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb7affd640 1 -- 172.21.15.42:0/3952911745 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb8002de90 con 0x55e6e5cb5b70 2024-03-12T04:24:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb7affd640 1 -- 172.21.15.42:0/3952911745 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80003b50 con 0x55e6e5cb5b70 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 -- 172.21.15.42:0/3952911745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5b70 msgr2=0x55e6e5cbbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbf50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005ab0 tx=0x7fbb80004ae0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 -- 172.21.15.42:0/3952911745 shutdown_connections 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cbe750 0x55e6e5cb8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cbbf50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 --2- 172.21.15.42:0/3952911745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cb1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 -- 172.21.15.42:0/3952911745 >> 172.21.15.42:0/3952911745 conn(0x55e6e5caf980 msgr2=0x55e6e5cb69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 -- 172.21.15.42:0/3952911745 shutdown_connections 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.518+0000 7fbb9a235740 1 -- 172.21.15.42:0/3952911745 wait complete. 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 -- start start 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 Processor -- start 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cc5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cc3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5cc4da0 con 0x55e6e5cbe750 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5d24fb0 con 0x55e6e5cb5250 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e5d250f0 con 0x55e6e5cb5b70 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb9a235740 10 client.0 _ll_register_callbacks cb 0x7fbb880077c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52026/0 (socket says 172.21.15.42:52026) 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 -- 172.21.15.42:0/2958368076 learned_addr learned my addr 172.21.15.42:0/2958368076 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb94b25640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cc3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95326640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cc5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 -- 172.21.15.42:0/2958368076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 msgr2=0x55e6e5cc3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cc3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 -- 172.21.15.42:0/2958368076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 msgr2=0x55e6e5cc5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cc5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95b27640 1 -- 172.21.15.42:0/2958368076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9010e880 con 0x55e6e5cbe750 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.519+0000 7fbb95326640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cc5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb95b27640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc46d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbb88004a80 tx=0x7fbb88007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d391b6f2b8e698c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb78ff9640 1 -- 172.21.15.42:0/2958368076 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8802a040 con 0x55e6e5cbe750 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb78ff9640 1 -- 172.21.15.42:0/2958368076 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbb8800e550 con 0x55e6e5cbe750 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb8801bd30 con 0x55e6e5cbe750 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb78ff9640 1 -- 172.21.15.42:0/2958368076 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8800def0 con 0x55e6e5cbe750 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 10 client.18015 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6e5bdd8c0 con 0x55e6e5cbe750 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb78ff9640 1 -- 172.21.15.42:0/2958368076 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb8803c020 con 0x55e6e5cbe750 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb78ff9640 1 -- 172.21.15.42:0/2958368076 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbb8800c030 con 0x55e6e5cbe750 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 10 client.18015 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 10 client.18015 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 20 client.18015 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5d268e0 0x55e6e5d28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 4 client.18015 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:16.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6e59a8890 con 0x55e6e5d268e0 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.520+0000 7fbb95326640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5d268e0 0x55e6e5d28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb78ff9640 10 client.18015 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb95326640 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5d268e0 0x55e6e5d28d00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4da183f11735cce4 server_cookie=77bb5872fa6c2d4d in_seq=0 out_seq=0 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb78ff9640 1 -- 172.21.15.42:0/2958368076 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6e59a8890 con 0x55e6e5d268e0 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb78ff9640 10 client.18015 handle_command_reply: tid=0 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 client.18015 shutdown 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cbe750 msgr2=0x55e6e5cc46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc46d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbb88004a80 tx=0x7fbb88007be0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 shutdown_connections 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x55e6e5d268e0 0x55e6e5d28d00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6e5cbe750 0x55e6e5cc46d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6e5cb5b70 0x55e6e5cc3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb78ff9640 0 client.18015 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 --2- 172.21.15.42:0/2958368076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e6e5cb5250 0x55e6e5cc5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 >> 172.21.15.42:0/2958368076 conn(0x55e6e5caf980 msgr2=0x55e6e5cae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 shutdown_connections 2024-03-12T04:24:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 1 -- 172.21.15.42:0/2958368076 wait complete. 2024-03-12T04:24:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.521+0000 7fbb9a235740 20 client.18015 trim_cache size 0 max 16384 2024-03-12T04:24:16.531 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb7c0a1ba0 msgr2=0x7fbb7c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb7c0a1ba0 0x7fbb7c0a4060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb8801d000 tx=0x7fbb88019000 comp rx=0 tx=0).stop 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb900685f0 msgr2=0x7fbb90100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb900685f0 0x7fbb90100040 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00c920 tx=0x7fbb8c00cde0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 shutdown_connections 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbb7c0a1ba0 0x7fbb7c0a4060 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb90100ac0 0x7fbb901a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb901064c0 0x7fbb90100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 --2- 172.21.15.42:0/486723093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb900685f0 0x7fbb90100040 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 >> 172.21.15.42:0/486723093 conn(0x7fbb90077180 msgr2=0x7fbb900fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 shutdown_connections 2024-03-12T04:24:16.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.522+0000 7fbb9758a640 1 -- 172.21.15.42:0/486723093 wait complete. 2024-03-12T04:24:16.534 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:16 smithi042 ceph-mon[32260]: osdmap e96: 12 total, 12 up, 12 in 2024-03-12T04:24:16.535 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:16.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 -- 172.21.15.42:0/1778327992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c80fff70 msgr2=0x7f68c8100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/1778327992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c80fff70 0x7f68c8100350 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009900 tx=0x7f68c402bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 -- 172.21.15.42:0/1778327992 shutdown_connections 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/1778327992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68c810db20 0x7f68c810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/1778327992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c8100920 0x7f68c810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/1778327992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c80fff70 0x7f68c8100350 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 -- 172.21.15.42:0/1778327992 >> 172.21.15.42:0/1778327992 conn(0x7f68c80771e0 msgr2=0x7f68c80775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 -- 172.21.15.42:0/1778327992 shutdown_connections 2024-03-12T04:24:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.637+0000 7f68cf3bc640 1 -- 172.21.15.42:0/1778327992 wait complete. 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 Processor -- start 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 -- start start 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80fff70 0x7f68c819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68c8100920 0x7f68c819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c810db20 0x7f68c8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8199e20 con 0x7f68c810db20 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8199f90 con 0x7f68c80fff70 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cf3bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c819a150 con 0x7f68c8100920 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c810db20 0x7f68c8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c810db20 0x7f68c8199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52044/0 (socket says 172.21.15.42:52044) 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 -- 172.21.15.42:0/159706676 learned_addr learned my addr 172.21.15.42:0/159706676 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cc957640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68c8100920 0x7f68c819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd158640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80fff70 0x7f68c819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 -- 172.21.15.42:0/159706676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68c8100920 msgr2=0x7f68c819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68c8100920 0x7f68c819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 -- 172.21.15.42:0/159706676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80fff70 msgr2=0x7f68c819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80fff70 0x7f68c819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.638+0000 7f68cd959640 1 -- 172.21.15.42:0/159706676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4009560 con 0x7f68c810db20 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.639+0000 7f68cd959640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c810db20 0x7f68c8199780 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f68c000cab0 tx=0x7f68c000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.639+0000 7f68be7fc640 1 -- 172.21.15.42:0/159706676 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c0007e10 con 0x7f68c810db20 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.639+0000 7f68be7fc640 1 -- 172.21.15.42:0/159706676 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68c000f900 con 0x7f68c810db20 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.639+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c819a430 con 0x7f68c810db20 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.639+0000 7f68be7fc640 1 -- 172.21.15.42:0/159706676 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c0018970 con 0x7f68c810db20 2024-03-12T04:24:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.639+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c81a6640 con 0x7f68c810db20 2024-03-12T04:24:16.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.641+0000 7f68be7fc640 1 -- 172.21.15.42:0/159706676 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f68c000fa70 con 0x7f68c810db20 2024-03-12T04:24:16.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.641+0000 7f68d1f8f740 1 Processor -- start 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.641+0000 7f68d1f8f740 1 -- start start 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68be7fc640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f68b40a1c20 0x7f68b40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4e7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b4f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b4a2960 con 0x55af9b4f0750 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b456310 con 0x55af9b4e7b70 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b453840 con 0x55af9b4e7250 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cc957640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f68b40a1c20 0x7f68b40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68be7fc640 1 -- 172.21.15.42:0/159706676 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f68c00cda80 con 0x7f68c810db20 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b4f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b4f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52054/0 (socket says 172.21.15.42:52054) 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 -- 172.21.15.42:0/2657826164 learned_addr learned my addr 172.21.15.42:0/2657826164 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd158640 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cc957640 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 -- 172.21.15.42:0/2657826164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 msgr2=0x55af9b4e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 -- 172.21.15.42:0/2657826164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 msgr2=0x55af9b4f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 -- 172.21.15.42:0/2657826164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af9b411060 con 0x55af9b4f0750 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cc957640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f68b40a1c20 0x7f68b40a40e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68b80058a0 tx=0x7f68b800b770 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68cd959640 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b4f2b40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f68c00a28e0 tx=0x7f68c0099fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f7a9b0a96cfa67 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.642+0000 7f68b37fe640 1 -- 172.21.15.42:0/2657826164 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c009a670 con 0x55af9b4f0750 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68b37fe640 1 -- 172.21.15.42:0/2657826164 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68c009e7c0 con 0x55af9b4f0750 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68b37fe640 1 -- 172.21.15.42:0/2657826164 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00850f0 con 0x55af9b4f0750 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2657826164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 msgr2=0x55af9b4f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b4f2b40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f68c00a28e0 tx=0x7f68c0099fd0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2657826164 shutdown_connections 2024-03-12T04:24:16.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b4f2b40 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2657826164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2657826164 >> 172.21.15.42:0/2657826164 conn(0x55af9b4e1980 msgr2=0x55af9b4e1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2657826164 shutdown_connections 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.643+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2657826164 wait complete. 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 -- start start 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 Processor -- start 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4ed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4f0750 0x55af9b4ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b44f020 con 0x55af9b4e7b70 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b122430 con 0x55af9b4f0750 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b0dee00 con 0x55af9b4e7250 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68d1f8f740 10 client.0 _ll_register_callbacks cb 0x55af9b3f6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cc957640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cc957640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52058/0 (socket says 172.21.15.42:52058) 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cc957640 1 -- 172.21.15.42:0/3695375382 learned_addr learned my addr 172.21.15.42:0/3695375382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cd158640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cd959640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4f0750 0x55af9b4ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cc957640 1 -- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 msgr2=0x55af9b4ed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cc957640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4ed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cc957640 1 -- 172.21.15.42:0/3695375382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4f0750 msgr2=0x55af9b4ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.644+0000 7f68cc957640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4f0750 0x55af9b4ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68cc957640 1 -- 172.21.15.42:0/3695375382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c81a6640 con 0x55af9b4e7b70 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68cd959640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4f0750 0x55af9b4ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68cc957640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edc60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f68b801e6c0 tx=0x7f68b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a80cada11088dda server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68b0ff9640 1 -- 172.21.15.42:0/3695375382 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b801c840 con 0x55af9b4e7b70 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68b0ff9640 1 -- 172.21.15.42:0/3695375382 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f68b801c9a0 con 0x55af9b4e7b70 2024-03-12T04:24:16.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af9b14e7e0 con 0x55af9b4e7b70 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68b0ff9640 1 -- 172.21.15.42:0/3695375382 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68b801cb60 con 0x55af9b4e7b70 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 10 client.18027 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af9b44d6a0 con 0x55af9b4e7b70 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68b0ff9640 1 -- 172.21.15.42:0/3695375382 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68b8033020 con 0x55af9b4e7b70 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68b0ff9640 1 -- 172.21.15.42:0/3695375382 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f68b800f3c0 con 0x55af9b4e7b70 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 10 client.18027 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 10 client.18027 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 20 client.18027 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b500ba0 0x55af9b502fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 4 client.18027 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.645+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af9b18b720 con 0x55af9b500ba0 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.646+0000 7f68cd158640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b500ba0 0x55af9b502fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.646+0000 7f68cd158640 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b500ba0 0x55af9b502fc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9332ea839ae815e server_cookie=2a94ee8c851f8917 in_seq=0 out_seq=0 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.646+0000 7f68b0ff9640 10 client.18027 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68b0ff9640 1 -- 172.21.15.42:0/3695375382 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af9b18b720 con 0x55af9b500ba0 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68b0ff9640 10 client.18027 handle_command_reply: tid=0 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 client.18027 shutdown 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 msgr2=0x55af9b4edc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edc60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f68b801e6c0 tx=0x7f68b8004730 comp rx=0 tx=0).stop 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 shutdown_connections 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b500ba0 0x55af9b502fc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4f0750 0x55af9b4ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edc60 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/3695375382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68b0ff9640 0 client.18027 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 >> 172.21.15.42:0/3695375382 conn(0x55af9b4e1980 msgr2=0x55af9b4e36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 shutdown_connections 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 1 -- 172.21.15.42:0/3695375382 wait complete. 2024-03-12T04:24:16.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.647+0000 7f68d1f8f740 20 client.18027 trim_cache size 0 max 16384 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 Processor -- start 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 -- start start 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7250 0x55af9b4efe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4f0750 0x55af9b4ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b3d4130 con 0x55af9b4e7b70 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b508190 con 0x55af9b4e7250 2024-03-12T04:24:16.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b4e6c10 con 0x55af9b4f0750 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52060/0 (socket says 172.21.15.42:52060) 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 -- 172.21.15.42:0/2861269345 learned_addr learned my addr 172.21.15.42:0/2861269345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cd959640 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4f0750 0x55af9b4ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 -- 172.21.15.42:0/2861269345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4f0750 msgr2=0x55af9b4ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4f0750 0x55af9b4ea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 -- 172.21.15.42:0/2861269345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7250 msgr2=0x55af9b4efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7250 0x55af9b4efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.659+0000 7f68cc957640 1 -- 172.21.15.42:0/2861269345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af9b3e9c20 con 0x55af9b4e7b70 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68cc957640 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edfc0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f68b800f820 tx=0x7f68b8012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4923638e438959 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68b2ffd640 1 -- 172.21.15.42:0/2861269345 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800be40 con 0x55af9b4e7b70 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68b2ffd640 1 -- 172.21.15.42:0/2861269345 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68b8033050 con 0x55af9b4e7b70 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68b2ffd640 1 -- 172.21.15.42:0/2861269345 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b801cec0 con 0x55af9b4e7b70 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2861269345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 msgr2=0x55af9b4edfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edfc0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f68b800f820 tx=0x7f68b8012f90 comp rx=0 tx=0).stop 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2861269345 shutdown_connections 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4f0750 0x55af9b4ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4e7b70 0x55af9b4edfc0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/2861269345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7250 0x55af9b4efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2861269345 >> 172.21.15.42:0/2861269345 conn(0x55af9b4e1980 msgr2=0x55af9b4e8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2861269345 shutdown_connections 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.660+0000 7f68d1f8f740 1 -- 172.21.15.42:0/2861269345 wait complete. 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 -- start start 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 Processor -- start 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4f6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b557090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b4f72b0 con 0x55af9b4f0750 2024-03-12T04:24:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b5575d0 con 0x55af9b4e7b70 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9b557710 con 0x55af9b4e7250 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68d1f8f740 10 client.0 _ll_register_callbacks cb 0x7f68c000a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b557090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b557090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52070/0 (socket says 172.21.15.42:52070) 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 -- 172.21.15.42:0/1679041774 learned_addr learned my addr 172.21.15.42:0/1679041774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd158640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cc957640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 -- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 msgr2=0x55af9b4f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 -- 172.21.15.42:0/1679041774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 msgr2=0x55af9b4f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cd959640 1 -- 172.21.15.42:0/1679041774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af9b44d6a0 con 0x55af9b4f0750 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.661+0000 7f68cc957640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68cd959640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b557090 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f68c00076d0 tx=0x7f68c009aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89662f5981c6d4da server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68b0ff9640 1 -- 172.21.15.42:0/1679041774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c009a670 con 0x55af9b4f0750 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68b0ff9640 1 -- 172.21.15.42:0/1679041774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f68c00850f0 con 0x55af9b4f0750 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68c009e7c0 con 0x55af9b4f0750 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68b0ff9640 1 -- 172.21.15.42:0/1679041774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68c00852b0 con 0x55af9b4f0750 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 10 client.18033 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af9b40f8c0 con 0x55af9b4f0750 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68b0ff9640 1 -- 172.21.15.42:0/1679041774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68c009e7c0 con 0x55af9b4f0750 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68b0ff9640 1 -- 172.21.15.42:0/1679041774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f68c0016020 con 0x55af9b4f0750 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 10 client.18033 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 10 client.18033 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 20 client.18033 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b558bc0 0x55af9b55afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 4 client.18033 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55af9b1da890 con 0x55af9b558bc0 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.662+0000 7f68cd158640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b558bc0 0x55af9b55afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68cd158640 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b558bc0 0x55af9b55afe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=245378ebb7021f22 server_cookie=9fd8954408ffa2cf in_seq=0 out_seq=0 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68b0ff9640 10 client.18033 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68b0ff9640 1 -- 172.21.15.42:0/1679041774 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55af9b1da890 con 0x55af9b558bc0 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68b0ff9640 10 client.18033 handle_command_reply: tid=0 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 client.18033 shutdown 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 msgr2=0x55af9b557090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b557090 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f68c00076d0 tx=0x7f68c009aec0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 shutdown_connections 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x55af9b558bc0 0x55af9b55afe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55af9b4f0750 0x55af9b557090 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af9b4e7b70 0x55af9b4f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68b0ff9640 0 client.18033 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:16.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 --2- 172.21.15.42:0/1679041774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af9b4e7250 0x55af9b4f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 >> 172.21.15.42:0/1679041774 conn(0x55af9b4e1980 msgr2=0x55af9b4e02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 shutdown_connections 2024-03-12T04:24:16.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 1 -- 172.21.15.42:0/1679041774 wait complete. 2024-03-12T04:24:16.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.663+0000 7f68d1f8f740 20 client.18033 trim_cache size 0 max 16384 2024-03-12T04:24:16.673 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f68b40a1c20 msgr2=0x7f68b40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f68b40a1c20 0x7f68b40a40e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68b80058a0 tx=0x7f68b800b770 comp rx=0 tx=0).stop 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c810db20 msgr2=0x7f68c8199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c810db20 0x7f68c8199780 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f68c000cab0 tx=0x7f68c000cf70 comp rx=0 tx=0).stop 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 shutdown_connections 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f68b40a1c20 0x7f68b40a40e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68c810db20 0x7f68c8199780 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68c8100920 0x7f68c819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 --2- 172.21.15.42:0/159706676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80fff70 0x7f68c819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 >> 172.21.15.42:0/159706676 conn(0x7f68c80771e0 msgr2=0x7f68c810f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 shutdown_connections 2024-03-12T04:24:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.664+0000 7f68cf3bc640 1 -- 172.21.15.42:0/159706676 wait complete. 2024-03-12T04:24:16.677 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:16.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:24:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3378917092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 msgr2=0x7f5fd410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3378917092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd410e020 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009cd0 tx=0x7f5fcc02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3378917092 shutdown_connections 2024-03-12T04:24:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3378917092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd410e020 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3378917092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd4101d30 0x7f5fd4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3378917092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd4101380 0x7f5fd4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3378917092 >> 172.21.15.42:0/3378917092 conn(0x7f5fd4078580 msgr2=0x7f5fd4078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3378917092 shutdown_connections 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.779+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3378917092 wait complete. 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 Processor -- start 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 -- start start 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd4101380 0x7f5fd4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd4073f00 0x7f5fd41a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41a5e70 con 0x7f5fd4106450 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41a5fe0 con 0x7f5fd4101380 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd97ca640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41a61a0 con 0x7f5fd4073f00 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd40739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52082/0 (socket says 172.21.15.42:52082) 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3563332909 learned_addr learned my addr 172.21.15.42:0/3563332909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd4073f00 0x7f5fd41a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd4101380 0x7f5fd4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3563332909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd4073f00 msgr2=0x7f5fd41a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.780+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd4073f00 0x7f5fd41a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3563332909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd4101380 msgr2=0x7f5fd4073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd4101380 0x7f5fd4073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3563332909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009900 con 0x7f5fd4106450 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd40739c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c970 tx=0x7f5fbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fd0ff9640 1 -- 172.21.15.42:0/3563332909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc007bb0 con 0x7f5fd4106450 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fd0ff9640 1 -- 172.21.15.42:0/3563332909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fbc007d10 con 0x7f5fd4106450 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fd0ff9640 1 -- 172.21.15.42:0/3563332909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00f450 con 0x7f5fd4106450 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd41a6480 con 0x7f5fd4106450 2024-03-12T04:24:16.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.781+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd41a69e0 con 0x7f5fd4106450 2024-03-12T04:24:16.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.783+0000 7f5fd0ff9640 1 -- 172.21.15.42:0/3563332909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f5fbc004eb0 con 0x7f5fd4106450 2024-03-12T04:24:16.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.783+0000 7f5fdc39d740 1 Processor -- start 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.783+0000 7f5fdc39d740 1 -- start start 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f4f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f58210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f5ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fd0ff9640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fc00a1c50 0x7f5fc00a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5f0a960 con 0x55f4c5f58750 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5ebe310 con 0x55f4c5f4fb70 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5ebb840 con 0x55f4c5f4f250 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f5ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fc00a1c50 0x7f5fc00a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f5ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52084/0 (socket says 172.21.15.42:52084) 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3739894924 learned_addr learned my addr 172.21.15.42:0/3739894924 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fd0ff9640 1 -- 172.21.15.42:0/3563332909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5fbc0cd2b0 con 0x7f5fd4106450 2024-03-12T04:24:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f58210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f4f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3739894924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 msgr2=0x55f4c5f4f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f4f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3739894924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 msgr2=0x55f4c5f58210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f58210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 -- 172.21.15.42:0/3739894924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4c5e79060 con 0x55f4c5f58750 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fc00a1c50 0x7f5fc00a4110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005cc0 tx=0x7f5fcc034520 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.784+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f5ab40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc099f40 tx=0x7f5fbc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238dda6d5dc79246 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fc97fa640 1 -- 172.21.15.42:0/3739894924 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0046c0 con 0x55f4c5f58750 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fc97fa640 1 -- 172.21.15.42:0/3739894924 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fbc099fb0 con 0x55f4c5f58750 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fc97fa640 1 -- 172.21.15.42:0/3739894924 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc096620 con 0x55f4c5f58750 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3739894924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 msgr2=0x55f4c5f5ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f5ab40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc099f40 tx=0x7f5fbc000bc0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3739894924 shutdown_connections 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f5ab40 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f58210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3739894924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f4f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3739894924 >> 172.21.15.42:0/3739894924 conn(0x55f4c5f49980 msgr2=0x55f4c5f49d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3739894924 shutdown_connections 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.785+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3739894924 wait complete. 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 -- start start 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 Processor -- start 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f4f250 0x55f4c5f557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f55ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f58750 0x55f4c5f56220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5eb7020 con 0x55f4c5f4f250 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5b8a430 con 0x55f4c5f4fb70 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5b46e00 con 0x55f4c5f58750 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fdc39d740 10 client.0 _ll_register_callbacks cb 0x55f4c5e5ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f4f250 0x55f4c5f557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f4f250 0x55f4c5f557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52098/0 (socket says 172.21.15.42:52098) 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 -- 172.21.15.42:0/3153657236 learned_addr learned my addr 172.21.15.42:0/3153657236 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f58750 0x55f4c5f56220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f55ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 -- 172.21.15.42:0/3153657236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f58750 msgr2=0x55f4c5f56220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f58750 0x55f4c5f56220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 -- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 msgr2=0x55f4c5f55ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f55ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.786+0000 7f5fd2ffd640 1 -- 172.21.15.42:0/3153657236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd41a69e0 con 0x55f4c5f4f250 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f4f250 0x55f4c5f557a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4010850 tx=0x7f5fc4010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb3d4e28feed683 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fba7fc640 1 -- 172.21.15.42:0/3153657236 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4015070 con 0x55f4c5f4f250 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fba7fc640 1 -- 172.21.15.42:0/3153657236 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5fc400fb90 con 0x55f4c5f4f250 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4c5bb67e0 con 0x55f4c5f4f250 2024-03-12T04:24:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fba7fc640 1 -- 172.21.15.42:0/3153657236 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fc400fd90 con 0x55f4c5f4f250 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 10 client.18045 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f4c5eb56a0 con 0x55f4c5f4f250 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fba7fc640 1 -- 172.21.15.42:0/3153657236 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fc4026020 con 0x55f4c5f4f250 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fba7fc640 1 -- 172.21.15.42:0/3153657236 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fc4018400 con 0x55f4c5f4f250 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 10 client.18045 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 10 client.18045 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 20 client.18045 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5f68c20 0x55f4c5f6b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 4 client.18045 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.787+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4c5bf3720 con 0x55f4c5f68c20 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.788+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5f68c20 0x55f4c5f6b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.788+0000 7f5fba7fc640 10 client.18045 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.788+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5f68c20 0x55f4c5f6b040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=717f77de89313e7b server_cookie=d9da92a7523d8adf in_seq=0 out_seq=0 2024-03-12T04:24:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fba7fc640 1 -- 172.21.15.42:0/3153657236 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4c5bf3720 con 0x55f4c5f68c20 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fba7fc640 10 client.18045 handle_command_reply: tid=0 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 client.18045 shutdown 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f4f250 msgr2=0x55f4c5f557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f4f250 0x55f4c5f557a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4010850 tx=0x7f5fc4010d10 comp rx=0 tx=0).stop 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 shutdown_connections 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5f68c20 0x55f4c5f6b040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f58750 0x55f4c5f56220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f55ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fba7fc640 0 client.18045 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/3153657236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f4f250 0x55f4c5f557a0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 >> 172.21.15.42:0/3153657236 conn(0x55f4c5f49980 msgr2=0x55f4c5f4b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 shutdown_connections 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 1 -- 172.21.15.42:0/3153657236 wait complete. 2024-03-12T04:24:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.789+0000 7f5fdc39d740 20 client.18045 trim_cache size 0 max 16384 2024-03-12T04:24:16.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 Processor -- start 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 -- start start 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4f250 0x55f4c5f57ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f56040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f52a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5e3c130 con 0x55f4c5f58750 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5f70290 con 0x55f4c5f4f250 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5f45080 con 0x55f4c5f4fb70 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f52a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f52a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52100/0 (socket says 172.21.15.42:52100) 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fd37fe640 1 -- 172.21.15.42:0/2838980749 learned_addr learned my addr 172.21.15.42:0/2838980749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4f250 0x55f4c5f57ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fd37fe640 1 -- 172.21.15.42:0/2838980749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4fb70 msgr2=0x55f4c5f56040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.801+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f56040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fd37fe640 1 -- 172.21.15.42:0/2838980749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4f250 msgr2=0x55f4c5f57ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4f250 0x55f4c5f57ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fd37fe640 1 -- 172.21.15.42:0/2838980749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4c5e51c20 con 0x55f4c5f58750 2024-03-12T04:24:16.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4f250 0x55f4c5f57ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f52a00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009cd0 tx=0x7f5fcc046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166f31c0e29a7d74 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fc8ff9640 1 -- 172.21.15.42:0/2838980749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc034ba0 con 0x55f4c5f58750 2024-03-12T04:24:16.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fc8ff9640 1 -- 172.21.15.42:0/2838980749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fcc032a80 con 0x55f4c5f58750 2024-03-12T04:24:16.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fc8ff9640 1 -- 172.21.15.42:0/2838980749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0042a0 con 0x55f4c5f58750 2024-03-12T04:24:16.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fdc39d740 1 -- 172.21.15.42:0/2838980749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 msgr2=0x55f4c5f52a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.802+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f52a00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009cd0 tx=0x7f5fcc046000 comp rx=0 tx=0).stop 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- 172.21.15.42:0/2838980749 shutdown_connections 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5f52a00 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f56040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/2838980749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4f250 0x55f4c5f57ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- 172.21.15.42:0/2838980749 >> 172.21.15.42:0/2838980749 conn(0x55f4c5f49980 msgr2=0x55f4c5f50b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- 172.21.15.42:0/2838980749 shutdown_connections 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- 172.21.15.42:0/2838980749 wait complete. 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- start start 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 Processor -- start 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f5e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5fb9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5fb97d0 con 0x55f4c5f58750 2024-03-12T04:24:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5fb9940 con 0x55f4c5f4fb70 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c5fb9b00 con 0x55f4c5f4f250 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fdc39d740 10 client.0 _ll_register_callbacks cb 0x7f5fbc096290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5fb9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5fb9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52104/0 (socket says 172.21.15.42:52104) 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fd37fe640 1 -- 172.21.15.42:0/242148727 learned_addr learned my addr 172.21.15.42:0/242148727 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fcbfff640 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.803+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f5e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fd37fe640 1 -- 172.21.15.42:0/242148727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 msgr2=0x55f4c5f5e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f5e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fd37fe640 1 -- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 msgr2=0x55f4c5f5eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f5eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fd37fe640 1 -- 172.21.15.42:0/242148727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc048040 con 0x55f4c5f58750 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fd37fe640 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5fb9290 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0098d0 tx=0x7f5fcc0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b02c9a79c14d86 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fba7fc640 1 -- 172.21.15.42:0/242148727 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc003f40 con 0x55f4c5f58750 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fba7fc640 1 -- 172.21.15.42:0/242148727 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5fcc0040a0 con 0x55f4c5f58750 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fbc099fb0 con 0x55f4c5f58750 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fba7fc640 1 -- 172.21.15.42:0/242148727 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fcc004260 con 0x55f4c5f58750 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fdc39d740 10 client.18051 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f4c5e778c0 con 0x55f4c5f58750 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.804+0000 7f5fba7fc640 1 -- 172.21.15.42:0/242148727 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fbc099fb0 con 0x55f4c5f58750 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fba7fc640 1 -- 172.21.15.42:0/242148727 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fcc05aad0 con 0x55f4c5f58750 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fdc39d740 10 client.18051 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fdc39d740 10 client.18051 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fdc39d740 20 client.18051 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5fc0e60 0x55f4c5fc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fdc39d740 4 client.18051 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f4c5c42890 con 0x55f4c5fc0e60 2024-03-12T04:24:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5fc0e60 0x55f4c5fc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fd2ffd640 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5fc0e60 0x55f4c5fc3280 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abff5c29e3fcdfc2 server_cookie=b2be44e288429582 in_seq=0 out_seq=0 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fba7fc640 10 client.18051 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fba7fc640 1 -- 172.21.15.42:0/242148727 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f4c5c42890 con 0x55f4c5fc0e60 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fba7fc640 10 client.18051 handle_command_reply: tid=0 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.805+0000 7f5fdc39d740 1 client.18051 shutdown 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 msgr2=0x55f4c5fb9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5fb9290 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0098d0 tx=0x7f5fcc0039f0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 shutdown_connections 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f4c5fc0e60 0x55f4c5fc3280 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4c5f58750 0x55f4c5fb9290 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4c5f4fb70 0x55f4c5f5eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 --2- 172.21.15.42:0/242148727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4c5f4f250 0x55f4c5f5e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fba7fc640 0 client.18051 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 >> 172.21.15.42:0/242148727 conn(0x55f4c5f49980 msgr2=0x55f4c5f482f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 shutdown_connections 2024-03-12T04:24:16.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 1 -- 172.21.15.42:0/242148727 wait complete. 2024-03-12T04:24:16.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fdc39d740 20 client.18051 trim_cache size 0 max 16384 2024-03-12T04:24:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fc00a1c50 msgr2=0x7f5fc00a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fc00a1c50 0x7f5fc00a4110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005cc0 tx=0x7f5fcc034520 comp rx=0 tx=0).stop 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 msgr2=0x7f5fd40739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd40739c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c970 tx=0x7f5fbc00ce30 comp rx=0 tx=0).stop 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 shutdown_connections 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f5fc00a1c50 0x7f5fc00a4110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd4073f00 0x7f5fd41a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4106450 0x7f5fd40739c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 --2- 172.21.15.42:0/3563332909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd4101380 0x7f5fd4073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 >> 172.21.15.42:0/3563332909 conn(0x7f5fd4078580 msgr2=0x7f5fd40ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 shutdown_connections 2024-03-12T04:24:16.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.806+0000 7f5fd97ca640 1 -- 172.21.15.42:0/3563332909 wait complete. 2024-03-12T04:24:16.820 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:16.820 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:24:16.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 -- 172.21.15.42:0/1305966160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c100260 msgr2=0x7f819c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 --2- 172.21.15.42:0/1305966160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c100260 0x7f819c100640 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8194009930 tx=0x7f819402bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 -- 172.21.15.42:0/1305966160 shutdown_connections 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 --2- 172.21.15.42:0/1305966160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c105330 0x7f819c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 --2- 172.21.15.42:0/1305966160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c100c10 0x7f819c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 --2- 172.21.15.42:0/1305966160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c100260 0x7f819c100640 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 -- 172.21.15.42:0/1305966160 >> 172.21.15.42:0/1305966160 conn(0x7f819c077180 msgr2=0x7f819c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 -- 172.21.15.42:0/1305966160 shutdown_connections 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.921+0000 7f81a182e640 1 -- 172.21.15.42:0/1305966160 wait complete. 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 Processor -- start 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 -- start start 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c100260 0x7f819c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c105330 0x7f819c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c19bfc0 0x7f819c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c19c9b0 con 0x7f819c105330 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c19cb20 con 0x7f819c100260 2024-03-12T04:24:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f81a182e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c19cc90 con 0x7f819c19bfc0 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f819a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c105330 0x7f819c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f819a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c105330 0x7f819c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52120/0 (socket says 172.21.15.42:52120) 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.922+0000 7f819a7fc640 1 -- 172.21.15.42:0/3942338346 learned_addr learned my addr 172.21.15.42:0/3942338346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819affd640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c100260 0x7f819c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819b7fe640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c19bfc0 0x7f819c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819a7fc640 1 -- 172.21.15.42:0/3942338346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c19bfc0 msgr2=0x7f819c19c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819a7fc640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c19bfc0 0x7f819c19c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819a7fc640 1 -- 172.21.15.42:0/3942338346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c100260 msgr2=0x7f819c197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819a7fc640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c100260 0x7f819c197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819a7fc640 1 -- 172.21.15.42:0/3942338346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8194009560 con 0x7f819c105330 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819affd640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c100260 0x7f819c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f819a7fc640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c105330 0x7f819c197c40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f818400d770 tx=0x7f818400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f81a082c640 1 -- 172.21.15.42:0/3942338346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184009c30 con 0x7f819c105330 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c1a4270 con 0x7f819c105330 2024-03-12T04:24:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f81a082c640 1 -- 172.21.15.42:0/3942338346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f818400de80 con 0x7f819c105330 2024-03-12T04:24:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f81a082c640 1 -- 172.21.15.42:0/3942338346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400f600 con 0x7f819c105330 2024-03-12T04:24:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.923+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c1a4720 con 0x7f819c105330 2024-03-12T04:24:16.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.925+0000 7f81a082c640 1 -- 172.21.15.42:0/3942338346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8184010460 con 0x7f819c105330 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.925+0000 7f81a4403740 1 Processor -- start 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a4403740 1 -- start start 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a4403740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f363630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a4403740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a4403740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f36c750 0x56457f36eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a4403740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f31e960 con 0x56457f363250 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a4403740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f2d2310 con 0x56457f363b70 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a4403740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f2cf840 con 0x56457f36c750 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a082c640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f81880a1ba0 0x7f81880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f363630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f363630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52126/0 (socket says 172.21.15.42:52126) 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 -- 172.21.15.42:0/2051170011 learned_addr learned my addr 172.21.15.42:0/2051170011 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f81880a1ba0 0x7f81880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819affd640 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f81a082c640 1 -- 172.21.15.42:0/3942338346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f81840cd680 con 0x7f819c105330 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819a7fc640 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f36c750 0x56457f36eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 -- 172.21.15.42:0/2051170011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f36c750 msgr2=0x56457f36eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f36c750 0x56457f36eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 -- 172.21.15.42:0/2051170011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 msgr2=0x56457f36c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 -- 172.21.15.42:0/2051170011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56457f28d060 con 0x56457f363250 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.926+0000 7f819b7fe640 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f363630 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f818c012c50 tx=0x7f818c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bf8712777d41d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f819b7fe640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f81880a1ba0 0x7f81880a4060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f818c010c40 tx=0x7f818c017000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f8191ffb640 1 -- 172.21.15.42:0/2051170011 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c012e00 con 0x56457f363250 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f8191ffb640 1 -- 172.21.15.42:0/2051170011 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f818c009c00 con 0x56457f363250 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f8191ffb640 1 -- 172.21.15.42:0/2051170011 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c005020 con 0x56457f363250 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 -- 172.21.15.42:0/2051170011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 msgr2=0x56457f363630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f363630 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f818c012c50 tx=0x7f818c010740 comp rx=0 tx=0).stop 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 -- 172.21.15.42:0/2051170011 shutdown_connections 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f36c750 0x56457f36eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 --2- 172.21.15.42:0/2051170011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f363630 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 -- 172.21.15.42:0/2051170011 >> 172.21.15.42:0/2051170011 conn(0x56457f35d980 msgr2=0x56457f35dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 -- 172.21.15.42:0/2051170011 shutdown_connections 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.927+0000 7f81a4403740 1 -- 172.21.15.42:0/2051170011 wait complete. 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 -- start start 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 Processor -- start 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f3697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363b70 0x56457f369d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f36c750 0x56457f36a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f2cb020 con 0x56457f363250 2024-03-12T04:24:16.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457ef9e430 con 0x56457f36c750 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457ef5ae00 con 0x56457f363b70 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f81a4403740 10 client.0 _ll_register_callbacks cb 0x56457f272cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f819affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f3697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f819affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f3697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52128/0 (socket says 172.21.15.42:52128) 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f819affd640 1 -- 172.21.15.42:0/3279710482 learned_addr learned my addr 172.21.15.42:0/3279710482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f819a7fc640 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363b70 0x56457f369d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f819b7fe640 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f36c750 0x56457f36a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f819affd640 1 -- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363b70 msgr2=0x56457f369d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.928+0000 7f819affd640 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363b70 0x56457f369d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f819affd640 1 -- 172.21.15.42:0/3279710482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f36c750 msgr2=0x56457f36a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f819affd640 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f36c750 0x56457f36a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f819affd640 1 -- 172.21.15.42:0/3279710482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c01a040 con 0x56457f363250 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f819affd640 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f3697d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8194036040 tx=0x7f819402c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43bcce4d3258153 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f817affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f817affd640 1 -- 172.21.15.42:0/3279710482 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194004450 con 0x56457f363250 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f817affd640 1 -- 172.21.15.42:0/3279710482 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f81940035f0 con 0x56457f363250 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56457efca7e0 con 0x56457f363250 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f817affd640 1 -- 172.21.15.42:0/3279710482 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81940037b0 con 0x56457f363250 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 10 client.18063 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56457f2c96a0 con 0x56457f363250 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f817affd640 1 -- 172.21.15.42:0/3279710482 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56457efca7e0 con 0x56457f363250 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f817affd640 1 -- 172.21.15.42:0/3279710482 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f819403c040 con 0x56457f363250 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 10 client.18063 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 10 client.18063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 20 client.18063 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f37caa0 0x56457f37eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 4 client.18063 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.929+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56457f007720 con 0x56457f37caa0 2024-03-12T04:24:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.930+0000 7f819a7fc640 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f37caa0 0x56457f37eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.930+0000 7f819a7fc640 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f37caa0 0x56457f37eec0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f5131ebda04f677 server_cookie=78477346d4a66b25 in_seq=0 out_seq=0 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.930+0000 7f817affd640 10 client.18063 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f817affd640 1 -- 172.21.15.42:0/3279710482 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56457f007720 con 0x56457f37caa0 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f817affd640 10 client.18063 handle_command_reply: tid=0 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 client.18063 shutdown 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 msgr2=0x56457f3697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f3697d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8194036040 tx=0x7f819402c980 comp rx=0 tx=0).stop 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 shutdown_connections 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f37caa0 0x56457f37eec0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f36c750 0x56457f36a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363b70 0x56457f369d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f817affd640 0 client.18063 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 --2- 172.21.15.42:0/3279710482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f363250 0x56457f3697d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 >> 172.21.15.42:0/3279710482 conn(0x56457f35d980 msgr2=0x56457f35f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 shutdown_connections 2024-03-12T04:24:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 1 -- 172.21.15.42:0/3279710482 wait complete. 2024-03-12T04:24:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.931+0000 7f81a4403740 20 client.18063 trim_cache size 0 max 16384 2024-03-12T04:24:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 Processor -- start 2024-03-12T04:24:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 -- start start 2024-03-12T04:24:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f36bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f366a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f250130 con 0x56457f36c750 2024-03-12T04:24:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f379650 con 0x56457f363b70 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f81a4403740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f37c8a0 con 0x56457f363250 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f366a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f366a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52140/0 (socket says 172.21.15.42:52140) 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 -- 172.21.15.42:0/2558129250 learned_addr learned my addr 172.21.15.42:0/2558129250 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819affd640 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f36bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819a7fc640 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 -- 172.21.15.42:0/2558129250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 msgr2=0x56457f36bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f36bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 -- 172.21.15.42:0/2558129250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 msgr2=0x56457f36a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819b7fe640 1 -- 172.21.15.42:0/2558129250 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56457f265c20 con 0x56457f36c750 2024-03-12T04:24:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.943+0000 7f819a7fc640 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f819b7fe640 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f366a30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f818c005c30 tx=0x7f818c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967f82c140c7365a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81917fa640 1 -- 172.21.15.42:0/2558129250 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c012ec0 con 0x56457f36c750 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81917fa640 1 -- 172.21.15.42:0/2558129250 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f818c0096f0 con 0x56457f36c750 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81917fa640 1 -- 172.21.15.42:0/2558129250 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c005040 con 0x56457f36c750 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 -- 172.21.15.42:0/2558129250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 msgr2=0x56457f366a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f366a30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f818c005c30 tx=0x7f818c009b00 comp rx=0 tx=0).stop 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 -- 172.21.15.42:0/2558129250 shutdown_connections 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f366a30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f36a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 --2- 172.21.15.42:0/2558129250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f36bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 -- 172.21.15.42:0/2558129250 >> 172.21.15.42:0/2558129250 conn(0x56457f35d980 msgr2=0x56457f364a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 -- 172.21.15.42:0/2558129250 shutdown_connections 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.944+0000 7f81a4403740 1 -- 172.21.15.42:0/2558129250 wait complete. 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 -- start start 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 Processor -- start 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f3729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f372f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f3cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f3cc4b0 con 0x56457f36c750 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f3cc620 con 0x56457f363b70 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457f3cc7e0 con 0x56457f363250 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f81a4403740 10 client.0 _ll_register_callbacks cb 0x7f818c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f3cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f3cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52148/0 (socket says 172.21.15.42:52148) 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 -- 172.21.15.42:0/4023421112 learned_addr learned my addr 172.21.15.42:0/4023421112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819affd640 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f3729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819a7fc640 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f372f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 -- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 msgr2=0x56457f3729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f3729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 -- 172.21.15.42:0/4023421112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 msgr2=0x56457f372f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f372f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.945+0000 7f819b7fe640 1 -- 172.21.15.42:0/4023421112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56457f28d060 con 0x56457f36c750 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f819b7fe640 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f3cbf70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f818c005cb0 tx=0x7f818c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc80a42b91356b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f817affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f817affd640 1 -- 172.21.15.42:0/4023421112 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c00cd40 con 0x56457f36c750 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f817affd640 1 -- 172.21.15.42:0/4023421112 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f818c01f480 con 0x56457f36c750 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f818c009c00 con 0x56457f36c750 2024-03-12T04:24:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f817affd640 1 -- 172.21.15.42:0/4023421112 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f818c00cea0 con 0x56457f36c750 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 10 client.18069 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56457f28b8c0 con 0x56457f36c750 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f817affd640 1 -- 172.21.15.42:0/4023421112 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f818c009c00 con 0x56457f36c750 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f817affd640 1 -- 172.21.15.42:0/4023421112 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f818c0316a0 con 0x56457f36c750 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 10 client.18069 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 10 client.18069 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 20 client.18069 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f3d4ef0 0x56457f3d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 4 client.18069 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.946+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56457f056890 con 0x56457f3d4ef0 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f819affd640 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f3d4ef0 0x56457f3d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:16.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f819affd640 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f3d4ef0 0x56457f3d7310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e36ff43c086aad6 server_cookie=ca72ca51c2b99035 in_seq=0 out_seq=0 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f817affd640 10 client.18069 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f817affd640 1 -- 172.21.15.42:0/4023421112 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56457f056890 con 0x56457f3d4ef0 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f817affd640 10 client.18069 handle_command_reply: tid=0 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 client.18069 shutdown 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 msgr2=0x56457f3cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f3cbf70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f818c005cb0 tx=0x7f818c010740 comp rx=0 tx=0).stop 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 shutdown_connections 2024-03-12T04:24:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x56457f3d4ef0 0x56457f3d7310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56457f36c750 0x56457f3cbf70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56457f363b70 0x56457f372f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f817affd640 0 client.18069 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 --2- 172.21.15.42:0/4023421112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56457f363250 0x56457f3729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 >> 172.21.15.42:0/4023421112 conn(0x56457f35d980 msgr2=0x56457f35c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 shutdown_connections 2024-03-12T04:24:16.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 1 -- 172.21.15.42:0/4023421112 wait complete. 2024-03-12T04:24:16.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.947+0000 7f81a4403740 20 client.18069 trim_cache size 0 max 16384 2024-03-12T04:24:16.964 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:16.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f81880a1ba0 msgr2=0x7f81880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f81880a1ba0 0x7f81880a4060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f818c010c40 tx=0x7f818c017000 comp rx=0 tx=0).stop 2024-03-12T04:24:16.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c105330 msgr2=0x7f819c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c105330 0x7f819c197c40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f818400d770 tx=0x7f818400dc30 comp rx=0 tx=0).stop 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 shutdown_connections 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f81880a1ba0 0x7f81880a4060 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c19bfc0 0x7f819c19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f819c105330 0x7f819c197c40 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 --2- 172.21.15.42:0/3942338346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c100260 0x7f819c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 >> 172.21.15.42:0/3942338346 conn(0x7f819c077180 msgr2=0x7f819c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 shutdown_connections 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:16.948+0000 7f81a182e640 1 -- 172.21.15.42:0/3942338346 wait complete. 2024-03-12T04:24:16.966 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:16.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:24:17.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 -- 172.21.15.42:0/1944744469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c069a80 msgr2=0x7f2a3c069e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 --2- 172.21.15.42:0/1944744469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c069a80 0x7f2a3c069e60 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009930 tx=0x7f2a2c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 -- 172.21.15.42:0/1944744469 shutdown_connections 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 --2- 172.21.15.42:0/1944744469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a3c106450 0x7f2a3c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 --2- 172.21.15.42:0/1944744469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a3c06a430 0x7f2a3c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 --2- 172.21.15.42:0/1944744469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c069a80 0x7f2a3c069e60 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 -- 172.21.15.42:0/1944744469 >> 172.21.15.42:0/1944744469 conn(0x7f2a3c077150 msgr2=0x7f2a3c077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 -- 172.21.15.42:0/1944744469 shutdown_connections 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 -- 172.21.15.42:0/1944744469 wait complete. 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 Processor -- start 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.069+0000 7f2a41855640 1 -- start start 2024-03-12T04:24:17.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a3c069a80 0x7f2a3c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c106450 0x7f2a3c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a3c19bf30 0x7f2a3c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c19c920 con 0x7f2a3c106450 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c19ca90 con 0x7f2a3c19bf30 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c19cc00 con 0x7f2a3c069a80 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c106450 0x7f2a3c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c106450 0x7f2a3c197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52178/0 (socket says 172.21.15.42:52178) 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3761691228 learned_addr learned my addr 172.21.15.42:0/3761691228 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3affd640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a3c069a80 0x7f2a3c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3761691228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a3c069a80 msgr2=0x7f2a3c197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a3c069a80 0x7f2a3c197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3761691228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a3c19bf30 msgr2=0x7f2a3c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a3c19bf30 0x7f2a3c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3761691228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c009560 con 0x7f2a3c106450 2024-03-12T04:24:17.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c106450 0x7f2a3c197bb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c970 tx=0x7f2a3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:17.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a40853640 1 -- 172.21.15.42:0/3761691228 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30004440 con 0x7f2a3c106450 2024-03-12T04:24:17.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c1a41e0 con 0x7f2a3c106450 2024-03-12T04:24:17.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a40853640 1 -- 172.21.15.42:0/3761691228 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a30007d10 con 0x7f2a3c106450 2024-03-12T04:24:17.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a40853640 1 -- 172.21.15.42:0/3761691228 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000f450 con 0x7f2a3c106450 2024-03-12T04:24:17.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.070+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c1a4690 con 0x7f2a3c106450 2024-03-12T04:24:17.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.072+0000 7f2a40853640 1 -- 172.21.15.42:0/3761691228 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f2a30003680 con 0x7f2a3c106450 2024-03-12T04:24:17.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 Processor -- start 2024-03-12T04:24:17.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 -- start start 2024-03-12T04:24:17.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a40853640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72b70 0x560064b7b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064b2d960 con 0x560064b72250 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064ae1310 con 0x560064b7b750 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a3b7fe640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a4442b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064ade840 con 0x560064b72b70 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a40853640 1 -- 172.21.15.42:0/3761691228 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2a300cdaa0 con 0x7f2a3c106450 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b72630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52188/0 (socket says 172.21.15.42:52188) 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a3affd640 1 -- 172.21.15.42:0/1201633626 learned_addr learned my addr 172.21.15.42:0/1201633626 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72b70 0x560064b7b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a3affd640 1 -- 172.21.15.42:0/1201633626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72b70 msgr2=0x560064b7b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.073+0000 7f2a3affd640 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72b70 0x560064b7b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a3affd640 1 -- 172.21.15.42:0/1201633626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 msgr2=0x560064b7db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a3affd640 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a3affd640 1 -- 172.21.15.42:0/1201633626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560064a9c060 con 0x560064b72250 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72b70 0x560064b7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a3b7fe640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005fd0 tx=0x7f2a34007f20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a3affd640 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b72630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c036040 tx=0x7f2a2c0047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771342e346cd82a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a21ffb640 1 -- 172.21.15.42:0/1201633626 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c02c770 con 0x560064b72250 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a21ffb640 1 -- 172.21.15.42:0/1201633626 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a2c02de90 con 0x560064b72250 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a21ffb640 1 -- 172.21.15.42:0/1201633626 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c003a20 con 0x560064b72250 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a4442b740 1 -- 172.21.15.42:0/1201633626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 msgr2=0x560064b72630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.074+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b72630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c036040 tx=0x7f2a2c0047f0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- 172.21.15.42:0/1201633626 shutdown_connections 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b7db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72b70 0x560064b7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1201633626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b72630 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- 172.21.15.42:0/1201633626 >> 172.21.15.42:0/1201633626 conn(0x560064b6c980 msgr2=0x560064b6cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- 172.21.15.42:0/1201633626 shutdown_connections 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- 172.21.15.42:0/1201633626 wait complete. 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- start start 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 Processor -- start 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 0x560064b78c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 0x560064b79170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064ada020 con 0x560064b72250 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600647ad430 con 0x560064b72b70 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064769e00 con 0x560064b7b750 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a4442b740 10 client.0 _ll_register_callbacks cb 0x560064a81cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b786f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52192/0 (socket says 172.21.15.42:52192) 2024-03-12T04:24:17.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.075+0000 7f2a3affd640 1 -- 172.21.15.42:0/4247540944 learned_addr learned my addr 172.21.15.42:0/4247540944 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 0x560064b78c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3b7fe640 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 0x560064b79170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3affd640 1 -- 172.21.15.42:0/4247540944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 msgr2=0x560064b79170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3affd640 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 0x560064b79170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3affd640 1 -- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 msgr2=0x560064b78c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3affd640 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 0x560064b78c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3affd640 1 -- 172.21.15.42:0/4247540944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c10e830 con 0x560064b72250 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a3affd640 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b786f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c02bc70 tx=0x7f2a2c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817ab6bc38fd56b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a1affd640 1 -- 172.21.15.42:0/4247540944 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0053c0 con 0x560064b72250 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a1affd640 1 -- 172.21.15.42:0/4247540944 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2a2c005520 con 0x560064b72250 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600647d97e0 con 0x560064b72250 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a1affd640 1 -- 172.21.15.42:0/4247540944 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a2c034440 con 0x560064b72250 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a4442b740 10 client.18081 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:17.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560064ad86a0 con 0x560064b72250 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.076+0000 7f2a1affd640 1 -- 172.21.15.42:0/4247540944 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5600647d97e0 con 0x560064b72250 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a1affd640 1 -- 172.21.15.42:0/4247540944 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2a2c03c040 con 0x560064b72250 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a4442b740 10 client.18081 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a4442b740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a4442b740 20 client.18081 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a4442b740 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064b8f9a0 0x560064b91d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a4442b740 4 client.18081 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560064816720 con 0x560064b8f9a0 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064b8f9a0 0x560064b91d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064b8f9a0 0x560064b91d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90393f377be80c06 server_cookie=f3da7381c644b5d9 in_seq=0 out_seq=0 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.077+0000 7f2a1affd640 10 client.18081 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:17.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a1affd640 1 -- 172.21.15.42:0/4247540944 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560064816720 con 0x560064b8f9a0 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a1affd640 10 client.18081 handle_command_reply: tid=0 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 client.18081 shutdown 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 msgr2=0x560064b786f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b786f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c02bc70 tx=0x7f2a2c004dc0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 shutdown_connections 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064b8f9a0 0x560064b91d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 0x560064b79170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 0x560064b78c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a1affd640 0 client.18081 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 --2- 172.21.15.42:0/4247540944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b786f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 >> 172.21.15.42:0/4247540944 conn(0x560064b6c980 msgr2=0x560064b6e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 shutdown_connections 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 1 -- 172.21.15.42:0/4247540944 wait complete. 2024-03-12T04:24:17.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.078+0000 7f2a4442b740 20 client.18081 trim_cache size 0 max 16384 2024-03-12T04:24:17.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 Processor -- start 2024-03-12T04:24:17.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 -- start start 2024-03-12T04:24:17.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b7ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 0x560064b78f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 0x560064b75970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064a5f130 con 0x560064b72250 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064b96f80 con 0x560064b72b70 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a4442b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064b7db40 con 0x560064b7b750 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b7ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b7ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52204/0 (socket says 172.21.15.42:52204) 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.090+0000 7f2a3affd640 1 -- 172.21.15.42:0/1085111242 learned_addr learned my addr 172.21.15.42:0/1085111242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a3affd640 1 -- 172.21.15.42:0/1085111242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 msgr2=0x560064b75970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a3affd640 1 --2- 172.21.15.42:0/1085111242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 0x560064b75970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a3affd640 1 -- 172.21.15.42:0/1085111242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 msgr2=0x560064b78f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a3affd640 1 --2- 172.21.15.42:0/1085111242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 0x560064b78f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a3affd640 1 -- 172.21.15.42:0/1085111242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560064a74c20 con 0x560064b72250 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a3affd640 1 --2- 172.21.15.42:0/1085111242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b7ae20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c02cb00 tx=0x7f2a2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae61afeb65451ca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a217fa640 1 -- 172.21.15.42:0/1085111242 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c02cbc0 con 0x560064b72250 2024-03-12T04:24:17.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a217fa640 1 -- 172.21.15.42:0/1085111242 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a2c0046c0 con 0x560064b72250 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a217fa640 1 -- 172.21.15.42:0/1085111242 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0035f0 con 0x560064b72250 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 -- 172.21.15.42:0/1085111242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 msgr2=0x560064b7ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1085111242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b7ae20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c02cb00 tx=0x7f2a2c000bc0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 -- 172.21.15.42:0/1085111242 shutdown_connections 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1085111242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b7b750 0x560064b75970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1085111242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b72b70 0x560064b78f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 --2- 172.21.15.42:0/1085111242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72250 0x560064b7ae20 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 -- 172.21.15.42:0/1085111242 >> 172.21.15.42:0/1085111242 conn(0x560064b6c980 msgr2=0x560064b739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 -- 172.21.15.42:0/1085111242 shutdown_connections 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.091+0000 7f2a4442b740 1 -- 172.21.15.42:0/1085111242 wait complete. 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 -- start start 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 Processor -- start 2024-03-12T04:24:17.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72250 0x560064b86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72b70 0x560064b80d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064b81920 con 0x560064b72b70 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064b81a90 con 0x560064b7b750 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560064b81c50 con 0x560064b72250 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a4442b740 10 client.0 _ll_register_callbacks cb 0x7f2a2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72b70 0x560064b80d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72b70 0x560064b80d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52218/0 (socket says 172.21.15.42:52218) 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3478264954 learned_addr learned my addr 172.21.15.42:0/3478264954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a3affd640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72250 0x560064b86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.092+0000 7f2a3b7fe640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3478264954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72250 msgr2=0x560064b86cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72250 0x560064b86cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 msgr2=0x560064b812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a3a7fc640 1 -- 172.21.15.42:0/3478264954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560064ad86a0 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a3b7fe640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a3a7fc640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72b70 0x560064b80d80 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f2a3009a620 tx=0x7f2a3009a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb557b0f64c54ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a1affd640 1 -- 172.21.15.42:0/3478264954 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3009e7e0 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a1affd640 1 -- 172.21.15.42:0/3478264954 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2a300a2e30 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a2c02de90 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a1affd640 1 -- 172.21.15.42:0/3478264954 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a300b3120 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 10 client.18087 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560064a9a8c0 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a1affd640 1 -- 172.21.15.42:0/3478264954 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a2c02de90 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a1affd640 1 -- 172.21.15.42:0/3478264954 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2a300ba110 con 0x560064b72b70 2024-03-12T04:24:17.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 10 client.18087 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 10 client.18087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 20 client.18087 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064bebd50 0x560064bee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 4 client.18087 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.093+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560064865890 con 0x560064bebd50 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a3affd640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064bebd50 0x560064bee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a3affd640 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064bebd50 0x560064bee170 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ef1b3abe0d7691 server_cookie=5a5ecbe721bfcab7 in_seq=0 out_seq=0 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a1affd640 10 client.18087 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a1affd640 1 -- 172.21.15.42:0/3478264954 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560064865890 con 0x560064bebd50 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a1affd640 10 client.18087 handle_command_reply: tid=0 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 client.18087 shutdown 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72b70 msgr2=0x560064b80d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72b70 0x560064b80d80 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f2a3009a620 tx=0x7f2a3009a580 comp rx=0 tx=0).stop 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 shutdown_connections 2024-03-12T04:24:17.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x560064bebd50 0x560064bee170 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560064b7b750 0x560064b812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560064b72b70 0x560064b80d80 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a1affd640 0 client.18087 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 --2- 172.21.15.42:0/3478264954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560064b72250 0x560064b86cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.094+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 >> 172.21.15.42:0/3478264954 conn(0x560064b6c980 msgr2=0x560064b6b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 shutdown_connections 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a4442b740 1 -- 172.21.15.42:0/3478264954 wait complete. 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a4442b740 20 client.18087 trim_cache size 0 max 16384 2024-03-12T04:24:17.104 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:17.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 msgr2=0x7f2a240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005fd0 tx=0x7f2a34007f20 comp rx=0 tx=0).stop 2024-03-12T04:24:17.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c106450 msgr2=0x7f2a3c197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:17.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c106450 0x7f2a3c197bb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c970 tx=0x7f2a3000ce30 comp rx=0 tx=0).stop 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 shutdown_connections 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2a240a1ba0 0x7f2a240a4060 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a3c19bf30 0x7f2a3c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a3c106450 0x7f2a3c197bb0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 --2- 172.21.15.42:0/3761691228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a3c069a80 0x7f2a3c197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 >> 172.21.15.42:0/3761691228 conn(0x7f2a3c077150 msgr2=0x7f2a3c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 shutdown_connections 2024-03-12T04:24:17.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:17.095+0000 7f2a41855640 1 -- 172.21.15.42:0/3761691228 wait complete. 2024-03-12T04:24:17.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:17 smithi119 ceph-mon[36119]: pgmap v281: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:24:17.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:17 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3950511704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:24:17.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:17 smithi136 ceph-mon[35571]: pgmap v281: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:24:17.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:17 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3950511704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:24:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:17 smithi042 ceph-mon[32260]: pgmap v281: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:24:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:17 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3950511704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:24:18.109 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:24:18.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:24:18.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:24:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.226+0000 7f9ea505c640 1 -- 172.21.15.42:0/972059065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0100260 msgr2=0x7f9ea0100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.226+0000 7f9ea505c640 1 --2- 172.21.15.42:0/972059065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0100260 0x7f9ea0100640 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009930 tx=0x7f9e9002bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.226+0000 7f9ea505c640 1 -- 172.21.15.42:0/972059065 shutdown_connections 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.226+0000 7f9ea505c640 1 --2- 172.21.15.42:0/972059065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ea0105330 0x7f9ea010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.226+0000 7f9ea505c640 1 --2- 172.21.15.42:0/972059065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ea0100c10 0x7f9ea0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.226+0000 7f9ea505c640 1 --2- 172.21.15.42:0/972059065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0100260 0x7f9ea0100640 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.226+0000 7f9ea505c640 1 -- 172.21.15.42:0/972059065 >> 172.21.15.42:0/972059065 conn(0x7f9ea0077180 msgr2=0x7f9ea0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.227+0000 7f9ea505c640 1 -- 172.21.15.42:0/972059065 shutdown_connections 2024-03-12T04:24:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.227+0000 7f9ea505c640 1 -- 172.21.15.42:0/972059065 wait complete. 2024-03-12T04:24:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.227+0000 7f9ea505c640 1 Processor -- start 2024-03-12T04:24:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9ea505c640 1 -- start start 2024-03-12T04:24:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9ea505c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ea0100260 0x7f9ea0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9ea505c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0105330 0x7f9ea0102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9ea505c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ea0102770 0x7f9ea01a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9ea505c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01a5ea0 con 0x7f9ea0105330 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9ea505c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01a6010 con 0x7f9ea0102770 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9ea505c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01a61d0 con 0x7f9ea0100260 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0105330 0x7f9ea0102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0105330 0x7f9ea0102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52240/0 (socket says 172.21.15.42:52240) 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 -- 172.21.15.42:0/2582917974 learned_addr learned my addr 172.21.15.42:0/2582917974 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ea0102770 0x7f9ea01a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ea0100260 0x7f9ea0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 -- 172.21.15.42:0/2582917974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ea0100260 msgr2=0x7f9ea0101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ea0100260 0x7f9ea0101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 -- 172.21.15.42:0/2582917974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ea0102770 msgr2=0x7f9ea01a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ea0102770 0x7f9ea01a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 -- 172.21.15.42:0/2582917974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90009560 con 0x7f9ea0105330 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.228+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0105330 0x7f9ea0102230 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c970 tx=0x7f9e9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.229+0000 7f9e87fff640 1 -- 172.21.15.42:0/2582917974 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94007bb0 con 0x7f9ea0105330 2024-03-12T04:24:18.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.229+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea01a64b0 con 0x7f9ea0105330 2024-03-12T04:24:18.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.229+0000 7f9e87fff640 1 -- 172.21.15.42:0/2582917974 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e94007d10 con 0x7f9ea0105330 2024-03-12T04:24:18.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.229+0000 7f9e87fff640 1 -- 172.21.15.42:0/2582917974 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94004dc0 con 0x7f9ea0105330 2024-03-12T04:24:18.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.229+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea01a6a10 con 0x7f9ea0105330 2024-03-12T04:24:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9e87fff640 1 -- 172.21.15.42:0/2582917974 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f9e94004f40 con 0x7f9ea0105330 2024-03-12T04:24:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 Processor -- start 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 -- start start 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da6917e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da691872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da69189d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da690ecf90 con 0x55da691878b0 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da690ea4c0 con 0x55da6917e380 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.231+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da690e5ca0 con 0x55da6917ed30 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da69189d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da69189d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52256/0 (socket says 172.21.15.42:52256) 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3043355419 learned_addr learned my addr 172.21.15.42:0/3043355419 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e87fff640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e880a1bd0 0x7f9e880a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da691872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da6917e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e880a1bd0 0x7f9e880a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e87fff640 1 -- 172.21.15.42:0/2582917974 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9e940cd540 con 0x7f9ea0105330 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3043355419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 msgr2=0x55da691872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da691872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3043355419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 msgr2=0x55da6917e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da6917e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3043355419 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da690a80b0 con 0x55da691878b0 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da6917e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da69189d00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9e9409a0c0 tx=0x7f9e94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e9aab0d3e4403e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e880a1bd0 0x7f9e880a4090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009fd0 tx=0x7f9e98009600 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.232+0000 7f9e857fa640 1 -- 172.21.15.42:0/3043355419 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940046c0 con 0x55da691878b0 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9e857fa640 1 -- 172.21.15.42:0/3043355419 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e9409a830 con 0x55da691878b0 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9e857fa640 1 -- 172.21.15.42:0/3043355419 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940caed0 con 0x55da691878b0 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3043355419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 msgr2=0x55da69189d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da69189d00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9e9409a0c0 tx=0x7f9e94000bc0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3043355419 shutdown_connections 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da69189d00 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da691872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3043355419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da6917e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3043355419 >> 172.21.15.42:0/3043355419 conn(0x55da69178840 msgr2=0x55da69178c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3043355419 shutdown_connections 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.233+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3043355419 wait complete. 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 -- start start 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 Processor -- start 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da69184840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da69184d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da691852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6908af40 con 0x55da691878b0 2024-03-12T04:24:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da68db94b0 con 0x55da6917e380 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6906b670 con 0x55da6917ed30 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9ea7c32740 10 client.0 _ll_register_callbacks cb 0x55da690d8f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da691852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da691852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52264/0 (socket says 172.21.15.42:52264) 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 -- 172.21.15.42:0/3518132464 learned_addr learned my addr 172.21.15.42:0/3518132464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da69184d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da69184840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 -- 172.21.15.42:0/3518132464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 msgr2=0x55da69184d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da69184d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 -- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 msgr2=0x55da69184840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da69184840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9f59e640 1 -- 172.21.15.42:0/3518132464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea01a6a10 con 0x55da691878b0 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.234+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da69184840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:18.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da691852c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800abc0 tx=0x7f9e98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9439543f9b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9e82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9e82ff5640 1 -- 172.21.15.42:0/3518132464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98010d20 con 0x55da691878b0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9e82ff5640 1 -- 172.21.15.42:0/3518132464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9e98004b20 con 0x55da691878b0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da690bff40 con 0x55da691878b0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9e82ff5640 1 -- 172.21.15.42:0/3518132464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e98010e80 con 0x55da691878b0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 10 client.18099 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da690e4320 con 0x55da691878b0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9e82ff5640 1 -- 172.21.15.42:0/3518132464 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55da690bff40 con 0x55da691878b0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9e82ff5640 1 -- 172.21.15.42:0/3518132464 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e98026030 con 0x55da691878b0 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 10 client.18099 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 10 client.18099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 20 client.18099 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da69197e70 0x55da6919a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 4 client.18099 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.235+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da690dc310 con 0x55da69197e70 2024-03-12T04:24:18.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.236+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da69197e70 0x55da6919a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.236+0000 7f9e82ff5640 10 client.18099 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.236+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da69197e70 0x55da6919a2b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ac1df4d71b875 server_cookie=e38f0165f511550a in_seq=0 out_seq=0 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9e82ff5640 1 -- 172.21.15.42:0/3518132464 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da690dc310 con 0x55da69197e70 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9e82ff5640 10 client.18099 handle_command_reply: tid=0 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 client.18099 shutdown 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 msgr2=0x55da691852c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da691852c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800abc0 tx=0x7f9e98004a40 comp rx=0 tx=0).stop 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 shutdown_connections 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da69197e70 0x55da6919a2b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da691852c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917ed30 0x55da69184d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3518132464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917e380 0x55da69184840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 >> 172.21.15.42:0/3518132464 conn(0x55da69178840 msgr2=0x55da6917a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9e82ff5640 0 client.18099 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 shutdown_connections 2024-03-12T04:24:18.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3518132464 wait complete. 2024-03-12T04:24:18.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.237+0000 7f9ea7c32740 20 client.18099 trim_cache size 0 max 16384 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.252+0000 7f9ea7c32740 1 Processor -- start 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.252+0000 7f9ea7c32740 1 -- start start 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.252+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da69186f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.252+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da6917ed30 0x55da691850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da691878b0 0x55da69181ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da690bff70 con 0x55da6917ed30 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da691948f0 con 0x55da691878b0 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6919f560 con 0x55da6917e380 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da6917ed30 0x55da691850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da6917ed30 0x55da691850e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52268/0 (socket says 172.21.15.42:52268) 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3690901986 learned_addr learned my addr 172.21.15.42:0/3690901986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da69186f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da691878b0 0x55da69181ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3690901986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 msgr2=0x55da69186f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da69186f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3690901986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da691878b0 msgr2=0x55da69181ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da691878b0 0x55da69181ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 -- 172.21.15.42:0/3690901986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da69080c00 con 0x55da6917ed30 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da6917ed30 0x55da691850e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9e94096b00 tx=0x7f9e940964b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d823b9dec27445a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e84ff9640 1 -- 172.21.15.42:0/3690901986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940046c0 con 0x55da6917ed30 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e84ff9640 1 -- 172.21.15.42:0/3690901986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e940a28d0 con 0x55da6917ed30 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.253+0000 7f9e84ff9640 1 -- 172.21.15.42:0/3690901986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940caed0 con 0x55da6917ed30 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3690901986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da6917ed30 msgr2=0x55da691850e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da6917ed30 0x55da691850e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9e94096b00 tx=0x7f9e940964b0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3690901986 shutdown_connections 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da691878b0 0x55da69181ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da6917ed30 0x55da691850e0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/3690901986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da69186f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3690901986 >> 172.21.15.42:0/3690901986 conn(0x55da69178840 msgr2=0x55da69183110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3690901986 shutdown_connections 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- 172.21.15.42:0/3690901986 wait complete. 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- start start 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 Processor -- start 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da6918d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917ed30 0x55da6918d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da6918dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da691ee3e0 con 0x55da691878b0 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da691ee520 con 0x55da6917ed30 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.254+0000 7f9ea7c32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da691ee660 con 0x55da6917e380 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9ea7c32740 10 client.0 _ll_register_callbacks cb 0x7f9e94099f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da6918dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da6918dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52274/0 (socket says 172.21.15.42:52274) 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 -- 172.21.15.42:0/118116007 learned_addr learned my addr 172.21.15.42:0/118116007 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da6918d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917ed30 0x55da6918d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 -- 172.21.15.42:0/118116007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 msgr2=0x55da6918d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da6918d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 -- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917ed30 msgr2=0x55da6918d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917ed30 0x55da6918d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 -- 172.21.15.42:0/118116007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da690a80b0 con 0x55da691878b0 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9e59c640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917ed30 0x55da6918d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e9f59e640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da6918dce0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9e98000f80 tx=0x7f9e98005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8c61eae2b2ef2f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e82ff5640 1 -- 172.21.15.42:0/118116007 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980288d0 con 0x55da691878b0 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e82ff5640 1 -- 172.21.15.42:0/118116007 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9e98028a30 con 0x55da691878b0 2024-03-12T04:24:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da6910b900 con 0x55da691878b0 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9e82ff5640 1 -- 172.21.15.42:0/118116007 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e98028bf0 con 0x55da691878b0 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9ea7c32740 10 client.18105 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.255+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da690a6910 con 0x55da691878b0 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9e82ff5640 1 -- 172.21.15.42:0/118116007 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e98048020 con 0x55da691878b0 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9e82ff5640 1 -- 172.21.15.42:0/118116007 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e98026040 con 0x55da691878b0 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9ea7c32740 10 client.18105 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9ea7c32740 10 client.18105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9ea7c32740 20 client.18105 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da691efd40 0x55da691f2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9ea7c32740 4 client.18105 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55da68de5860 con 0x55da691efd40 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.256+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da691efd40 0x55da691f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.257+0000 7f9e82ff5640 10 client.18105 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.257+0000 7f9e9ed9d640 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da691efd40 0x55da691f2160 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baafaa1ddada673 server_cookie=b8f6b8e9b7f3fc8f in_seq=0 out_seq=0 2024-03-12T04:24:18.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9e82ff5640 1 -- 172.21.15.42:0/118116007 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55da68de5860 con 0x55da691efd40 2024-03-12T04:24:18.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9e82ff5640 10 client.18105 handle_command_reply: tid=0 2024-03-12T04:24:18.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 client.18105 shutdown 2024-03-12T04:24:18.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 msgr2=0x55da6918dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da6918dce0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9e98000f80 tx=0x7f9e98005bc0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 shutdown_connections 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55da691efd40 0x55da691f2160 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55da691878b0 0x55da6918dce0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da6917ed30 0x55da6918d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9e82ff5640 0 client.18105 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 --2- 172.21.15.42:0/118116007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da6917e380 0x55da6918d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 >> 172.21.15.42:0/118116007 conn(0x55da69178840 msgr2=0x55da69177050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 shutdown_connections 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 1 -- 172.21.15.42:0/118116007 wait complete. 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.258+0000 7f9ea7c32740 20 client.18105 trim_cache size 0 max 16384 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:24:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:24:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "c5cfd82b-cac8-48f0-b08b-22aa031df68b", 2024-03-12T04:24:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:24:18.269 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e880a1bd0 msgr2=0x7f9e880a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e880a1bd0 0x7f9e880a4090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009fd0 tx=0x7f9e98009600 comp rx=0 tx=0).stop 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0105330 msgr2=0x7f9ea0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0105330 0x7f9ea0102230 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c970 tx=0x7f9e9400ce30 comp rx=0 tx=0).stop 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 shutdown_connections 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9e880a1bd0 0x7f9e880a4090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ea0102770 0x7f9ea01a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ea0105330 0x7f9ea0102230 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 --2- 172.21.15.42:0/2582917974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ea0100260 0x7f9ea0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 >> 172.21.15.42:0/2582917974 conn(0x7f9ea0077180 msgr2=0x7f9ea0068710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 shutdown_connections 2024-03-12T04:24:18.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.259+0000 7f9ea505c640 1 -- 172.21.15.42:0/2582917974 wait complete. 2024-03-12T04:24:18.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:24:18.273 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:24:18.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 -- 172.21.15.42:0/3662878180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c101d10 msgr2=0x7f3e0c105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 --2- 172.21.15.42:0/3662878180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c101d10 0x7f3e0c105d50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009930 tx=0x7f3df402bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 -- 172.21.15.42:0/3662878180 shutdown_connections 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 --2- 172.21.15.42:0/3662878180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c106430 0x7f3e0c10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 --2- 172.21.15.42:0/3662878180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c101d10 0x7f3e0c105d50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 --2- 172.21.15.42:0/3662878180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0c101360 0x7f3e0c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 -- 172.21.15.42:0/3662878180 >> 172.21.15.42:0/3662878180 conn(0x7f3e0c0fcc20 msgr2=0x7f3e0c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 -- 172.21.15.42:0/3662878180 shutdown_connections 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.375+0000 7f3e10913640 1 -- 172.21.15.42:0/3662878180 wait complete. 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 Processor -- start 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 -- start start 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0c101360 0x7f3e0c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c106430 0x7f3e0c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c19c010 0x7f3e0c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c19ca00 con 0x7f3e0c106430 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c19cb70 con 0x7f3e0c19c010 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e10913640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c19cce0 con 0x7f3e0c101360 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c106430 0x7f3e0c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c106430 0x7f3e0c197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52290/0 (socket says 172.21.15.42:52290) 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 -- 172.21.15.42:0/725066990 learned_addr learned my addr 172.21.15.42:0/725066990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0c101360 0x7f3e0c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 -- 172.21.15.42:0/725066990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0c101360 msgr2=0x7f3e0c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0c101360 0x7f3e0c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 -- 172.21.15.42:0/725066990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c19c010 msgr2=0x7f3e0c19c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c19c010 0x7f3e0c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 -- 172.21.15.42:0/725066990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df4009560 con 0x7f3e0c106430 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.376+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c106430 0x7f3e0c197ce0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f3df4000c00 tx=0x7f3df4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.377+0000 7f3e037fe640 1 -- 172.21.15.42:0/725066990 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df402cb70 con 0x7f3e0c106430 2024-03-12T04:24:18.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.377+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c1a42c0 con 0x7f3e0c106430 2024-03-12T04:24:18.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.377+0000 7f3e037fe640 1 -- 172.21.15.42:0/725066990 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3df402de90 con 0x7f3e0c106430 2024-03-12T04:24:18.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.377+0000 7f3e037fe640 1 -- 172.21.15.42:0/725066990 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df40053c0 con 0x7f3e0c106430 2024-03-12T04:24:18.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.377+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c1a4770 con 0x7f3e0c106430 2024-03-12T04:24:18.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.379+0000 7f3e037fe640 1 -- 172.21.15.42:0/725066990 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3df40035f0 con 0x7f3e0c106430 2024-03-12T04:24:18.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.379+0000 7f3e037fe640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3df80a1ba0 0x7f3df80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.379+0000 7f3e135c7740 1 Processor -- start 2024-03-12T04:24:18.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.379+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3df80a1ba0 0x7f3df80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.379+0000 7f3e037fe640 1 -- 172.21.15.42:0/725066990 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3df40efb60 con 0x7f3e0c106430 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.379+0000 7f3e135c7740 1 -- start start 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 0x558f642daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e135c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f6423e280 con 0x558f642cf2b0 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e135c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f6423b7b0 con 0x558f642d87b0 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e135c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f64236f90 con 0x558f642cfbd0 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52294/0 (socket says 172.21.15.42:52294) 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3708056852 learned_addr learned my addr 172.21.15.42:0/3708056852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3df80a1ba0 0x7f3df80a4060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e04005fd0 tx=0x7f3e04005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3708056852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 msgr2=0x558f642d8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3708056852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 msgr2=0x558f642daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 0x558f642daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3708056852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f641f5570 con 0x558f642cf2b0 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642cf690 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400fec0 tx=0x7f3e04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea69916bc807dbf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e00ff9640 1 -- 172.21.15.42:0/3708056852 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0401e650 con 0x558f642cf2b0 2024-03-12T04:24:18.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e00ff9640 1 -- 172.21.15.42:0/3708056852 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e04011e50 con 0x558f642cf2b0 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.380+0000 7f3e00ff9640 1 -- 172.21.15.42:0/3708056852 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0401da90 con 0x558f642cf2b0 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 -- 172.21.15.42:0/3708056852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 msgr2=0x558f642cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642cf690 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400fec0 tx=0x7f3e04012000 comp rx=0 tx=0).stop 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 -- 172.21.15.42:0/3708056852 shutdown_connections 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 0x558f642daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3708056852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642cf690 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 -- 172.21.15.42:0/3708056852 >> 172.21.15.42:0/3708056852 conn(0x558f642c9990 msgr2=0x558f642c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 -- 172.21.15.42:0/3708056852 shutdown_connections 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 -- 172.21.15.42:0/3708056852 wait complete. 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 -- start start 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.381+0000 7f3e135c7740 1 Processor -- start 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 0x558f642d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f63f0a430 con 0x558f642cf2b0 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f63ec6e00 con 0x558f642d87b0 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f641bbfd0 con 0x558f642cfbd0 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 10 client.0 _ll_register_callbacks cb 0x558f64211fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642d8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52310/0 (socket says 172.21.15.42:52310) 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 -- 172.21.15.42:0/2327278254 learned_addr learned my addr 172.21.15.42:0/2327278254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 -- 172.21.15.42:0/2327278254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 msgr2=0x558f642d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 -- 172.21.15.42:0/2327278254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 msgr2=0x558f642d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 0x558f642d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 -- 172.21.15.42:0/2327278254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0400fb70 con 0x558f642cf2b0 2024-03-12T04:24:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642d8230 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3e0401dba0 tx=0x7f3e0401dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8928b557fa92cbac server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3df1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3df1ffb640 1 -- 172.21.15.42:0/2327278254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0402d070 con 0x558f642cf2b0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3df1ffb640 1 -- 172.21.15.42:0/2327278254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3e04032470 con 0x558f642cf2b0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.382+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f63f367e0 con 0x558f642cf2b0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3df1ffb640 1 -- 172.21.15.42:0/2327278254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e04032630 con 0x558f642cf2b0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 10 client.18117 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f64235610 con 0x558f642cf2b0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3df1ffb640 1 -- 172.21.15.42:0/2327278254 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e0403f020 con 0x558f642cf2b0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3df1ffb640 1 -- 172.21.15.42:0/2327278254 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e040279f0 con 0x558f642cf2b0 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 10 client.18117 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 20 client.18117 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f642e8950 0x558f642ead60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 4 client.18117 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f63f73720 con 0x558f642e8950 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.383+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f642e8950 0x558f642ead60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3df1ffb640 10 client.18117 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f642e8950 0x558f642ead60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c6ecd0e4b3b3e server_cookie=4b471ef0a76556d5 in_seq=0 out_seq=0 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3df1ffb640 1 -- 172.21.15.42:0/2327278254 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f63f73720 con 0x558f642e8950 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3df1ffb640 10 client.18117 handle_command_reply: tid=0 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 client.18117 shutdown 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 msgr2=0x558f642d8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642d8230 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3e0401dba0 tx=0x7f3e0401dbd0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 shutdown_connections 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f642e8950 0x558f642ead60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642d87b0 0x558f642d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cfbd0 0x558f642d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3df1ffb640 0 client.18117 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 --2- 172.21.15.42:0/2327278254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642d8230 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.384+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 >> 172.21.15.42:0/2327278254 conn(0x558f642c9990 msgr2=0x558f642cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.385+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 shutdown_connections 2024-03-12T04:24:18.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.385+0000 7f3e135c7740 1 -- 172.21.15.42:0/2327278254 wait complete. 2024-03-12T04:24:18.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.385+0000 7f3e135c7740 20 client.18117 trim_cache size 0 max 16384 2024-03-12T04:24:18.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.399+0000 7f3e135c7740 1 Processor -- start 2024-03-12T04:24:18.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e135c7740 1 -- start start 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cf2b0 0x558f642d7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 0x558f642d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642d87b0 0x558f642d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e135c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f642ca9e0 con 0x558f642d87b0 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e135c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f642e4570 con 0x558f642cfbd0 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e135c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f642e4dd0 con 0x558f642cf2b0 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642d87b0 0x558f642d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642d87b0 0x558f642d4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52316/0 (socket says 172.21.15.42:52316) 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 -- 172.21.15.42:0/1373328327 learned_addr learned my addr 172.21.15.42:0/1373328327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 -- 172.21.15.42:0/1373328327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cf2b0 msgr2=0x558f642d7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cf2b0 0x558f642d7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cf2b0 0x558f642d7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 -- 172.21.15.42:0/1373328327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 msgr2=0x558f642d8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 0x558f642d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 -- 172.21.15.42:0/1373328327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f641d1ac0 con 0x558f642d87b0 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cf2b0 0x558f642d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3e0ad9d640 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642d87b0 0x558f642d4e50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005c40 tx=0x7f3dfc00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8987d6a84451e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3df3fff640 1 -- 172.21.15.42:0/1373328327 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc009e70 con 0x558f642d87b0 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3df3fff640 1 -- 172.21.15.42:0/1373328327 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dfc010910 con 0x558f642d87b0 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.400+0000 7f3df3fff640 1 -- 172.21.15.42:0/1373328327 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00f8e0 con 0x558f642d87b0 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 -- 172.21.15.42:0/1373328327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642d87b0 msgr2=0x558f642d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642d87b0 0x558f642d4e50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005c40 tx=0x7f3dfc00d6b0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 -- 172.21.15.42:0/1373328327 shutdown_connections 2024-03-12T04:24:18.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642d87b0 0x558f642d4e50 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 0x558f642d8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 --2- 172.21.15.42:0/1373328327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642cf2b0 0x558f642d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 -- 172.21.15.42:0/1373328327 >> 172.21.15.42:0/1373328327 conn(0x558f642c9990 msgr2=0x558f642d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 -- 172.21.15.42:0/1373328327 shutdown_connections 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 -- 172.21.15.42:0/1373328327 wait complete. 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 -- start start 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.401+0000 7f3e135c7740 1 Processor -- start 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642dbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 0x558f642dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642d87b0 0x558f642dca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f642dff90 con 0x558f642cf2b0 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f6433ee80 con 0x558f642cfbd0 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f6433f010 con 0x558f642d87b0 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 10 client.0 _ll_register_callbacks cb 0x7f3e04015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642dbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642dbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52330/0 (socket says 172.21.15.42:52330) 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3456914261 learned_addr learned my addr 172.21.15.42:0/3456914261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0ad9d640 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642d87b0 0x558f642dca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3456914261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642d87b0 msgr2=0x558f642dca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642d87b0 0x558f642dca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3456914261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 msgr2=0x558f642dc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 0x558f642dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 -- 172.21.15.42:0/3456914261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc009970 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e0a59c640 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642dbfd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3e0401e040 tx=0x7f3e04005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ba6fabd709a7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3df1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3df1ffb640 1 -- 172.21.15.42:0/3456914261 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0402d070 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3df1ffb640 1 -- 172.21.15.42:0/3456914261 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3e04040080 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e04011e50 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3df1ffb640 1 -- 172.21.15.42:0/3456914261 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e0400fd90 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 10 client.18123 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.402+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f641f76a0 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3df1ffb640 1 -- 172.21.15.42:0/3456914261 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e04011e50 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3df1ffb640 1 -- 172.21.15.42:0/3456914261 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e0401e740 con 0x558f642cf2b0 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e135c7740 10 client.18123 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e135c7740 10 client.18123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e135c7740 20 client.18123 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:18.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f643403f0 0x558f64342830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e135c7740 4 client.18123 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f63fc2890 con 0x558f643403f0 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f643403f0 0x558f64342830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3df1ffb640 10 client.18123 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.403+0000 7f3e09d9b640 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f643403f0 0x558f64342830 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53babf70c427533 server_cookie=204f061946f732aa in_seq=0 out_seq=0 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3df1ffb640 1 -- 172.21.15.42:0/3456914261 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558f63fc2890 con 0x558f643403f0 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3df1ffb640 10 client.18123 handle_command_reply: tid=0 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 client.18123 shutdown 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 msgr2=0x558f642dbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642dbfd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3e0401e040 tx=0x7f3e04005370 comp rx=0 tx=0).stop 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 shutdown_connections 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x558f643403f0 0x558f64342830 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558f642d87b0 0x558f642dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f642cfbd0 0x558f642dc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 --2- 172.21.15.42:0/3456914261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f642cf2b0 0x558f642dbfd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 >> 172.21.15.42:0/3456914261 conn(0x558f642c9990 msgr2=0x558f642c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3df1ffb640 0 client.18123 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 shutdown_connections 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 1 -- 172.21.15.42:0/3456914261 wait complete. 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e135c7740 20 client.18123 trim_cache size 0 max 16384 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "c5cfd82b-cac8-48f0-b08b-22aa031df68b": { 2024-03-12T04:24:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:24:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "c5cfd82b-cac8-48f0-b08b-22aa031df68b", 2024-03-12T04:24:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:24:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:24:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:24:18.415 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:24:18.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3df80a1ba0 msgr2=0x7f3df80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3df80a1ba0 0x7f3df80a4060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e04005fd0 tx=0x7f3e04005850 comp rx=0 tx=0).stop 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c106430 msgr2=0x7f3e0c197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c106430 0x7f3e0c197ce0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f3df4000c00 tx=0x7f3df4004270 comp rx=0 tx=0).stop 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 shutdown_connections 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3df80a1ba0 0x7f3df80a4060 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c19c010 0x7f3e0c19c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e0c106430 0x7f3e0c197ce0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.404+0000 7f3e10913640 1 --2- 172.21.15.42:0/725066990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0c101360 0x7f3e0c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.405+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 >> 172.21.15.42:0/725066990 conn(0x7f3e0c0fcc20 msgr2=0x7f3e0c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.405+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 shutdown_connections 2024-03-12T04:24:18.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:18.405+0000 7f3e10913640 1 -- 172.21.15.42:0/725066990 wait complete. 2024-03-12T04:24:18.418 INFO:tasks.cephfs.filesystem:scrub status for tag:c5cfd82b-cac8-48f0-b08b-22aa031df68b - {'path': '/', 'tag': 'c5cfd82b-cac8-48f0-b08b-22aa031df68b', 'options': 'recursive,force'} 2024-03-12T04:24:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:19 smithi119 ceph-mon[36119]: pgmap v282: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 251 B/s wr, 12 op/s 2024-03-12T04:24:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:19 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:24:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:19 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:19 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:24:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:19 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:19 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:24:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:19 smithi136 ceph-mon[35571]: pgmap v282: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 251 B/s wr, 12 op/s 2024-03-12T04:24:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:19 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:24:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:19 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:19 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:24:19.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:19 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:19.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:19 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:24:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:19 smithi042 ceph-mon[32260]: pgmap v282: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 251 B/s wr, 12 op/s 2024-03-12T04:24:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:19 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:24:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:19 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:19 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:24:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:19 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:19 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:24:21.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:21 smithi119 ceph-mon[36119]: pgmap v283: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-12T04:24:21.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:21 smithi136 ceph-mon[35571]: pgmap v283: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-12T04:24:21.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:21 smithi042 ceph-mon[32260]: pgmap v283: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-12T04:24:23.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:23 smithi119 ceph-mon[36119]: pgmap v284: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-12T04:24:23.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:23 smithi136 ceph-mon[35571]: pgmap v284: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-12T04:24:23.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:23 smithi042 ceph-mon[32260]: pgmap v284: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-12T04:24:24.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:24 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:24:24.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:24 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:24:24.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:24 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:24:25.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:25 smithi119 ceph-mon[36119]: pgmap v285: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:24:25.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:25 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:24:25.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:25 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:24:25.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:25 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:24:25.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:25 smithi136 ceph-mon[35571]: pgmap v285: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:24:25.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:25 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:24:25.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:25 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:24:25.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:25 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:24:25.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:25 smithi042 ceph-mon[32260]: pgmap v285: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-12T04:24:25.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:25 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:24:25.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:25 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:24:25.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:25 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:24:26.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:26 smithi119 ceph-mon[36119]: pgmap v286: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 100 B/s wr, 10 op/s 2024-03-12T04:24:26.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:26 smithi136 ceph-mon[35571]: pgmap v286: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 100 B/s wr, 10 op/s 2024-03-12T04:24:26.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:26 smithi042 ceph-mon[32260]: pgmap v286: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 100 B/s wr, 10 op/s 2024-03-12T04:24:28.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:28 smithi119 ceph-mon[36119]: pgmap v287: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:24:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:28 smithi136 ceph-mon[35571]: pgmap v287: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:24:28.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:28 smithi042 ceph-mon[32260]: pgmap v287: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-12T04:24:30.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:30 smithi119 ceph-mon[36119]: pgmap v288: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:30.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:30 smithi136 ceph-mon[35571]: pgmap v288: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:30.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:30 smithi042 ceph-mon[32260]: pgmap v288: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:32.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:32 smithi119 ceph-mon[36119]: pgmap v289: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:32.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:32 smithi136 ceph-mon[35571]: pgmap v289: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:32.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:32 smithi042 ceph-mon[32260]: pgmap v289: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:34.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:34 smithi119 ceph-mon[36119]: pgmap v290: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:34 smithi136 ceph-mon[35571]: pgmap v290: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:34.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:34 smithi042 ceph-mon[32260]: pgmap v290: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:36.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:36 smithi119 ceph-mon[36119]: pgmap v291: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:36 smithi136 ceph-mon[35571]: pgmap v291: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:36.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:36 smithi042 ceph-mon[32260]: pgmap v291: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:38.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:38 smithi119 ceph-mon[36119]: pgmap v292: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:38.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:38 smithi136 ceph-mon[35571]: pgmap v292: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:38.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:38 smithi042 ceph-mon[32260]: pgmap v292: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:40.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:40 smithi119 ceph-mon[36119]: pgmap v293: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:40.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:40 smithi136 ceph-mon[35571]: pgmap v293: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:40.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:40 smithi042 ceph-mon[32260]: pgmap v293: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:42.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:42 smithi119 ceph-mon[36119]: pgmap v294: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:42 smithi136 ceph-mon[35571]: pgmap v294: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:42.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:42 smithi042 ceph-mon[32260]: pgmap v294: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:44.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:44 smithi119 ceph-mon[36119]: pgmap v295: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:44.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:44 smithi136 ceph-mon[35571]: pgmap v295: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:44 smithi042 ceph-mon[32260]: pgmap v295: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:47.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:47 smithi119 ceph-mon[36119]: pgmap v296: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:47.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:47 smithi136 ceph-mon[35571]: pgmap v296: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:47.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:47 smithi042 ceph-mon[32260]: pgmap v296: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:48.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:24:48.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 -- 172.21.15.42:0/2158392600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 msgr2=0x7f180c10d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 --2- 172.21.15.42:0/2158392600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c10d5e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009900 tx=0x7f17fc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 -- 172.21.15.42:0/2158392600 shutdown_connections 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 --2- 172.21.15.42:0/2158392600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f180c10db20 0x7f180c10ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 --2- 172.21.15.42:0/2158392600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c10d5e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 --2- 172.21.15.42:0/2158392600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f180c0fff70 0x7f180c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 -- 172.21.15.42:0/2158392600 >> 172.21.15.42:0/2158392600 conn(0x7f180c077190 msgr2=0x7f180c0775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 -- 172.21.15.42:0/2158392600 shutdown_connections 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.534+0000 7f18133d7640 1 -- 172.21.15.42:0/2158392600 wait complete. 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 Processor -- start 2024-03-12T04:24:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 -- start start 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f180c0fff70 0x7f180c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f180c10db20 0x7f180c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c199e20 con 0x7f180c100890 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c199f90 con 0x7f180c0fff70 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f18133d7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19a150 con 0x7f180c10db20 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f1810972640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f1810972640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c19fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55838/0 (socket says 172.21.15.42:55838) 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.535+0000 7f1810972640 1 -- 172.21.15.42:0/1411193584 learned_addr learned my addr 172.21.15.42:0/1411193584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1811173640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f180c0fff70 0x7f180c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1811974640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f180c10db20 0x7f180c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1810972640 1 -- 172.21.15.42:0/1411193584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f180c10db20 msgr2=0x7f180c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1810972640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f180c10db20 0x7f180c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1810972640 1 -- 172.21.15.42:0/1411193584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f180c0fff70 msgr2=0x7f180c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1810972640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f180c0fff70 0x7f180c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1810972640 1 -- 172.21.15.42:0/1411193584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc009560 con 0x7f180c100890 2024-03-12T04:24:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f1810972640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c19fb10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006010 tx=0x7f17fc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f18027fc640 1 -- 172.21.15.42:0/1411193584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0044c0 con 0x7f180c100890 2024-03-12T04:24:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f18027fc640 1 -- 172.21.15.42:0/1411193584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17fc003aa0 con 0x7f180c100890 2024-03-12T04:24:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f18027fc640 1 -- 172.21.15.42:0/1411193584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc005540 con 0x7f180c100890 2024-03-12T04:24:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c19a3d0 con 0x7f180c100890 2024-03-12T04:24:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.536+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c1a6550 con 0x7f180c100890 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f18027fc640 1 -- 172.21.15.42:0/1411193584 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f17fc003c10 con 0x7f180c100890 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 Processor -- start 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 -- start start 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12ec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f18027fc640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17f80a1c20 0x7f17f80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c125b280 con 0x5579c12ec2b0 2024-03-12T04:24:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c12587b0 con 0x5579c12f57b0 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.539+0000 7f1815fad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c1253f90 con 0x5579c12ecbd0 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12ec690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55844/0 (socket says 172.21.15.42:55844) 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 -- 172.21.15.42:0/3712465045 learned_addr learned my addr 172.21.15.42:0/3712465045 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1810972640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17f80a1c20 0x7f17f80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f18027fc640 1 -- 172.21.15.42:0/1411193584 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f17fc0efa00 con 0x7f180c100890 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811173640 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 -- 172.21.15.42:0/3712465045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 msgr2=0x5579c12f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 -- 172.21.15.42:0/3712465045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 msgr2=0x5579c12f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 -- 172.21.15.42:0/3712465045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579c1212570 con 0x5579c12ec2b0 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1810972640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17f80a1c20 0x7f17f80a40e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0ed330 tx=0x7f17fc0a7040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f1811974640 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12ec690 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1804009f90 tx=0x7f1804009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d546cc3ee107edd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f17f77fe640 1 -- 172.21.15.42:0/3712465045 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804004b20 con 0x5579c12ec2b0 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f17f77fe640 1 -- 172.21.15.42:0/3712465045 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1804012d40 con 0x5579c12ec2b0 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.540+0000 7f17f77fe640 1 -- 172.21.15.42:0/3712465045 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804005570 con 0x5579c12ec2b0 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 -- 172.21.15.42:0/3712465045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 msgr2=0x5579c12ec690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12ec690 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1804009f90 tx=0x7f1804009830 comp rx=0 tx=0).stop 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 -- 172.21.15.42:0/3712465045 shutdown_connections 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 --2- 172.21.15.42:0/3712465045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12ec690 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 -- 172.21.15.42:0/3712465045 >> 172.21.15.42:0/3712465045 conn(0x5579c12e6990 msgr2=0x5579c12e6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 -- 172.21.15.42:0/3712465045 shutdown_connections 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 -- 172.21.15.42:0/3712465045 wait complete. 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 -- start start 2024-03-12T04:24:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.541+0000 7f1815fad740 1 Processor -- start 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1815fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ec2b0 0x5579c12f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1815fad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ecbd0 0x5579c12f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1815fad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12f57b0 0x5579c12f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1815fad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c0f27430 con 0x5579c12ecbd0 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1815fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c0ee3e00 con 0x5579c12ec2b0 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1815fad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c11d8fd0 con 0x5579c12f57b0 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1815fad740 10 client.0 _ll_register_callbacks cb 0x5579c122efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ecbd0 0x5579c12f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ecbd0 0x5579c12f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55856/0 (socket says 172.21.15.42:55856) 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 -- 172.21.15.42:0/2524981800 learned_addr learned my addr 172.21.15.42:0/2524981800 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811974640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12f57b0 0x5579c12f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811974640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ec2b0 0x5579c12f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 -- 172.21.15.42:0/2524981800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12f57b0 msgr2=0x5579c12f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12f57b0 0x5579c12f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 -- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ec2b0 msgr2=0x5579c12f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ec2b0 0x5579c12f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811173640 1 -- 172.21.15.42:0/2524981800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1804014410 con 0x5579c12ecbd0 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.542+0000 7f1811974640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ec2b0 0x5579c12f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1811173640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ecbd0 0x5579c12f25a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f180800bad0 tx=0x7f180800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c277fcd79d2d116 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f17f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f17f4ff9640 1 -- 172.21.15.42:0/2524981800 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808016070 con 0x5579c12ecbd0 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f17f4ff9640 1 -- 172.21.15.42:0/2524981800 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1808003e00 con 0x5579c12ecbd0 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579c0f537e0 con 0x5579c12ecbd0 2024-03-12T04:24:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f17f4ff9640 1 -- 172.21.15.42:0/2524981800 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1808005020 con 0x5579c12ecbd0 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 10 client.18135 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5579c1252610 con 0x5579c12ecbd0 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f17f4ff9640 1 -- 172.21.15.42:0/2524981800 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5579c0f537e0 con 0x5579c12ecbd0 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f17f4ff9640 1 -- 172.21.15.42:0/2524981800 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f180801b470 con 0x5579c12ecbd0 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 10 client.18135 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 20 client.18135 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c1305920 0x5579c1307d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 4 client.18135 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.543+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579c0f90720 con 0x5579c1305920 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.544+0000 7f1811974640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c1305920 0x5579c1307d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.544+0000 7f1811974640 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c1305920 0x5579c1307d40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a07c7a56176431 server_cookie=70154535c8925ed3 in_seq=0 out_seq=0 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.544+0000 7f17f4ff9640 10 client.18135 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f17f4ff9640 1 -- 172.21.15.42:0/2524981800 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579c0f90720 con 0x5579c1305920 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f17f4ff9640 10 client.18135 handle_command_reply: tid=0 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 client.18135 shutdown 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ecbd0 msgr2=0x5579c12f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ecbd0 0x5579c12f25a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f180800bad0 tx=0x7f180800bf90 comp rx=0 tx=0).stop 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 shutdown_connections 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c1305920 0x5579c1307d40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12f57b0 0x5579c12f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ecbd0 0x5579c12f25a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f17f4ff9640 0 client.18135 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 --2- 172.21.15.42:0/2524981800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ec2b0 0x5579c12f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 >> 172.21.15.42:0/2524981800 conn(0x5579c12e6990 msgr2=0x5579c12e8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 shutdown_connections 2024-03-12T04:24:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 1 -- 172.21.15.42:0/2524981800 wait complete. 2024-03-12T04:24:48.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.545+0000 7f1815fad740 20 client.18135 trim_cache size 0 max 16384 2024-03-12T04:24:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.560+0000 7f1815fad740 1 Processor -- start 2024-03-12T04:24:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.560+0000 7f1815fad740 1 -- start start 2024-03-12T04:24:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1815fad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1815fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1815fad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12f57b0 0x5579c12ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1815fad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c12e2410 con 0x5579c12f57b0 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1815fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c1302480 con 0x5579c12ecbd0 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1815fad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c12f3a30 con 0x5579c12ec2b0 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12f57b0 0x5579c12ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12f57b0 0x5579c12ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55860/0 (socket says 172.21.15.42:55860) 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 -- 172.21.15.42:0/4277708006 learned_addr learned my addr 172.21.15.42:0/4277708006 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811173640 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 -- 172.21.15.42:0/4277708006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ec2b0 msgr2=0x5579c12fc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 -- 172.21.15.42:0/4277708006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ecbd0 msgr2=0x5579c12fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 -- 172.21.15.42:0/4277708006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579c11eeac0 con 0x5579c12f57b0 2024-03-12T04:24:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f1811974640 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12f57b0 0x5579c12ef800 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1804012760 tx=0x7f1804021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae0bb95e4f7be6c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f17f6ffd640 1 -- 172.21.15.42:0/4277708006 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804012890 con 0x5579c12f57b0 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f17f6ffd640 1 -- 172.21.15.42:0/4277708006 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f18040129f0 con 0x5579c12f57b0 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.561+0000 7f17f6ffd640 1 -- 172.21.15.42:0/4277708006 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180402d920 con 0x5579c12f57b0 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 -- 172.21.15.42:0/4277708006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12f57b0 msgr2=0x5579c12ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12f57b0 0x5579c12ef800 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1804012760 tx=0x7f1804021040 comp rx=0 tx=0).stop 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 -- 172.21.15.42:0/4277708006 shutdown_connections 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12f57b0 0x5579c12ef800 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 --2- 172.21.15.42:0/4277708006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 -- 172.21.15.42:0/4277708006 >> 172.21.15.42:0/4277708006 conn(0x5579c12e6990 msgr2=0x5579c12f0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 -- 172.21.15.42:0/4277708006 shutdown_connections 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 -- 172.21.15.42:0/4277708006 wait complete. 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 -- start start 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.562+0000 7f1815fad740 1 Processor -- start 2024-03-12T04:24:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1815fad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1815fad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1815fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1815fad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c12fa2e0 con 0x5579c12ec2b0 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1815fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c12fa450 con 0x5579c12f57b0 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1815fad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579c12fa610 con 0x5579c12ecbd0 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1815fad740 10 client.0 _ll_register_callbacks cb 0x7f1804009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55866/0 (socket says 172.21.15.42:55866) 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 -- 172.21.15.42:0/230815496 learned_addr learned my addr 172.21.15.42:0/230815496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811173640 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 -- 172.21.15.42:0/230815496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 msgr2=0x5579c12fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 -- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 msgr2=0x5579c12f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 -- 172.21.15.42:0/230815496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1804035040 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f1811974640 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f18040050b0 tx=0x7f180402d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba67b97f75eaa9c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.563+0000 7f17f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f17f4ff9640 1 -- 172.21.15.42:0/230815496 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180402d560 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f17f4ff9640 1 -- 172.21.15.42:0/230815496 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f180402d8a0 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579c1301e10 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f17f4ff9640 1 -- 172.21.15.42:0/230815496 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18040049e0 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 10 client.18141 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5579c12146a0 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f17f4ff9640 1 -- 172.21.15.42:0/230815496 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5579c1301e10 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f17f4ff9640 1 -- 172.21.15.42:0/230815496 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1804045460 con 0x5579c12ec2b0 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 10 client.18141 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 20 client.18141 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c135de20 0x5579c1360210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 4 client.18141 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f1804012d40 con 0x5579c135de20 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.564+0000 7f1811173640 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c135de20 0x5579c1360210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f17f4ff9640 10 client.18141 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1811173640 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c135de20 0x5579c1360210 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab5a8358d996062 server_cookie=c9dec9275b5a871 in_seq=0 out_seq=0 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f17f4ff9640 1 -- 172.21.15.42:0/230815496 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f1804012d40 con 0x5579c135de20 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f17f4ff9640 10 client.18141 handle_command_reply: tid=0 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 client.18141 shutdown 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 msgr2=0x5579c12fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f18040050b0 tx=0x7f180402d7a0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 shutdown_connections 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5579c135de20 0x5579c1360210 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579c12f57b0 0x5579c12f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f17f4ff9640 0 client.18141 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579c12ecbd0 0x5579c12fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 --2- 172.21.15.42:0/230815496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579c12ec2b0 0x5579c12fc650 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 >> 172.21.15.42:0/230815496 conn(0x5579c12e6990 msgr2=0x5579c12e86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 shutdown_connections 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 1 -- 172.21.15.42:0/230815496 wait complete. 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.565+0000 7f1815fad740 20 client.18141 trim_cache size 0 max 16384 2024-03-12T04:24:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:24:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:24:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:24:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:24:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17f80a1c20 msgr2=0x7f17f80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17f80a1c20 0x7f17f80a40e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0ed330 tx=0x7f17fc0a7040 comp rx=0 tx=0).stop 2024-03-12T04:24:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 msgr2=0x7f180c19fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c19fb10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006010 tx=0x7f17fc0040c0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 shutdown_connections 2024-03-12T04:24:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f17f80a1c20 0x7f17f80a40e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f180c10db20 0x7f180c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f180c100890 0x7f180c19fb10 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 --2- 172.21.15.42:0/1411193584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f180c0fff70 0x7f180c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 >> 172.21.15.42:0/1411193584 conn(0x7f180c077190 msgr2=0x7f180c10f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 shutdown_connections 2024-03-12T04:24:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.566+0000 7f18133d7640 1 -- 172.21.15.42:0/1411193584 wait complete. 2024-03-12T04:24:48.582 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:24:48.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:24:48.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 -- 172.21.15.42:0/1312039842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c100ca0 msgr2=0x7f296c104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 --2- 172.21.15.42:0/1312039842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c100ca0 0x7f296c104ce0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2960009930 tx=0x7f296002bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:48.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 -- 172.21.15.42:0/1312039842 shutdown_connections 2024-03-12T04:24:48.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 --2- 172.21.15.42:0/1312039842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c1053c0 0x7f296c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 --2- 172.21.15.42:0/1312039842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c100ca0 0x7f296c104ce0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 --2- 172.21.15.42:0/1312039842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296c1002f0 0x7f296c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 -- 172.21.15.42:0/1312039842 >> 172.21.15.42:0/1312039842 conn(0x7f296c077180 msgr2=0x7f296c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 -- 172.21.15.42:0/1312039842 shutdown_connections 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.683+0000 7f2970c88640 1 -- 172.21.15.42:0/1312039842 wait complete. 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 Processor -- start 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 -- start start 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c1002f0 0x7f296c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296c1053c0 0x7f296c1022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c102800 0x7f296c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1a5f60 con 0x7f296c1002f0 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1a60d0 con 0x7f296c1053c0 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2970c88640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1a6290 con 0x7f296c102800 2024-03-12T04:24:48.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c1002f0 0x7f296c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c1002f0 0x7f296c101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55874/0 (socket says 172.21.15.42:55874) 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 -- 172.21.15.42:0/2080172886 learned_addr learned my addr 172.21.15.42:0/2080172886 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296ad9d640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c102800 0x7f296c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f2969d9b640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296c1053c0 0x7f296c1022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 -- 172.21.15.42:0/2080172886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c102800 msgr2=0x7f296c1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c102800 0x7f296c1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 -- 172.21.15.42:0/2080172886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296c1053c0 msgr2=0x7f296c1022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296c1053c0 0x7f296c1022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 -- 172.21.15.42:0/2080172886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2960009560 con 0x7f296c1002f0 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296ad9d640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c102800 0x7f296c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.684+0000 7f296a59c640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c1002f0 0x7f296c101d80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f295c00d950 tx=0x7f295c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.685+0000 7f295b7fe640 1 -- 172.21.15.42:0/2080172886 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c009c30 con 0x7f296c1002f0 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.685+0000 7f295b7fe640 1 -- 172.21.15.42:0/2080172886 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f295c009d90 con 0x7f296c1002f0 2024-03-12T04:24:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.685+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c1a6570 con 0x7f296c1002f0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.685+0000 7f295b7fe640 1 -- 172.21.15.42:0/2080172886 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00f6b0 con 0x7f296c1002f0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.685+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c1a6a80 con 0x7f296c1002f0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.686+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c067f70 con 0x7f296c1002f0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.687+0000 7f295b7fe640 1 -- 172.21.15.42:0/2080172886 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f295c010aa0 con 0x7f296c1002f0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.687+0000 7f295b7fe640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29540a1bd0 0x7f29540a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.687+0000 7f2969d9b640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29540a1bd0 0x7f29540a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.687+0000 7f295b7fe640 1 -- 172.21.15.42:0/2080172886 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f295c0cd520 con 0x7f296c1002f0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.688+0000 7f2969d9b640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29540a1bd0 0x7f29540a4090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2960036040 tx=0x7f2960005fd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:48.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.692+0000 7f295b7fe640 1 -- 172.21.15.42:0/2080172886 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f295c096410 con 0x7f296c1002f0 2024-03-12T04:24:48.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.911+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f296c109600 con 0x7f296c1002f0 2024-03-12T04:24:48.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.912+0000 7f295b7fe640 1 -- 172.21.15.42:0/2080172886 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f295c099ad0 con 0x7f296c1002f0 2024-03-12T04:24:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:24:48.915 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:24:48.915 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:24:48.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29540a1bd0 msgr2=0x7f29540a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29540a1bd0 0x7f29540a4090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2960036040 tx=0x7f2960005fd0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c1002f0 msgr2=0x7f296c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c1002f0 0x7f296c101d80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f295c00d950 tx=0x7f295c00de10 comp rx=0 tx=0).stop 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 shutdown_connections 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f29540a1bd0 0x7f29540a4090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c102800 0x7f296c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296c1053c0 0x7f296c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 --2- 172.21.15.42:0/2080172886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f296c1002f0 0x7f296c101d80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 >> 172.21.15.42:0/2080172886 conn(0x7f296c077180 msgr2=0x7f296c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 shutdown_connections 2024-03-12T04:24:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:48.914+0000 7f2970c88640 1 -- 172.21.15.42:0/2080172886 wait complete. 2024-03-12T04:24:48.927 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:48.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:24:49.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 -- 172.21.15.42:0/304137889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 msgr2=0x7f75901006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 --2- 172.21.15.42:0/304137889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901006f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7580009930 tx=0x7f758002bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 -- 172.21.15.42:0/304137889 shutdown_connections 2024-03-12T04:24:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 --2- 172.21.15.42:0/304137889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75901053e0 0x7f759010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 --2- 172.21.15.42:0/304137889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7590100cc0 0x7f7590104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 --2- 172.21.15.42:0/304137889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901006f0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 -- 172.21.15.42:0/304137889 >> 172.21.15.42:0/304137889 conn(0x7f75900771e0 msgr2=0x7f75900775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 -- 172.21.15.42:0/304137889 shutdown_connections 2024-03-12T04:24:49.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.030+0000 7f75948b5640 1 -- 172.21.15.42:0/304137889 wait complete. 2024-03-12T04:24:49.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 Processor -- start 2024-03-12T04:24:49.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 -- start start 2024-03-12T04:24:49.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75901053e0 0x7f7590198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75901990b0 0x7f75901a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75901a3d30 con 0x7f7590100310 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75901a3ea0 con 0x7f75901990b0 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f75948b5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75901a4060 con 0x7f75901053e0 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f758e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f758e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55888/0 (socket says 172.21.15.42:55888) 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f758e59c640 1 -- 172.21.15.42:0/2214244078 learned_addr learned my addr 172.21.15.42:0/2214244078 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f758ed9d640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75901990b0 0x7f75901a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.031+0000 7f7587fff640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75901053e0 0x7f7590198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f758e59c640 1 -- 172.21.15.42:0/2214244078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75901053e0 msgr2=0x7f7590198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f758e59c640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75901053e0 0x7f7590198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f758e59c640 1 -- 172.21.15.42:0/2214244078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75901990b0 msgr2=0x7f75901a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f758e59c640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75901990b0 0x7f75901a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f758e59c640 1 -- 172.21.15.42:0/2214244078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580009560 con 0x7f7590100310 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f758e59c640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901985e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f7580009230 tx=0x7f75800040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f75877fe640 1 -- 172.21.15.42:0/2214244078 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800044c0 con 0x7f7590100310 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f75877fe640 1 -- 172.21.15.42:0/2214244078 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f758002de90 con 0x7f7590100310 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75901a42e0 con 0x7f7590100310 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f75877fe640 1 -- 172.21.15.42:0/2214244078 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580005400 con 0x7f7590100310 2024-03-12T04:24:49.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.032+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75901a47f0 con 0x7f7590100310 2024-03-12T04:24:49.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.034+0000 7f75877fe640 1 -- 172.21.15.42:0/2214244078 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7580005560 con 0x7f7590100310 2024-03-12T04:24:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 Processor -- start 2024-03-12T04:24:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 -- start start 2024-03-12T04:24:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f75877fe640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f757c0a1ba0 0x7f757c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 0x561c8ab6a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab73210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab73750 0x561c8ab75b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8ab25960 con 0x561c8ab6ab70 2024-03-12T04:24:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8aad9310 con 0x561c8ab73750 2024-03-12T04:24:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7597567740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8aad6840 con 0x561c8ab6a250 2024-03-12T04:24:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f758ed9d640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f757c0a1ba0 0x7f757c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7587fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab73210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7587fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab73210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55898/0 (socket says 172.21.15.42:55898) 2024-03-12T04:24:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f7587fff640 1 -- 172.21.15.42:0/824164626 learned_addr learned my addr 172.21.15.42:0/824164626 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.035+0000 7f75877fe640 1 -- 172.21.15.42:0/2214244078 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f75800ef7f0 con 0x7f7590100310 2024-03-12T04:24:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f758e59c640 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 0x561c8ab6a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f758ed9d640 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab73750 0x561c8ab75b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7587fff640 1 -- 172.21.15.42:0/824164626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 msgr2=0x561c8ab6a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7587fff640 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 0x561c8ab6a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7587fff640 1 -- 172.21.15.42:0/824164626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab73750 msgr2=0x561c8ab75b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7587fff640 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab73750 0x561c8ab75b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7587fff640 1 -- 172.21.15.42:0/824164626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c8aa94060 con 0x561c8ab6ab70 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f758ed9d640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f757c0a1ba0 0x7f757c0a4060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7588005fd0 tx=0x7f7588009f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7587fff640 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab73210 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f757800cb90 tx=0x7f757800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598c711140760603 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7584ff9640 1 -- 172.21.15.42:0/824164626 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578010630 con 0x561c8ab6ab70 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7584ff9640 1 -- 172.21.15.42:0/824164626 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f757800ce90 con 0x561c8ab6ab70 2024-03-12T04:24:49.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.036+0000 7f7584ff9640 1 -- 172.21.15.42:0/824164626 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757800f970 con 0x561c8ab6ab70 2024-03-12T04:24:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- 172.21.15.42:0/824164626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 msgr2=0x561c8ab73210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab73210 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f757800cb90 tx=0x7f757800e720 comp rx=0 tx=0).stop 2024-03-12T04:24:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- 172.21.15.42:0/824164626 shutdown_connections 2024-03-12T04:24:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab73750 0x561c8ab75b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab73210 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 --2- 172.21.15.42:0/824164626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 0x561c8ab6a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- 172.21.15.42:0/824164626 >> 172.21.15.42:0/824164626 conn(0x561c8ab64980 msgr2=0x561c8ab64d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- 172.21.15.42:0/824164626 shutdown_connections 2024-03-12T04:24:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- 172.21.15.42:0/824164626 wait complete. 2024-03-12T04:24:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- start start 2024-03-12T04:24:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 Processor -- start 2024-03-12T04:24:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6a250 0x561c8ab70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab70ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8ab711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8aad2020 con 0x561c8ab6a250 2024-03-12T04:24:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8a7a5430 con 0x561c8ab6ab70 2024-03-12T04:24:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.037+0000 7f7597567740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8a761e00 con 0x561c8ab73750 2024-03-12T04:24:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7597567740 10 client.0 _ll_register_callbacks cb 0x561c8aa79cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6a250 0x561c8ab70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6a250 0x561c8ab70760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55902/0 (socket says 172.21.15.42:55902) 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 -- 172.21.15.42:0/369256630 learned_addr learned my addr 172.21.15.42:0/369256630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758ed9d640 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8ab711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7587fff640 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab70ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 -- 172.21.15.42:0/369256630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 msgr2=0x561c8ab711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8ab711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 -- 172.21.15.42:0/369256630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 msgr2=0x561c8ab70ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab70ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 -- 172.21.15.42:0/369256630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757800c7c0 con 0x561c8ab6a250 2024-03-12T04:24:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f758e59c640 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6a250 0x561c8ab70760 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7580000c00 tx=0x7f75800bbe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191ae724060e97cf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7575ffb640 1 -- 172.21.15.42:0/369256630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800bcbf0 con 0x561c8ab6a250 2024-03-12T04:24:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7575ffb640 1 -- 172.21.15.42:0/369256630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f75800bc4f0 con 0x561c8ab6a250 2024-03-12T04:24:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c8a7d17e0 con 0x561c8ab6a250 2024-03-12T04:24:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7575ffb640 1 -- 172.21.15.42:0/369256630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75800c0640 con 0x561c8ab6a250 2024-03-12T04:24:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7597567740 10 client.18159 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561c8aad06a0 con 0x561c8ab6a250 2024-03-12T04:24:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.038+0000 7f7575ffb640 1 -- 172.21.15.42:0/369256630 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561c8a7d17e0 con 0x561c8ab6a250 2024-03-12T04:24:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7575ffb640 1 -- 172.21.15.42:0/369256630 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f758003c070 con 0x561c8ab6a250 2024-03-12T04:24:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7597567740 10 client.18159 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7597567740 10 client.18159 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7597567740 20 client.18159 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7597567740 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8ab83870 0x561c8ab85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7597567740 4 client.18159 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c8a80e720 con 0x561c8ab83870 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7587fff640 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8ab83870 0x561c8ab85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7587fff640 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8ab83870 0x561c8ab85cb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec9f480567e966f1 server_cookie=3b3bf790cf0896d in_seq=0 out_seq=0 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.039+0000 7f7575ffb640 10 client.18159 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7575ffb640 1 -- 172.21.15.42:0/369256630 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c8a80e720 con 0x561c8ab83870 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7575ffb640 10 client.18159 handle_command_reply: tid=0 2024-03-12T04:24:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 client.18159 shutdown 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6a250 msgr2=0x561c8ab70760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6a250 0x561c8ab70760 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7580000c00 tx=0x7f75800bbe50 comp rx=0 tx=0).stop 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 shutdown_connections 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8ab711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab70ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8ab83870 0x561c8ab85cb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 --2- 172.21.15.42:0/369256630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6a250 0x561c8ab70760 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7575ffb640 0 client.18159 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 >> 172.21.15.42:0/369256630 conn(0x561c8ab64980 msgr2=0x561c8ab666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 shutdown_connections 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 1 -- 172.21.15.42:0/369256630 wait complete. 2024-03-12T04:24:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.040+0000 7f7597567740 20 client.18159 trim_cache size 0 max 16384 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 Processor -- start 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 -- start start 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 0x561c8ab71000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab63a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab73750 0x561c8ab7a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8aa57130 con 0x561c8ab73750 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8ab69540 con 0x561c8ab6ab70 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7597567740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8ab65980 con 0x561c8ab6a250 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab73750 0x561c8ab7a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab73750 0x561c8ab7a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55914/0 (socket says 172.21.15.42:55914) 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 -- 172.21.15.42:0/3426724752 learned_addr learned my addr 172.21.15.42:0/3426724752 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 -- 172.21.15.42:0/3426724752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 msgr2=0x561c8ab71000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 0x561c8ab71000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 -- 172.21.15.42:0/3426724752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 msgr2=0x561c8ab63a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7587fff640 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab63a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab63a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 -- 172.21.15.42:0/3426724752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c8aa6cc20 con 0x561c8ab73750 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f7587fff640 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.052+0000 7f758ed9d640 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab73750 0x561c8ab7a630 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7588012700 tx=0x7f7588017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c7fe2116dba8c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7577fff640 1 -- 172.21.15.42:0/3426724752 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758801e720 con 0x561c8ab73750 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7577fff640 1 -- 172.21.15.42:0/3426724752 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f758801ed30 con 0x561c8ab73750 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7577fff640 1 -- 172.21.15.42:0/3426724752 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588020e60 con 0x561c8ab73750 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 -- 172.21.15.42:0/3426724752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab73750 msgr2=0x561c8ab7a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab73750 0x561c8ab7a630 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7588012700 tx=0x7f7588017000 comp rx=0 tx=0).stop 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 -- 172.21.15.42:0/3426724752 shutdown_connections 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab73750 0x561c8ab7a630 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6ab70 0x561c8ab63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 --2- 172.21.15.42:0/3426724752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab6a250 0x561c8ab71000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 -- 172.21.15.42:0/3426724752 >> 172.21.15.42:0/3426724752 conn(0x561c8ab64980 msgr2=0x561c8ab632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 -- 172.21.15.42:0/3426724752 shutdown_connections 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 -- 172.21.15.42:0/3426724752 wait complete. 2024-03-12T04:24:49.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 -- start start 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.053+0000 7f7597567740 1 Processor -- start 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7597567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6a250 0x561c8ab79760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7597567740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab79ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7597567740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8abd9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7597567740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8abda2e0 con 0x561c8ab6ab70 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7597567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8abda420 con 0x561c8ab6a250 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7597567740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8abda560 con 0x561c8ab73750 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7597567740 10 client.0 _ll_register_callbacks cb 0x7f757800c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab79ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab79ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55922/0 (socket says 172.21.15.42:55922) 2024-03-12T04:24:49.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 -- 172.21.15.42:0/4231970648 learned_addr learned my addr 172.21.15.42:0/4231970648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f758ed9d640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8abd9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f758e59c640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6a250 0x561c8ab79760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 -- 172.21.15.42:0/4231970648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 msgr2=0x561c8abd9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8abd9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 -- 172.21.15.42:0/4231970648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6a250 msgr2=0x561c8ab79760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6a250 0x561c8ab79760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 -- 172.21.15.42:0/4231970648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758801a070 con 0x561c8ab6ab70 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f758e59c640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6a250 0x561c8ab79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7587fff640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab79ca0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f7578005f90 tx=0x7f75780054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a40204e471399e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.054+0000 7f7575ffb640 1 -- 172.21.15.42:0/4231970648 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578020040 con 0x561c8ab6ab70 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7575ffb640 1 -- 172.21.15.42:0/4231970648 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7578025830 con 0x561c8ab6ab70 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f757800ce90 con 0x561c8ab6ab70 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7575ffb640 1 -- 172.21.15.42:0/4231970648 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7578005640 con 0x561c8ab6ab70 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 10 client.18165 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561c8aa928c0 con 0x561c8ab6ab70 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7575ffb640 1 -- 172.21.15.42:0/4231970648 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f757800ce90 con 0x561c8ab6ab70 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7575ffb640 1 -- 172.21.15.42:0/4231970648 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f75780046f0 con 0x561c8ab6ab70 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 10 client.18165 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 10 client.18165 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 20 client.18165 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8abdbd90 0x561c8abde1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 4 client.18165 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561c8a85d890 con 0x561c8abdbd90 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f758e59c640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8abdbd90 0x561c8abde1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f7575ffb640 10 client.18165 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.055+0000 7f758e59c640 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8abdbd90 0x561c8abde1b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b84103198763bc0f server_cookie=9aa98f8f797c582b in_seq=0 out_seq=0 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7575ffb640 1 -- 172.21.15.42:0/4231970648 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561c8a85d890 con 0x561c8abdbd90 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7575ffb640 10 client.18165 handle_command_reply: tid=0 2024-03-12T04:24:49.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 client.18165 shutdown 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 msgr2=0x561c8ab79ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab79ca0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f7578005f90 tx=0x7f75780054d0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 shutdown_connections 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c8ab73750 0x561c8abd9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x561c8abdbd90 0x561c8abde1b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c8ab6ab70 0x561c8ab79ca0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7575ffb640 0 client.18165 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 --2- 172.21.15.42:0/4231970648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c8ab6a250 0x561c8ab79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 >> 172.21.15.42:0/4231970648 conn(0x561c8ab64980 msgr2=0x561c8ab63b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 shutdown_connections 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 1 -- 172.21.15.42:0/4231970648 wait complete. 2024-03-12T04:24:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f7597567740 20 client.18165 trim_cache size 0 max 16384 2024-03-12T04:24:49.076 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:49.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f757c0a1ba0 msgr2=0x7f757c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f757c0a1ba0 0x7f757c0a4060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7588005fd0 tx=0x7f7588009f90 comp rx=0 tx=0).stop 2024-03-12T04:24:49.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 msgr2=0x7f75901985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901985e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f7580009230 tx=0x7f75800040c0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 shutdown_connections 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f757c0a1ba0 0x7f757c0a4060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75901990b0 0x7f75901a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75901053e0 0x7f7590198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 --2- 172.21.15.42:0/2214244078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7590100310 0x7f75901985e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.056+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 >> 172.21.15.42:0/2214244078 conn(0x7f75900771e0 msgr2=0x7f75900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.057+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 shutdown_connections 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.057+0000 7f75948b5640 1 -- 172.21.15.42:0/2214244078 wait complete. 2024-03-12T04:24:49.078 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:49.079 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:24:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 -- 172.21.15.42:0/3051234099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 msgr2=0x7f09c40689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 --2- 172.21.15.42:0/3051234099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c40689d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009cd0 tx=0x7f09c002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 -- 172.21.15.42:0/3051234099 shutdown_connections 2024-03-12T04:24:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 --2- 172.21.15.42:0/3051234099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c41064c0 0x7f09c410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 --2- 172.21.15.42:0/3051234099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4068fa0 0x7f09c4105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 --2- 172.21.15.42:0/3051234099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c40689d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 -- 172.21.15.42:0/3051234099 >> 172.21.15.42:0/3051234099 conn(0x7f09c4077180 msgr2=0x7f09c4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 -- 172.21.15.42:0/3051234099 shutdown_connections 2024-03-12T04:24:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.181+0000 7f09cb735640 1 -- 172.21.15.42:0/3051234099 wait complete. 2024-03-12T04:24:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 Processor -- start 2024-03-12T04:24:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 -- start start 2024-03-12T04:24:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c41064c0 0x7f09c419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c4199750 0x7f09c4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c419a220 con 0x7f09c40685f0 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c419a390 con 0x7f09c4199750 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09cb735640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c419a550 con 0x7f09c41064c0 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09c94d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09c94d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55940/0 (socket says 172.21.15.42:55940) 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09c94d1640 1 -- 172.21.15.42:0/334381173 learned_addr learned my addr 172.21.15.42:0/334381173 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09c8cd0640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c41064c0 0x7f09c419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.182+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c4199750 0x7f09c4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c94d1640 1 -- 172.21.15.42:0/334381173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c41064c0 msgr2=0x7f09c419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c94d1640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c41064c0 0x7f09c419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c94d1640 1 -- 172.21.15.42:0/334381173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c4199750 msgr2=0x7f09c4199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c94d1640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c4199750 0x7f09c4199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c94d1640 1 -- 172.21.15.42:0/334381173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0009900 con 0x7f09c40685f0 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c8cd0640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c41064c0 0x7f09c419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c4199750 0x7f09c4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09c94d1640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c419f690 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f09c002c700 tx=0x7f09c0035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09ba7fc640 1 -- 172.21.15.42:0/334381173 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00329e0 con 0x7f09c40685f0 2024-03-12T04:24:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c41a6490 con 0x7f09c40685f0 2024-03-12T04:24:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09ba7fc640 1 -- 172.21.15.42:0/334381173 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09c0035e90 con 0x7f09c40685f0 2024-03-12T04:24:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09ba7fc640 1 -- 172.21.15.42:0/334381173 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c003f440 con 0x7f09c40685f0 2024-03-12T04:24:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.183+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c41a6910 con 0x7f09c40685f0 2024-03-12T04:24:49.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.185+0000 7f09ba7fc640 1 -- 172.21.15.42:0/334381173 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f09c003b040 con 0x7f09c40685f0 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 Processor -- start 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ba7fc640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f09b00a1ba0 0x7f09b00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 -- start start 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f09b00a1ba0 0x7f09b00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ba7fc640 1 -- 172.21.15.42:0/334381173 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f09c00f0270 con 0x7f09c40685f0 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffde250 0x557d9ffde630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ff99960 con 0x557d9ffe7750 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ff4d310 con 0x557d9ffde250 2024-03-12T04:24:49.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ff4a840 con 0x557d9ffdeb70 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09c9cd2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09c9cd2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55946/0 (socket says 172.21.15.42:55946) 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09c9cd2640 1 -- 172.21.15.42:0/1442662720 learned_addr learned my addr 172.21.15.42:0/1442662720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09c8cd0640 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09c94d1640 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffde250 0x557d9ffde630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.186+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f09b00a1ba0 0x7f09b00a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09bc013530 tx=0x7f09bc013930 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09c9cd2640 1 -- 172.21.15.42:0/1442662720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 msgr2=0x557d9ffe7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09c9cd2640 1 -- 172.21.15.42:0/1442662720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffde250 msgr2=0x557d9ffde630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffde250 0x557d9ffde630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09c9cd2640 1 -- 172.21.15.42:0/1442662720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d9ff08060 con 0x557d9ffe7750 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe9b40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00bab0 tx=0x7f09bc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb3d2d126dea0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09af7fe640 1 -- 172.21.15.42:0/1442662720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc004020 con 0x557d9ffe7750 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09af7fe640 1 -- 172.21.15.42:0/1442662720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09bc019070 con 0x557d9ffe7750 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.187+0000 7f09af7fe640 1 -- 172.21.15.42:0/1442662720 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0037f0 con 0x557d9ffe7750 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- 172.21.15.42:0/1442662720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 msgr2=0x557d9ffe9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe9b40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00bab0 tx=0x7f09bc01d000 comp rx=0 tx=0).stop 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- 172.21.15.42:0/1442662720 shutdown_connections 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe9b40 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/1442662720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffde250 0x557d9ffde630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- 172.21.15.42:0/1442662720 >> 172.21.15.42:0/1442662720 conn(0x557d9ffd8980 msgr2=0x557d9ffd8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- 172.21.15.42:0/1442662720 shutdown_connections 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- 172.21.15.42:0/1442662720 wait complete. 2024-03-12T04:24:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- start start 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 Processor -- start 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffde250 0x557d9ffe47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffe4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ff46020 con 0x557d9ffe7750 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9fc19430 con 0x557d9ffdeb70 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9fbd5e00 con 0x557d9ffde250 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.188+0000 7f09ce3e0740 10 client.0 _ll_register_callbacks cb 0x557d9feedcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c8cd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffe4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55948/0 (socket says 172.21.15.42:55948) 2024-03-12T04:24:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 -- 172.21.15.42:0/3310332772 learned_addr learned my addr 172.21.15.42:0/3310332772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c94d1640 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffde250 0x557d9ffe47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 -- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffde250 msgr2=0x557d9ffe47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffde250 0x557d9ffe47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 -- 172.21.15.42:0/3310332772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 msgr2=0x557d9ffe4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffe4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 -- 172.21.15.42:0/3310332772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc011580 con 0x557d9ffe7750 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe5220 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f09bc011e60 tx=0x7f09bc00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289ca06de35f56c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09acff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09acff9640 1 -- 172.21.15.42:0/3310332772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00dbe0 con 0x557d9ffe7750 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d9fc457e0 con 0x557d9ffe7750 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09acff9640 1 -- 172.21.15.42:0/3310332772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f09bc00dd40 con 0x557d9ffe7750 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09acff9640 1 -- 172.21.15.42:0/3310332772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09bc005040 con 0x557d9ffe7750 2024-03-12T04:24:49.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09ce3e0740 10 client.18177 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.189+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557d9ff446a0 con 0x557d9ffe7750 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09acff9640 1 -- 172.21.15.42:0/3310332772 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09bc03d020 con 0x557d9ffe7750 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09acff9640 1 -- 172.21.15.42:0/3310332772 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09bc025020 con 0x557d9ffe7750 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09ce3e0740 10 client.18177 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09ce3e0740 10 client.18177 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09ce3e0740 20 client.18177 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557d9fff7b50 0x557d9fff9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09ce3e0740 4 client.18177 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d9fc82720 con 0x557d9fff7b50 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09c94d1640 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557d9fff7b50 0x557d9fff9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09acff9640 10 client.18177 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.190+0000 7f09c94d1640 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557d9fff7b50 0x557d9fff9f70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=71372592f22cd1de server_cookie=f222b976c2f54be8 in_seq=0 out_seq=0 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09acff9640 1 -- 172.21.15.42:0/3310332772 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d9fc82720 con 0x557d9fff7b50 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09acff9640 10 client.18177 handle_command_reply: tid=0 2024-03-12T04:24:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 client.18177 shutdown 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 msgr2=0x557d9ffe5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe5220 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f09bc011e60 tx=0x7f09bc00b590 comp rx=0 tx=0).stop 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 shutdown_connections 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557d9fff7b50 0x557d9fff9f70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffe7750 0x557d9ffe5220 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffe4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/3310332772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffde250 0x557d9ffe47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09acff9640 0 client.18177 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 >> 172.21.15.42:0/3310332772 conn(0x557d9ffd8980 msgr2=0x557d9ffda6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 shutdown_connections 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 1 -- 172.21.15.42:0/3310332772 wait complete. 2024-03-12T04:24:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.191+0000 7f09ce3e0740 20 client.18177 trim_cache size 0 max 16384 2024-03-12T04:24:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 Processor -- start 2024-03-12T04:24:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 -- start start 2024-03-12T04:24:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffe6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffe7750 0x557d9ffe1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9fecb130 con 0x557d9ffde250 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9fff35b0 con 0x557d9ffe7750 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ffd8840 con 0x557d9ffdeb70 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09c94d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffe6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09c94d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffe6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55958/0 (socket says 172.21.15.42:55958) 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.203+0000 7f09c94d1640 1 -- 172.21.15.42:0/2413852326 learned_addr learned my addr 172.21.15.42:0/2413852326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffe7750 0x557d9ffe1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c8cd0640 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c94d1640 1 -- 172.21.15.42:0/2413852326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 msgr2=0x557d9ffe5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c94d1640 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c94d1640 1 -- 172.21.15.42:0/2413852326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffe7750 msgr2=0x557d9ffe1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c94d1640 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffe7750 0x557d9ffe1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c94d1640 1 -- 172.21.15.42:0/2413852326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d9fee0c20 con 0x557d9ffde250 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffe7750 0x557d9ffe1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09c94d1640 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffe6ed0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f09c004f4b0 tx=0x7f09c0032b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d61cc19c9b8165f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09aeffd640 1 -- 172.21.15.42:0/2413852326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00c10c0 con 0x557d9ffde250 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09aeffd640 1 -- 172.21.15.42:0/2413852326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09c00f0040 con 0x557d9ffde250 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.204+0000 7f09aeffd640 1 -- 172.21.15.42:0/2413852326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00040d0 con 0x557d9ffde250 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2413852326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 msgr2=0x557d9ffe6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffe6ed0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f09c004f4b0 tx=0x7f09c0032b40 comp rx=0 tx=0).stop 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2413852326 shutdown_connections 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffe7750 0x557d9ffe1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffdeb70 0x557d9ffe5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2413852326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffe6ed0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2413852326 >> 172.21.15.42:0/2413852326 conn(0x557d9ffd8980 msgr2=0x557d9fffe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2413852326 shutdown_connections 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2413852326 wait complete. 2024-03-12T04:24:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 -- start start 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 Processor -- start 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffecc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.205+0000 7f09ce3e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffe7750 0x557d9ffed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ffed870 con 0x557d9ffde250 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ffed9e0 con 0x557d9ffdeb70 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09ce3e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da004e060 con 0x557d9ffe7750 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09ce3e0740 10 client.0 _ll_register_callbacks cb 0x7f09bc00b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55964/0 (socket says 172.21.15.42:55964) 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 -- 172.21.15.42:0/2687885923 learned_addr learned my addr 172.21.15.42:0/2687885923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c8cd0640 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c9cd2640 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffe7750 0x557d9ffed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 -- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffe7750 msgr2=0x557d9ffed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffe7750 0x557d9ffed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 -- 172.21.15.42:0/2687885923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 msgr2=0x557d9ffecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 -- 172.21.15.42:0/2687885923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c41a6910 con 0x557d9ffde250 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09c94d1640 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffee890 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f09c00b9940 tx=0x7f09c0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48e6bd2925baf56 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09acff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09acff9640 1 -- 172.21.15.42:0/2687885923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00bb330 con 0x557d9ffde250 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09acff9640 1 -- 172.21.15.42:0/2687885923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f09c00b95e0 con 0x557d9ffde250 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09bc019070 con 0x557d9ffde250 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09acff9640 1 -- 172.21.15.42:0/2687885923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c00b97a0 con 0x557d9ffde250 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09ce3e0740 10 client.18183 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.206+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557d9ff068c0 con 0x557d9ffde250 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09acff9640 1 -- 172.21.15.42:0/2687885923 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09c009f020 con 0x557d9ffde250 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09acff9640 1 -- 172.21.15.42:0/2687885923 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09c00bd2f0 con 0x557d9ffde250 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09ce3e0740 10 client.18183 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09ce3e0740 10 client.18183 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09ce3e0740 20 client.18183 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557da004f2f0 0x557da00516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09ce3e0740 4 client.18183 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d9fcd1890 con 0x557da004f2f0 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09c8cd0640 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557da004f2f0 0x557da00516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09acff9640 10 client.18183 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09c8cd0640 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557da004f2f0 0x557da00516e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9aa5b4c77f9e8f6 server_cookie=1303e044fae2781a in_seq=0 out_seq=0 2024-03-12T04:24:49.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09acff9640 1 -- 172.21.15.42:0/2687885923 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d9fcd1890 con 0x557da004f2f0 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.207+0000 7f09acff9640 10 client.18183 handle_command_reply: tid=0 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 client.18183 shutdown 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 msgr2=0x557d9ffee890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffee890 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f09c00b9940 tx=0x7f09c0004040 comp rx=0 tx=0).stop 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 shutdown_connections 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x557da004f2f0 0x557da00516e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d9ffe7750 0x557d9ffed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09acff9640 0 client.18183 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d9ffdeb70 0x557d9ffecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 --2- 172.21.15.42:0/2687885923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557d9ffde250 0x557d9ffee890 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 >> 172.21.15.42:0/2687885923 conn(0x557d9ffd8980 msgr2=0x557d9ffd72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 shutdown_connections 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 1 -- 172.21.15.42:0/2687885923 wait complete. 2024-03-12T04:24:49.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09ce3e0740 20 client.18183 trim_cache size 0 max 16384 2024-03-12T04:24:49.220 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:49.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f09b00a1ba0 msgr2=0x7f09b00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f09b00a1ba0 0x7f09b00a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09bc013530 tx=0x7f09bc013930 comp rx=0 tx=0).stop 2024-03-12T04:24:49.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 msgr2=0x7f09c419f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c419f690 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f09c002c700 tx=0x7f09c0035ca0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 shutdown_connections 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f09b00a1ba0 0x7f09b00a4060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c4199750 0x7f09c4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c41064c0 0x7f09c419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 --2- 172.21.15.42:0/334381173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09c40685f0 0x7f09c419f690 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.208+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 >> 172.21.15.42:0/334381173 conn(0x7f09c4077180 msgr2=0x7f09c410c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.209+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 shutdown_connections 2024-03-12T04:24:49.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.209+0000 7f09cb735640 1 -- 172.21.15.42:0/334381173 wait complete. 2024-03-12T04:24:49.223 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:49.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.325+0000 7f410a18a640 1 -- 172.21.15.42:0/3669767927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 msgr2=0x7f410410e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.325+0000 7f410a18a640 1 --2- 172.21.15.42:0/3669767927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f410410e030 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009cd0 tx=0x7f40fc02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.325+0000 7f410a18a640 1 -- 172.21.15.42:0/3669767927 shutdown_connections 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.325+0000 7f410a18a640 1 --2- 172.21.15.42:0/3669767927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f410410e030 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.325+0000 7f410a18a640 1 --2- 172.21.15.42:0/3669767927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4104101d40 0x7f4104105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.326+0000 7f410a18a640 1 --2- 172.21.15.42:0/3669767927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4104101390 0x7f4104101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.326+0000 7f410a18a640 1 -- 172.21.15.42:0/3669767927 >> 172.21.15.42:0/3669767927 conn(0x7f41040fcc10 msgr2=0x7f41040ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.326+0000 7f410a18a640 1 -- 172.21.15.42:0/3669767927 shutdown_connections 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.326+0000 7f410a18a640 1 -- 172.21.15.42:0/3669767927 wait complete. 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.326+0000 7f410a18a640 1 Processor -- start 2024-03-12T04:24:49.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.326+0000 7f410a18a640 1 -- start start 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.326+0000 7f410a18a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4104101390 0x7f4104197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f410a18a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f4104197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f410a18a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410419bfe0 0x7f410419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f410a18a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410419c9d0 con 0x7f4104106460 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f410a18a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410419cb40 con 0x7f410419bfe0 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f410a18a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410419ccb0 con 0x7f4104101390 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f4104197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f4104197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55984/0 (socket says 172.21.15.42:55984) 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 -- 172.21.15.42:0/3425275536 learned_addr learned my addr 172.21.15.42:0/3425275536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4103fff640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410419bfe0 0x7f410419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f41037fe640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4104101390 0x7f4104197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 -- 172.21.15.42:0/3425275536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4104101390 msgr2=0x7f4104197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4104101390 0x7f4104197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 -- 172.21.15.42:0/3425275536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410419bfe0 msgr2=0x7f410419c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410419bfe0 0x7f410419c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 -- 172.21.15.42:0/3425275536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc009900 con 0x7f4104106460 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f41037fe640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4104101390 0x7f4104197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4102ffd640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f4104197c60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f40f800d770 tx=0x7f40f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4100ff9640 1 -- 172.21.15.42:0/3425275536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8009c30 con 0x7f4104106460 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4100ff9640 1 -- 172.21.15.42:0/3425275536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40f800de80 con 0x7f4104106460 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41041a4290 con 0x7f4104106460 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f4100ff9640 1 -- 172.21.15.42:0/3425275536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f800f600 con 0x7f4104106460 2024-03-12T04:24:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.327+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41041a4740 con 0x7f4104106460 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.330+0000 7f4100ff9640 1 -- 172.21.15.42:0/3425275536 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f40f8010460 con 0x7f4104106460 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.330+0000 7f410ce35740 1 Processor -- start 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.330+0000 7f410ce35740 1 -- start start 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.330+0000 7f410ce35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3a250 0x55ee7fb3a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.330+0000 7f410ce35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f410ce35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb45b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f410ce35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7faf5960 con 0x55ee7fb3a250 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f410ce35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7faa9310 con 0x55ee7fb3ab70 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f410ce35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7faa6840 con 0x55ee7fb43750 2024-03-12T04:24:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3a250 0x55ee7fb3a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3a250 0x55ee7fb3a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55998/0 (socket says 172.21.15.42:55998) 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 -- 172.21.15.42:0/229274678 learned_addr learned my addr 172.21.15.42:0/229274678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4100ff9640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40ec0a1ba0 0x7f40ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f41037fe640 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4102ffd640 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb45b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 -- 172.21.15.42:0/229274678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 msgr2=0x55ee7fb45b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb45b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 -- 172.21.15.42:0/229274678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 msgr2=0x55ee7fb43210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb43210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 -- 172.21.15.42:0/229274678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee7fa64060 con 0x55ee7fb3a250 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4102ffd640 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb45b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40ec0a1ba0 0x7f40ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4100ff9640 1 -- 172.21.15.42:0/3425275536 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f40f80cd540 con 0x7f4104106460 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f4103fff640 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3a250 0x55ee7fb3a630 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02c700 tx=0x7f40fc032490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa787236097b0aa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f40e9ffb640 1 -- 172.21.15.42:0/229274678 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0325d0 con 0x55ee7fb3a250 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f40e9ffb640 1 -- 172.21.15.42:0/229274678 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40fc003970 con 0x55ee7fb3a250 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.331+0000 7f40e9ffb640 1 -- 172.21.15.42:0/229274678 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0044a0 con 0x55ee7fb3a250 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f4103fff640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40ec0a1ba0 0x7f40ec0a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40fc04d000 tx=0x7f40fc04a000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 -- 172.21.15.42:0/229274678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3a250 msgr2=0x55ee7fb3a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3a250 0x55ee7fb3a630 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02c700 tx=0x7f40fc032490 comp rx=0 tx=0).stop 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 -- 172.21.15.42:0/229274678 shutdown_connections 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb45b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb43210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 --2- 172.21.15.42:0/229274678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3a250 0x55ee7fb3a630 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 -- 172.21.15.42:0/229274678 >> 172.21.15.42:0/229274678 conn(0x55ee7fb34980 msgr2=0x55ee7fb34d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 -- 172.21.15.42:0/229274678 shutdown_connections 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 -- 172.21.15.42:0/229274678 wait complete. 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.332+0000 7f410ce35740 1 -- start start 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 1 Processor -- start 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3a250 0x55ee7fb40770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb40cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7faa2020 con 0x55ee7fb43750 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7f775430 con 0x55ee7fb3ab70 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7f731e00 con 0x55ee7fb3a250 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f410ce35740 10 client.0 _ll_register_callbacks cb 0x55ee7fa49cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56012/0 (socket says 172.21.15.42:56012) 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 -- 172.21.15.42:0/1310010620 learned_addr learned my addr 172.21.15.42:0/1310010620 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f41037fe640 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3a250 0x55ee7fb40770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4102ffd640 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb40cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 -- 172.21.15.42:0/1310010620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3a250 msgr2=0x55ee7fb40770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3a250 0x55ee7fb40770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 -- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 msgr2=0x55ee7fb40cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb40cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.333+0000 7f4103fff640 1 -- 172.21.15.42:0/1310010620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410410e840 con 0x55ee7fb43750 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f4103fff640 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb411f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02c630 tx=0x7f40fc0041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede5b44ecfae51ff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f40e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f40e2ffd640 1 -- 172.21.15.42:0/1310010620 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc003cf0 con 0x55ee7fb43750 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f40e2ffd640 1 -- 172.21.15.42:0/1310010620 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f40fc0325d0 con 0x55ee7fb43750 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee7f7a17e0 con 0x55ee7fb43750 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f40e2ffd640 1 -- 172.21.15.42:0/1310010620 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40fc003e50 con 0x55ee7fb43750 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 10 client.18195 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee7faa06a0 con 0x55ee7fb43750 2024-03-12T04:24:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f40e2ffd640 1 -- 172.21.15.42:0/1310010620 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ee7f7a17e0 con 0x55ee7fb43750 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f40e2ffd640 1 -- 172.21.15.42:0/1310010620 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40fc04ac70 con 0x55ee7fb43750 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 10 client.18195 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 10 client.18195 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 20 client.18195 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fb53890 0x55ee7fb55c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 4 client.18195 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.334+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee7f7de720 con 0x55ee7fb53890 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.335+0000 7f41037fe640 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fb53890 0x55ee7fb55c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.335+0000 7f41037fe640 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fb53890 0x55ee7fb55c80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4604b2cda4dc2652 server_cookie=1138c43f86d58c2c in_seq=0 out_seq=0 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.335+0000 7f40e2ffd640 10 client.18195 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f40e2ffd640 1 -- 172.21.15.42:0/1310010620 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee7f7de720 con 0x55ee7fb53890 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f40e2ffd640 10 client.18195 handle_command_reply: tid=0 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 client.18195 shutdown 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 msgr2=0x55ee7fb411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb411f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02c630 tx=0x7f40fc0041c0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 shutdown_connections 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fb53890 0x55ee7fb55c80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb411f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb40cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f40e2ffd640 0 client.18195 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 --2- 172.21.15.42:0/1310010620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3a250 0x55ee7fb40770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 >> 172.21.15.42:0/1310010620 conn(0x55ee7fb34980 msgr2=0x55ee7fb366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 shutdown_connections 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 1 -- 172.21.15.42:0/1310010620 wait complete. 2024-03-12T04:24:49.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.336+0000 7f410ce35740 20 client.18195 trim_cache size 0 max 16384 2024-03-12T04:24:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 Processor -- start 2024-03-12T04:24:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 -- start start 2024-03-12T04:24:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb42ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb3d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7fa27130 con 0x55ee7fb43750 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7fb30080 con 0x55ee7fb3a250 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f410ce35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7fb45b40 con 0x55ee7fb3ab70 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb3d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb3d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56020/0 (socket says 172.21.15.42:56020) 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 -- 172.21.15.42:0/2775314882 learned_addr learned my addr 172.21.15.42:0/2775314882 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f41037fe640 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb42ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4102ffd640 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 -- 172.21.15.42:0/2775314882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3ab70 msgr2=0x55ee7fb41010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb41010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 -- 172.21.15.42:0/2775314882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 msgr2=0x55ee7fb42ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb42ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.348+0000 7f4103fff640 1 -- 172.21.15.42:0/2775314882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee7fa3cc20 con 0x55ee7fb43750 2024-03-12T04:24:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f4103fff640 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb3d9d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f40fc037040 tx=0x7f40fc005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15669007ab96555 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f40e97fa640 1 -- 172.21.15.42:0/2775314882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc005bd0 con 0x55ee7fb43750 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f40e97fa640 1 -- 172.21.15.42:0/2775314882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40fc0346a0 con 0x55ee7fb43750 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f40e97fa640 1 -- 172.21.15.42:0/2775314882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc032830 con 0x55ee7fb43750 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 -- 172.21.15.42:0/2775314882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 msgr2=0x55ee7fb3d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb3d9d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f40fc037040 tx=0x7f40fc005f80 comp rx=0 tx=0).stop 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 -- 172.21.15.42:0/2775314882 shutdown_connections 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb43750 0x55ee7fb3d9d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 --2- 172.21.15.42:0/2775314882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb42ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 -- 172.21.15.42:0/2775314882 >> 172.21.15.42:0/2775314882 conn(0x55ee7fb34980 msgr2=0x55ee7fb3ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 -- 172.21.15.42:0/2775314882 shutdown_connections 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.349+0000 7f410ce35740 1 -- 172.21.15.42:0/2775314882 wait complete. 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 -- start start 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 Processor -- start 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb49790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7fba9e30 con 0x55ee7fb3ab70 2024-03-12T04:24:49.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7fba9fa0 con 0x55ee7fb3a250 2024-03-12T04:24:49.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7fbaa160 con 0x55ee7fb43750 2024-03-12T04:24:49.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f410ce35740 10 client.0 _ll_register_callbacks cb 0x7f40fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f4102ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb49790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f4102ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb49790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56026/0 (socket says 172.21.15.42:56026) 2024-03-12T04:24:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f4102ffd640 1 -- 172.21.15.42:0/2207196826 learned_addr learned my addr 172.21.15.42:0/2207196826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f41037fe640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.350+0000 7f4103fff640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f4102ffd640 1 -- 172.21.15.42:0/2207196826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 msgr2=0x55ee7fb49cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f4102ffd640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb49cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f4102ffd640 1 -- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 msgr2=0x55ee7fb49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f4102ffd640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb49250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f4102ffd640 1 -- 172.21.15.42:0/2207196826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee7faa06a0 con 0x55ee7fb3ab70 2024-03-12T04:24:49.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f4103fff640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb49cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f4102ffd640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb49790 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f40f8096b60 tx=0x7f40f809a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbb017a89d32e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f40e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f40e2ffd640 1 -- 172.21.15.42:0/2207196826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80a2930 con 0x55ee7fb3ab70 2024-03-12T04:24:49.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f40e2ffd640 1 -- 172.21.15.42:0/2207196826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f40f80caf30 con 0x55ee7fb3ab70 2024-03-12T04:24:49.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40fc003970 con 0x55ee7fb3ab70 2024-03-12T04:24:49.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f40e2ffd640 1 -- 172.21.15.42:0/2207196826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40f80cb0f0 con 0x55ee7fb3ab70 2024-03-12T04:24:49.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 10 client.18201 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee7fa628c0 con 0x55ee7fb3ab70 2024-03-12T04:24:49.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f40e2ffd640 1 -- 172.21.15.42:0/2207196826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40f80b6080 con 0x55ee7fb3ab70 2024-03-12T04:24:49.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f40e2ffd640 1 -- 172.21.15.42:0/2207196826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40f80cd0d0 con 0x55ee7fb3ab70 2024-03-12T04:24:49.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 10 client.18201 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 10 client.18201 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:24:49.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 20 client.18201 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fbab9a0 0x55ee7fbadd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 4 client.18201 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:49.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.351+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee7f82d890 con 0x55ee7fbab9a0 2024-03-12T04:24:49.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f41037fe640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fbab9a0 0x55ee7fbadd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f40e2ffd640 10 client.18201 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:49.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f41037fe640 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fbab9a0 0x55ee7fbadd90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b18a76032c889e8 server_cookie=93bca3d53c5fff4d in_seq=0 out_seq=0 2024-03-12T04:24:49.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f40e2ffd640 1 -- 172.21.15.42:0/2207196826 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee7f82d890 con 0x55ee7fbab9a0 2024-03-12T04:24:49.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f40e2ffd640 10 client.18201 handle_command_reply: tid=0 2024-03-12T04:24:49.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 client.18201 shutdown 2024-03-12T04:24:49.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3ab70 msgr2=0x55ee7fb49790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb49790 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f40f8096b60 tx=0x7f40f809a890 comp rx=0 tx=0).stop 2024-03-12T04:24:49.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 shutdown_connections 2024-03-12T04:24:49.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55ee7fbab9a0 0x55ee7fbadd90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee7fb43750 0x55ee7fb49cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee7fb3ab70 0x55ee7fb49790 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f40e2ffd640 0 client.18201 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:24:49.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 --2- 172.21.15.42:0/2207196826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee7fb3a250 0x55ee7fb49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 >> 172.21.15.42:0/2207196826 conn(0x55ee7fb34980 msgr2=0x55ee7fb35db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 shutdown_connections 2024-03-12T04:24:49.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 1 -- 172.21.15.42:0/2207196826 wait complete. 2024-03-12T04:24:49.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.352+0000 7f410ce35740 20 client.18201 trim_cache size 0 max 16384 2024-03-12T04:24:49.368 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:49.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40ec0a1ba0 msgr2=0x7f40ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40ec0a1ba0 0x7f40ec0a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40fc04d000 tx=0x7f40fc04a000 comp rx=0 tx=0).stop 2024-03-12T04:24:49.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 msgr2=0x7f4104197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f4104197c60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f40f800d770 tx=0x7f40f800dc30 comp rx=0 tx=0).stop 2024-03-12T04:24:49.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 shutdown_connections 2024-03-12T04:24:49.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f40ec0a1ba0 0x7f40ec0a4060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410419bfe0 0x7f410419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4104106460 0x7f4104197c60 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 --2- 172.21.15.42:0/3425275536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4104101390 0x7f4104197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 >> 172.21.15.42:0/3425275536 conn(0x7f41040fcc10 msgr2=0x7f41040fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 shutdown_connections 2024-03-12T04:24:49.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.353+0000 7f410a18a640 1 -- 172.21.15.42:0/3425275536 wait complete. 2024-03-12T04:24:49.372 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:49.373 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:24:49.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:49 smithi119 ceph-mon[36119]: pgmap v297: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:49.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:49 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2080172886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:24:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:49 smithi136 ceph-mon[35571]: pgmap v297: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:49.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:49 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2080172886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:24:49.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 -- 172.21.15.42:0/4104415828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068101380 msgr2=0x7f2068101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 --2- 172.21.15.42:0/4104415828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068101380 0x7f2068101760 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2058009930 tx=0x7f205802bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 -- 172.21.15.42:0/4104415828 shutdown_connections 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 --2- 172.21.15.42:0/4104415828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2068106450 0x7f206810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 --2- 172.21.15.42:0/4104415828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2068101d30 0x7f2068105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 --2- 172.21.15.42:0/4104415828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068101380 0x7f2068101760 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 -- 172.21.15.42:0/4104415828 >> 172.21.15.42:0/4104415828 conn(0x7f2068078580 msgr2=0x7f2068078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 -- 172.21.15.42:0/4104415828 shutdown_connections 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 -- 172.21.15.42:0/4104415828 wait complete. 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 Processor -- start 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.475+0000 7f206d3bc640 1 -- start start 2024-03-12T04:24:49.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f206d3bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2068101380 0x7f2068073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f206d3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068106450 0x7f20680739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f206d3bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2068073f00 0x7f20681a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f206d3bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20681a5ea0 con 0x7f2068106450 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f206d3bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20681a6010 con 0x7f2068073f00 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f206d3bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20681a61d0 con 0x7f2068101380 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068106450 0x7f20680739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068106450 0x7f20680739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56048/0 (socket says 172.21.15.42:56048) 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 -- 172.21.15.42:0/3922755853 learned_addr learned my addr 172.21.15.42:0/3922755853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f2066ffd640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2068101380 0x7f2068073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20677fe640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2068073f00 0x7f20681a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 -- 172.21.15.42:0/3922755853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2068101380 msgr2=0x7f2068073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2068101380 0x7f2068073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 -- 172.21.15.42:0/3922755853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2068073f00 msgr2=0x7f20681a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2068073f00 0x7f20681a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 -- 172.21.15.42:0/3922755853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058009560 con 0x7f2068106450 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20677fe640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2068073f00 0x7f20681a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f2066ffd640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2068101380 0x7f2068073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f20667fc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068106450 0x7f20680739c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d770 tx=0x7f205c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f204ffff640 1 -- 172.21.15.42:0/3922755853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c009c30 con 0x7f2068106450 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20681a64b0 con 0x7f2068106450 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f204ffff640 1 -- 172.21.15.42:0/3922755853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f205c00de80 con 0x7f2068106450 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.476+0000 7f204ffff640 1 -- 172.21.15.42:0/3922755853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c00f600 con 0x7f2068106450 2024-03-12T04:24:49.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.477+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20681a6a10 con 0x7f2068106450 2024-03-12T04:24:49.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.478+0000 7f204ffff640 1 -- 172.21.15.42:0/3922755853 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f205c010460 con 0x7f2068106450 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 Processor -- start 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 -- start start 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f204ffff640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f20500a1bd0 0x7f20500a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e43059e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 0x55e4305a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e430559960 con 0x55e43059eb70 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43050d310 con 0x55e43059e250 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f206ff8e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43050a840 con 0x55e4305a7750 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f20677fe640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f20500a1bd0 0x7f20500a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56052/0 (socket says 172.21.15.42:56052) 2024-03-12T04:24:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f20667fc640 1 -- 172.21.15.42:0/591554250 learned_addr learned my addr 172.21.15.42:0/591554250 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f204ffff640 1 -- 172.21.15.42:0/3922755853 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f205c0cd660 con 0x7f2068106450 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f2066ffd640 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e43059e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.479+0000 7f20677fe640 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 0x55e4305a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f20667fc640 1 -- 172.21.15.42:0/591554250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 msgr2=0x55e4305a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f20667fc640 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 0x55e4305a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f20667fc640 1 -- 172.21.15.42:0/591554250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 msgr2=0x55e43059e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f20667fc640 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e43059e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f20667fc640 1 -- 172.21.15.42:0/591554250 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4304c8060 con 0x55e43059eb70 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f2066ffd640 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e43059e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f20677fe640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f20500a1bd0 0x7f20500a4090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20600094d0 tx=0x7f2060013b20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f20667fc640 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a7210 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f205c0cadd0 tx=0x7f205c095e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb5ddf131268687 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f204d7fa640 1 -- 172.21.15.42:0/591554250 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0ac8c0 con 0x55e43059eb70 2024-03-12T04:24:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f204d7fa640 1 -- 172.21.15.42:0/591554250 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f205c09a950 con 0x55e43059eb70 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f204d7fa640 1 -- 172.21.15.42:0/591554250 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c098550 con 0x55e43059eb70 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f206ff8e740 1 -- 172.21.15.42:0/591554250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 msgr2=0x55e4305a7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.480+0000 7f206ff8e740 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a7210 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f205c0cadd0 tx=0x7f205c095e50 comp rx=0 tx=0).stop 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- 172.21.15.42:0/591554250 shutdown_connections 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 0x55e4305a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a7210 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 --2- 172.21.15.42:0/591554250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e43059e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- 172.21.15.42:0/591554250 >> 172.21.15.42:0/591554250 conn(0x55e430598980 msgr2=0x55e430598d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- 172.21.15.42:0/591554250 shutdown_connections 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- 172.21.15.42:0/591554250 wait complete. 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- start start 2024-03-12T04:24:49.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 Processor -- start 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059e250 0x55e4305a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305a5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e430506020 con 0x55e43059eb70 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4301d9430 con 0x55e4305a7750 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e430195e00 con 0x55e43059e250 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f206ff8e740 10 client.0 _ll_register_callbacks cb 0x55e4304adcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56058/0 (socket says 172.21.15.42:56058) 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f20667fc640 1 -- 172.21.15.42:0/2482234668 learned_addr learned my addr 172.21.15.42:0/2482234668 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f20677fe640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.481+0000 7f2066ffd640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059e250 0x55e4305a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f20667fc640 1 -- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059e250 msgr2=0x55e4305a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f20667fc640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059e250 0x55e4305a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f20667fc640 1 -- 172.21.15.42:0/2482234668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 msgr2=0x55e4305a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f20667fc640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f20667fc640 1 -- 172.21.15.42:0/2482234668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20681a6a10 con 0x55e43059eb70 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f20677fe640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f20667fc640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a4cf0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f205c0966b0 tx=0x7f205c0acd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8dc040ca5e4883 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f204aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f204aff5640 1 -- 172.21.15.42:0/2482234668 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c09e3a0 con 0x55e43059eb70 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f204aff5640 1 -- 172.21.15.42:0/2482234668 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f205c098550 con 0x55e43059eb70 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4302057e0 con 0x55e43059eb70 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f204aff5640 1 -- 172.21.15.42:0/2482234668 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f205c0968c0 con 0x55e43059eb70 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 10 client.18213 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e4305046a0 con 0x55e43059eb70 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f204aff5640 1 -- 172.21.15.42:0/2482234668 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e4302057e0 con 0x55e43059eb70 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f204aff5640 1 -- 172.21.15.42:0/2482234668 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f205c0a2560 con 0x55e43059eb70 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 10 client.18213 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 10 client.18213 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 20 client.18213 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4305b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 4 client.18213 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.482+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e430242720 con 0x55e4305b7730 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.483+0000 7f2066ffd640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4305b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.483+0000 7f204aff5640 10 client.18213 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.483+0000 7f2066ffd640 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4305b9b20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71e25c7bc0c3ece8 server_cookie=bd58b3151f4a9421 in_seq=0 out_seq=0 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f204aff5640 1 -- 172.21.15.42:0/2482234668 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e430242720 con 0x55e4305b7730 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f204aff5640 10 client.18213 handle_command_reply: tid=0 2024-03-12T04:24:49.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 client.18213 shutdown 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 msgr2=0x55e4305a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a4cf0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f205c0966b0 tx=0x7f205c0acd00 comp rx=0 tx=0).stop 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 shutdown_connections 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4305b9b20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f204aff5640 0 client.18213 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a4cf0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2482234668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059e250 0x55e4305a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 >> 172.21.15.42:0/2482234668 conn(0x55e430598980 msgr2=0x55e43059a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 shutdown_connections 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 1 -- 172.21.15.42:0/2482234668 wait complete. 2024-03-12T04:24:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.484+0000 7f206ff8e740 20 client.18213 trim_cache size 0 max 16384 2024-03-12T04:24:49.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 Processor -- start 2024-03-12T04:24:49.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 -- start start 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e4305a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 0x55e4305a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43048b130 con 0x55e43059eb70 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4305984b0 con 0x55e43059e250 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f206ff8e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43059d570 con 0x55e4305a7750 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56072/0 (socket says 172.21.15.42:56072) 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 -- 172.21.15.42:0/173455117 learned_addr learned my addr 172.21.15.42:0/173455117 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f2066ffd640 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e4305a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 -- 172.21.15.42:0/173455117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 msgr2=0x55e4305a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 0x55e4305a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 -- 172.21.15.42:0/173455117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 msgr2=0x55e4305a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e4305a6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 -- 172.21.15.42:0/173455117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4304a0c20 con 0x55e43059eb70 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.496+0000 7f20667fc640 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a5050 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f205c098500 tx=0x7f205c0a29f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bf1f6e94586251 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f204cff9640 1 -- 172.21.15.42:0/173455117 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0964f0 con 0x55e43059eb70 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f204cff9640 1 -- 172.21.15.42:0/173455117 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f205c09e3a0 con 0x55e43059eb70 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f204cff9640 1 -- 172.21.15.42:0/173455117 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0968c0 con 0x55e43059eb70 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 -- 172.21.15.42:0/173455117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 msgr2=0x55e4305a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a5050 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f205c098500 tx=0x7f205c0a29f0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 -- 172.21.15.42:0/173455117 shutdown_connections 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4305a7750 0x55e4305a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059eb70 0x55e4305a5050 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 --2- 172.21.15.42:0/173455117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e43059e250 0x55e4305a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 -- 172.21.15.42:0/173455117 >> 172.21.15.42:0/173455117 conn(0x55e430598980 msgr2=0x55e4305be480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 -- 172.21.15.42:0/173455117 shutdown_connections 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 -- 172.21.15.42:0/173455117 wait complete. 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.497+0000 7f206ff8e740 1 -- start start 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 1 Processor -- start 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059e250 0x55e4305ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059eb70 0x55e4305aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4305ad500 con 0x55e43059e250 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4305ad670 con 0x55e4305a7750 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4305ad830 con 0x55e43059eb70 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f206ff8e740 10 client.0 _ll_register_callbacks cb 0x7f205c010c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059e250 0x55e4305ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059e250 0x55e4305ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56084/0 (socket says 172.21.15.42:56084) 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 -- 172.21.15.42:0/2267638673 learned_addr learned my addr 172.21.15.42:0/2267638673 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f20667fc640 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059eb70 0x55e4305aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f20677fe640 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 -- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059eb70 msgr2=0x55e4305aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059eb70 0x55e4305aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 -- 172.21.15.42:0/2267638673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 msgr2=0x55e4305acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 -- 172.21.15.42:0/2267638673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4305046a0 con 0x55e43059e250 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f2066ffd640 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059e250 0x55e4305ae6e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f205802c320 tx=0x7f2058004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d582994d0c3622c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.498+0000 7f204aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f204aff5640 1 -- 172.21.15.42:0/2267638673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058004340 con 0x55e43059e250 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f204aff5640 1 -- 172.21.15.42:0/2267638673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2058043040 con 0x55e43059e250 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f205c09a950 con 0x55e43059e250 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f204aff5640 1 -- 172.21.15.42:0/2267638673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20580044a0 con 0x55e43059e250 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 10 client.18219 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e4304c68c0 con 0x55e43059e250 2024-03-12T04:24:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f204aff5640 1 -- 172.21.15.42:0/2267638673 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f205c09a950 con 0x55e43059e250 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f204aff5640 1 -- 172.21.15.42:0/2267638673 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f205804f020 con 0x55e43059e250 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 10 client.18219 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 10 client.18219 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 20 client.18219 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4306111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 4 client.18219 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e430291890 con 0x55e4305b7730 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.499+0000 7f20667fc640 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4306111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f204aff5640 10 client.18219 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f20667fc640 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4306111a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5c7a042b2b96405 server_cookie=3ed1ad70745d77c1 in_seq=0 out_seq=0 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f204aff5640 1 -- 172.21.15.42:0/2267638673 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e430291890 con 0x55e4305b7730 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f204aff5640 10 client.18219 handle_command_reply: tid=0 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 client.18219 shutdown 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059e250 msgr2=0x55e4305ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059e250 0x55e4305ae6e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f205802c320 tx=0x7f2058004b70 comp rx=0 tx=0).stop 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 shutdown_connections 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55e4305b7730 0x55e4306111a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4305a7750 0x55e4305acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43059eb70 0x55e4305aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f204aff5640 0 client.18219 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 --2- 172.21.15.42:0/2267638673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43059e250 0x55e4305ae6e0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 >> 172.21.15.42:0/2267638673 conn(0x55e430598980 msgr2=0x55e4305972f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 shutdown_connections 2024-03-12T04:24:49.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 1 -- 172.21.15.42:0/2267638673 wait complete. 2024-03-12T04:24:49.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.500+0000 7f206ff8e740 20 client.18219 trim_cache size 0 max 16384 2024-03-12T04:24:49.511 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f20500a1bd0 msgr2=0x7f20500a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f20500a1bd0 0x7f20500a4090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20600094d0 tx=0x7f2060013b20 comp rx=0 tx=0).stop 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068106450 msgr2=0x7f20680739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068106450 0x7f20680739c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d770 tx=0x7f205c00dc30 comp rx=0 tx=0).stop 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 shutdown_connections 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f20500a1bd0 0x7f20500a4090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2068073f00 0x7f20681a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2068106450 0x7f20680739c0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 --2- 172.21.15.42:0/3922755853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2068101380 0x7f2068073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 >> 172.21.15.42:0/3922755853 conn(0x7f2068078580 msgr2=0x7f20680ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 shutdown_connections 2024-03-12T04:24:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.501+0000 7f206d3bc640 1 -- 172.21.15.42:0/3922755853 wait complete. 2024-03-12T04:24:49.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:49 smithi042 ceph-mon[32260]: pgmap v297: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:24:49.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:49 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2080172886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:24:49.514 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:24:49.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:24:49.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 -- 172.21.15.42:0/218756940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 msgr2=0x7fb00810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 --2- 172.21.15.42:0/218756940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb00810b1f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7faffc009900 tx=0x7faffc02bcd0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 -- 172.21.15.42:0/218756940 shutdown_connections 2024-03-12T04:24:49.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 --2- 172.21.15.42:0/218756940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00810b7c0 0x7fb00810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 --2- 172.21.15.42:0/218756940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb00810b1f0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 --2- 172.21.15.42:0/218756940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0081023f0 0x7fb0081027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 -- 172.21.15.42:0/218756940 >> 172.21.15.42:0/218756940 conn(0x7fb0080771e0 msgr2=0x7fb0080775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 -- 172.21.15.42:0/218756940 shutdown_connections 2024-03-12T04:24:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 -- 172.21.15.42:0/218756940 wait complete. 2024-03-12T04:24:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.616+0000 7fb00d727640 1 Processor -- start 2024-03-12T04:24:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb00d727640 1 -- start start 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb00d727640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0081023f0 0x7fb008197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb00d727640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb008197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb00d727640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00810b7c0 0x7fb00819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb00d727640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00819c620 con 0x7fb008102d10 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb00d727640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00819c790 con 0x7fb0081023f0 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb00d727640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00819c950 con 0x7fb00810b7c0 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb008197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb008197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56098/0 (socket says 172.21.15.42:56098) 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 -- 172.21.15.42:0/758218372 learned_addr learned my addr 172.21.15.42:0/758218372 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0077fe640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00810b7c0 0x7fb00819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb006ffd640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0081023f0 0x7fb008197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 -- 172.21.15.42:0/758218372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00810b7c0 msgr2=0x7fb00819c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00810b7c0 0x7fb00819c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 -- 172.21.15.42:0/758218372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0081023f0 msgr2=0x7fb008197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0081023f0 0x7fb008197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0067fc640 1 -- 172.21.15.42:0/758218372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc009560 con 0x7fb008102d10 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb006ffd640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0081023f0 0x7fb008197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.617+0000 7fb0077fe640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00810b7c0 0x7fb00819c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.618+0000 7fb0067fc640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb008197ca0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7faffc006010 tx=0x7faffc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.618+0000 7fafeffff640 1 -- 172.21.15.42:0/758218372 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0044c0 con 0x7fb008102d10 2024-03-12T04:24:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.618+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00819cbd0 con 0x7fb008102d10 2024-03-12T04:24:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.618+0000 7fafeffff640 1 -- 172.21.15.42:0/758218372 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faffc003aa0 con 0x7fb008102d10 2024-03-12T04:24:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.618+0000 7fafeffff640 1 -- 172.21.15.42:0/758218372 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc005540 con 0x7fb008102d10 2024-03-12T04:24:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.618+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0081a44b0 con 0x7fb008102d10 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.620+0000 7fafeffff640 1 -- 172.21.15.42:0/758218372 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7faffc003c10 con 0x7fb008102d10 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.620+0000 7fb0103cf740 1 Processor -- start 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.620+0000 7fb0103cf740 1 -- start start 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fafeffff640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faff00a1ba0 0x7faff00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f1f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0103cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f1a8960 con 0x55e70f1edb70 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0103cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f15c310 con 0x55e70f1f6750 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0103cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f159840 con 0x55e70f1ed250 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0077fe640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faff00a1ba0 0x7faff00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56100/0 (socket says 172.21.15.42:56100) 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 -- 172.21.15.42:0/1978408287 learned_addr learned my addr 172.21.15.42:0/1978408287 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fafeffff640 1 -- 172.21.15.42:0/758218372 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faffc0f0120 con 0x7fb008102d10 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0077fe640 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f1f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb006ffd640 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 -- 172.21.15.42:0/1978408287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 msgr2=0x55e70f1ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 -- 172.21.15.42:0/1978408287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 msgr2=0x55e70f1f8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f1f8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 -- 172.21.15.42:0/1978408287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e70f117060 con 0x55e70f1edb70 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0077fe640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faff00a1ba0 0x7faff00a4060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb000005fd0 tx=0x7fb000014af0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fb0067fc640 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f6210 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7faffc005940 tx=0x7faffc0bcc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6611c8d49aea7bde server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fafed7fa640 1 -- 172.21.15.42:0/1978408287 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0056a0 con 0x55e70f1edb70 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fafed7fa640 1 -- 172.21.15.42:0/1978408287 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faffc0bd580 con 0x55e70f1edb70 2024-03-12T04:24:49.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.621+0000 7fafed7fa640 1 -- 172.21.15.42:0/1978408287 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0bb180 con 0x55e70f1edb70 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 -- 172.21.15.42:0/1978408287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 msgr2=0x55e70f1f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f6210 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7faffc005940 tx=0x7faffc0bcc50 comp rx=0 tx=0).stop 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 -- 172.21.15.42:0/1978408287 shutdown_connections 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f1f8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f6210 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 --2- 172.21.15.42:0/1978408287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 -- 172.21.15.42:0/1978408287 >> 172.21.15.42:0/1978408287 conn(0x55e70f1e7980 msgr2=0x55e70f1e7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 -- 172.21.15.42:0/1978408287 shutdown_connections 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 -- 172.21.15.42:0/1978408287 wait complete. 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 -- start start 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.622+0000 7fb0103cf740 1 Processor -- start 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1edb70 0x55e70f1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1f6750 0x55e70f1f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0103cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f155020 con 0x55e70f1f6750 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0103cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70ee28430 con 0x55e70f1edb70 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0103cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70ede4e00 con 0x55e70f1ed250 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0103cf740 10 client.0 _ll_register_callbacks cb 0x55e70f0fccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1f6750 0x55e70f1f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1f6750 0x55e70f1f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56110/0 (socket says 172.21.15.42:56110) 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 -- 172.21.15.42:0/452388147 learned_addr learned my addr 172.21.15.42:0/452388147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb006ffd640 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0067fc640 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1edb70 0x55e70f1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 -- 172.21.15.42:0/452388147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 msgr2=0x55e70f1f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 -- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1edb70 msgr2=0x55e70f1f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1edb70 0x55e70f1f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 -- 172.21.15.42:0/452388147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0081a44b0 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fb0077fe640 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1f6750 0x55e70f1f4220 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb0000142d0 tx=0x7fb000004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b049a370a41130a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fafeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.623+0000 7fafeaff5640 1 -- 172.21.15.42:0/452388147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000013990 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fafeaff5640 1 -- 172.21.15.42:0/452388147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb000013af0 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e70ee547e0 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fafeaff5640 1 -- 172.21.15.42:0/452388147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb000013cb0 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 10 client.18231 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e70f1536a0 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fafeaff5640 1 -- 172.21.15.42:0/452388147 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e70ee547e0 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fafeaff5640 1 -- 172.21.15.42:0/452388147 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0000203e0 con 0x55e70f1f6750 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 10 client.18231 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 10 client.18231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 20 client.18231 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f206a20 0x55e70f208e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 4 client.18231 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e70ee91720 con 0x55e70f206a20 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.624+0000 7fb006ffd640 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f206a20 0x55e70f208e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.625+0000 7fb006ffd640 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f206a20 0x55e70f208e40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e627efaa77334501 server_cookie=936dd8d653b8d446 in_seq=0 out_seq=0 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.625+0000 7fafeaff5640 10 client.18231 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.625+0000 7fafeaff5640 1 -- 172.21.15.42:0/452388147 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e70ee91720 con 0x55e70f206a20 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.625+0000 7fafeaff5640 10 client.18231 handle_command_reply: tid=0 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 client.18231 shutdown 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1f6750 msgr2=0x55e70f1f4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1f6750 0x55e70f1f4220 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb0000142d0 tx=0x7fb000004ab0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 shutdown_connections 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f206a20 0x55e70f208e40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1f6750 0x55e70f1f4220 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fafeaff5640 0 client.18231 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1edb70 0x55e70f1f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 --2- 172.21.15.42:0/452388147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1ed250 0x55e70f1f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 >> 172.21.15.42:0/452388147 conn(0x55e70f1e7980 msgr2=0x55e70f1e96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 shutdown_connections 2024-03-12T04:24:49.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 1 -- 172.21.15.42:0/452388147 wait complete. 2024-03-12T04:24:49.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.626+0000 7fb0103cf740 20 client.18231 trim_cache size 0 max 16384 2024-03-12T04:24:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.637+0000 7fb0103cf740 1 Processor -- start 2024-03-12T04:24:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.637+0000 7fb0103cf740 1 -- start start 2024-03-12T04:24:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1ed250 0x55e70f1f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1f6750 0x55e70f1f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0103cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f0da130 con 0x55e70f1edb70 2024-03-12T04:24:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0103cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f1ecee0 con 0x55e70f1ed250 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0103cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f20e0d0 con 0x55e70f1f6750 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56124/0 (socket says 172.21.15.42:56124) 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 -- 172.21.15.42:0/3898850486 learned_addr learned my addr 172.21.15.42:0/3898850486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb006ffd640 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1ed250 0x55e70f1f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 -- 172.21.15.42:0/3898850486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1f6750 msgr2=0x55e70f1f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0077fe640 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1f6750 0x55e70f1f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1f6750 0x55e70f1f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 -- 172.21.15.42:0/3898850486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1ed250 msgr2=0x55e70f1f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1ed250 0x55e70f1f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 -- 172.21.15.42:0/3898850486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e70f0efc20 con 0x55e70f1edb70 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0077fe640 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1f6750 0x55e70f1f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb006ffd640 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1ed250 0x55e70f1f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fb0067fc640 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f4040 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7faffc0ede90 tx=0x7faffc0bd6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a01795a25b182e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fafecff9640 1 -- 172.21.15.42:0/3898850486 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0e0070 con 0x55e70f1edb70 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fafecff9640 1 -- 172.21.15.42:0/3898850486 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faffc0bce80 con 0x55e70f1edb70 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.638+0000 7fafecff9640 1 -- 172.21.15.42:0/3898850486 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0edb10 con 0x55e70f1edb70 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- 172.21.15.42:0/3898850486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 msgr2=0x55e70f1f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f4040 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7faffc0ede90 tx=0x7faffc0bd6f0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- 172.21.15.42:0/3898850486 shutdown_connections 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1f6750 0x55e70f1f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1edb70 0x55e70f1f4040 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 --2- 172.21.15.42:0/3898850486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1ed250 0x55e70f1f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- 172.21.15.42:0/3898850486 >> 172.21.15.42:0/3898850486 conn(0x55e70f1e7980 msgr2=0x55e70f1eea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- 172.21.15.42:0/3898850486 shutdown_connections 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- 172.21.15.42:0/3898850486 wait complete. 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- start start 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 Processor -- start 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1ed250 0x55e70f1fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1edb70 0x55e70f1fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f25cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f25d470 con 0x55e70f1ed250 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.639+0000 7fb0103cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f25d5b0 con 0x55e70f1f6750 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb0103cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70f25d6f0 con 0x55e70f1edb70 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb0103cf740 10 client.0 _ll_register_callbacks cb 0x7faffc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1ed250 0x55e70f1fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1ed250 0x55e70f1fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56138/0 (socket says 172.21.15.42:56138) 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 -- 172.21.15.42:0/2904757986 learned_addr learned my addr 172.21.15.42:0/2904757986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb0067fc640 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1edb70 0x55e70f1fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb0077fe640 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f25cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 -- 172.21.15.42:0/2904757986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1edb70 msgr2=0x55e70f1fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1edb70 0x55e70f1fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 -- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 msgr2=0x55e70f25cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f25cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 -- 172.21.15.42:0/2904757986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e70f117060 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb006ffd640 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1ed250 0x55e70f1fc4d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7faff8005940 tx=0x7faff801af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bbab0e84cb9d87 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fafeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fafeaff5640 1 -- 172.21.15.42:0/2904757986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8018040 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fafeaff5640 1 -- 172.21.15.42:0/2904757986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faff8005230 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faffc0bd580 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fafeaff5640 1 -- 172.21.15.42:0/2904757986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faff80053f0 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb0103cf740 10 client.18237 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.640+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e70f1158c0 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fafeaff5640 1 -- 172.21.15.42:0/2904757986 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faffc0bd580 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fafeaff5640 1 -- 172.21.15.42:0/2904757986 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faff80128d0 con 0x55e70f1ed250 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0103cf740 10 client.18237 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0103cf740 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0103cf740 20 client.18237 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0103cf740 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f25eec0 0x55e70f2612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0103cf740 4 client.18237 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e70eee0890 con 0x55e70f25eec0 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0067fc640 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f25eec0 0x55e70f2612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fb0067fc640 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f25eec0 0x55e70f2612e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c76b340d8403c7 server_cookie=f90b39883bb34c8d in_seq=0 out_seq=0 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.641+0000 7fafeaff5640 10 client.18237 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fafeaff5640 1 -- 172.21.15.42:0/2904757986 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e70eee0890 con 0x55e70f25eec0 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fafeaff5640 10 client.18237 handle_command_reply: tid=0 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 client.18237 shutdown 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1ed250 msgr2=0x55e70f1fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1ed250 0x55e70f1fc4d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7faff8005940 tx=0x7faff801af40 comp rx=0 tx=0).stop 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 shutdown_connections 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x55e70f25eec0 0x55e70f2612e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e70f1f6750 0x55e70f25cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fafeaff5640 0 client.18237 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e70f1edb70 0x55e70f1fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 --2- 172.21.15.42:0/2904757986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e70f1ed250 0x55e70f1fc4d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 >> 172.21.15.42:0/2904757986 conn(0x55e70f1e7980 msgr2=0x55e70f1e62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 shutdown_connections 2024-03-12T04:24:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 1 -- 172.21.15.42:0/2904757986 wait complete. 2024-03-12T04:24:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb0103cf740 20 client.18237 trim_cache size 0 max 16384 2024-03-12T04:24:49.652 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:24:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faff00a1ba0 msgr2=0x7faff00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faff00a1ba0 0x7faff00a4060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb000005fd0 tx=0x7fb000014af0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 msgr2=0x7fb008197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb008197ca0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7faffc006010 tx=0x7faffc0040c0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 shutdown_connections 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faff00a1ba0 0x7faff00a4060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00810b7c0 0x7fb00819c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb008102d10 0x7fb008197ca0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 --2- 172.21.15.42:0/758218372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0081023f0 0x7fb008197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.642+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 >> 172.21.15.42:0/758218372 conn(0x7fb0080771e0 msgr2=0x7fb0080fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.643+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 shutdown_connections 2024-03-12T04:24:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:49.643+0000 7fb00d727640 1 -- 172.21.15.42:0/758218372 wait complete. 2024-03-12T04:24:50.656 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T04:24:50.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:24:50.656 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T04:24:50.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.773+0000 7fe569d2c640 1 -- 172.21.15.42:0/2596372581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 msgr2=0x7fe56410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.773+0000 7fe569d2c640 1 --2- 172.21.15.42:0/2596372581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56410cf00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009930 tx=0x7fe55c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 -- 172.21.15.42:0/2596372581 shutdown_connections 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 --2- 172.21.15.42:0/2596372581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56410cf00 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 --2- 172.21.15.42:0/2596372581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe564100c10 0x7fe564104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 --2- 172.21.15.42:0/2596372581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe564100260 0x7fe564100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 -- 172.21.15.42:0/2596372581 >> 172.21.15.42:0/2596372581 conn(0x7fe564077180 msgr2=0x7fe564077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 -- 172.21.15.42:0/2596372581 shutdown_connections 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 -- 172.21.15.42:0/2596372581 wait complete. 2024-03-12T04:24:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 Processor -- start 2024-03-12T04:24:50.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.774+0000 7fe569d2c640 1 -- start start 2024-03-12T04:24:50.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe569d2c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe564100260 0x7fe56419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe569d2c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe569d2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5641996b0 0x7fe564199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe569d2c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56419a180 con 0x7fe564105330 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe569d2c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56419a2f0 con 0x7fe5641996b0 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe569d2c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56419a4b0 con 0x7fe564100260 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56419fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56158/0 (socket says 172.21.15.42:56158) 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 -- 172.21.15.42:0/725933321 learned_addr learned my addr 172.21.15.42:0/725933321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe5637fe640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe564100260 0x7fe56419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 -- 172.21.15.42:0/725933321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe564100260 msgr2=0x7fe56419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe564100260 0x7fe56419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe563fff640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5641996b0 0x7fe564199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 -- 172.21.15.42:0/725933321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5641996b0 msgr2=0x7fe564199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5641996b0 0x7fe564199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 -- 172.21.15.42:0/725933321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c009560 con 0x7fe564105330 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe5637fe640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe564100260 0x7fe56419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe562ffd640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56419fb30 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00c970 tx=0x7fe54c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe560ff9640 1 -- 172.21.15.42:0/725933321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004440 con 0x7fe564105330 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5641a6450 con 0x7fe564105330 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe560ff9640 1 -- 172.21.15.42:0/725933321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe54c007d10 con 0x7fe564105330 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.775+0000 7fe560ff9640 1 -- 172.21.15.42:0/725933321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00f450 con 0x7fe564105330 2024-03-12T04:24:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.776+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5641a6930 con 0x7fe564105330 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe560ff9640 1 -- 172.21.15.42:0/725933321 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe54c003680 con 0x7fe564105330 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 Processor -- start 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 -- start start 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe560ff9640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe5500a1ba0 0x7fe5500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d12cf90 con 0x56000d1be380 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d12a4c0 con 0x56000d1c78b0 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe56c9de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d125ca0 con 0x56000d1bed30 2024-03-12T04:24:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe563fff640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe5500a1ba0 0x7fe5500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1be760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56160/0 (socket says 172.21.15.42:56160) 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe5637fe640 1 -- 172.21.15.42:0/2432209199 learned_addr learned my addr 172.21.15.42:0/2432209199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.778+0000 7fe560ff9640 1 -- 172.21.15.42:0/725933321 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe54c0cd9f0 con 0x7fe564105330 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe563fff640 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe562ffd640 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe5637fe640 1 -- 172.21.15.42:0/2432209199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 msgr2=0x56000d1c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe5637fe640 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe5637fe640 1 -- 172.21.15.42:0/2432209199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 msgr2=0x56000d1c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe5637fe640 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe5637fe640 1 -- 172.21.15.42:0/2432209199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000d0e80b0 con 0x56000d1be380 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe562ffd640 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe5637fe640 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1be760 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe5540103a0 tx=0x7fe554010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d387ee39b441c01 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe563fff640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe5500a1ba0 0x7fe5500a4060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005c40 tx=0x7fe55c02c5d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe559ffb640 1 -- 172.21.15.42:0/2432209199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554012710 con 0x56000d1be380 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe559ffb640 1 -- 172.21.15.42:0/2432209199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe554012d20 con 0x56000d1be380 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.779+0000 7fe559ffb640 1 -- 172.21.15.42:0/2432209199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554011cd0 con 0x56000d1be380 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 -- 172.21.15.42:0/2432209199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 msgr2=0x56000d1be760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1be760 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe5540103a0 tx=0x7fe554010860 comp rx=0 tx=0).stop 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 -- 172.21.15.42:0/2432209199 shutdown_connections 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2432209199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1be760 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 -- 172.21.15.42:0/2432209199 >> 172.21.15.42:0/2432209199 conn(0x56000d1b8840 msgr2=0x56000d1b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 -- 172.21.15.42:0/2432209199 shutdown_connections 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 -- 172.21.15.42:0/2432209199 wait complete. 2024-03-12T04:24:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 -- start start 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.780+0000 7fe56c9de740 1 Processor -- start 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1c4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe56c9de740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d0caf40 con 0x56000d1be380 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe56c9de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000cdf94b0 con 0x56000d1c78b0 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe56c9de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d0ab670 con 0x56000d1bed30 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe56c9de740 10 client.0 _ll_register_callbacks cb 0x56000d118f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1c4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56168/0 (socket says 172.21.15.42:56168) 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 -- 172.21.15.42:0/3797151321 learned_addr learned my addr 172.21.15.42:0/3797151321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe563fff640 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe562ffd640 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 -- 172.21.15.42:0/3797151321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 msgr2=0x56000d1c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 -- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 msgr2=0x56000d1c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 -- 172.21.15.42:0/3797151321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe554009c60 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe5637fe640 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1c4840 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe554006010 tx=0x7fe554011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ec75500c6336d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.781+0000 7fe542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe542ffd640 1 -- 172.21.15.42:0/3797151321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554021070 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe542ffd640 1 -- 172.21.15.42:0/3797151321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe554011e20 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56000d0fff40 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe542ffd640 1 -- 172.21.15.42:0/3797151321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe554004a40 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 10 client.18249 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56000d124320 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe542ffd640 1 -- 172.21.15.42:0/3797151321 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe554032020 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe542ffd640 1 -- 172.21.15.42:0/3797151321 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe554005040 con 0x56000d1be380 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 10 client.18249 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 20 client.18249 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d1d7c80 0x56000d1da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 4 client.18249 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56000d11c310 con 0x56000d1d7c80 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.782+0000 7fe562ffd640 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d1d7c80 0x56000d1da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.783+0000 7fe562ffd640 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d1d7c80 0x56000d1da070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6359e25fb9b374 server_cookie=85defadf4cd6f652 in_seq=0 out_seq=0 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.783+0000 7fe542ffd640 10 client.18249 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe542ffd640 1 -- 172.21.15.42:0/3797151321 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56000d11c310 con 0x56000d1d7c80 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe542ffd640 10 client.18249 handle_command_reply: tid=0 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 client.18249 shutdown 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 msgr2=0x56000d1c4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1c4840 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe554006010 tx=0x7fe554011750 comp rx=0 tx=0).stop 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 shutdown_connections 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d1d7c80 0x56000d1da070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1c78b0 0x56000d1c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1bed30 0x56000d1c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe542ffd640 0 client.18249 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3797151321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d1c4840 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 >> 172.21.15.42:0/3797151321 conn(0x56000d1b8840 msgr2=0x56000d1ba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 shutdown_connections 2024-03-12T04:24:50.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 1 -- 172.21.15.42:0/3797151321 wait complete. 2024-03-12T04:24:50.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.784+0000 7fe56c9de740 20 client.18249 trim_cache size 0 max 16384 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 Processor -- start 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 -- start start 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1be380 0x56000d1ba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1bed30 0x56000d1ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 0x56000d1cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d1d7a30 con 0x56000d1bed30 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d1b4110 con 0x56000d1be380 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe56c9de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d1c2db0 con 0x56000d1c78b0 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.799+0000 7fe562ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1bed30 0x56000d1ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1bed30 0x56000d1ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56178/0 (socket says 172.21.15.42:56178) 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 -- 172.21.15.42:0/2277803849 learned_addr learned my addr 172.21.15.42:0/2277803849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 -- 172.21.15.42:0/2277803849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 msgr2=0x56000d1cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 --2- 172.21.15.42:0/2277803849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 0x56000d1cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 -- 172.21.15.42:0/2277803849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1be380 msgr2=0x56000d1ba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 --2- 172.21.15.42:0/2277803849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1be380 0x56000d1ba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 -- 172.21.15.42:0/2277803849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000d0c0c00 con 0x56000d1bed30 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe562ffd640 1 --2- 172.21.15.42:0/2277803849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1bed30 0x56000d1ce8e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe54c004630 tx=0x7fe54c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134372d1154078d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe5597fa640 1 -- 172.21.15.42:0/2277803849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c06aea0 con 0x56000d1bed30 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe5597fa640 1 -- 172.21.15.42:0/2277803849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe54c09a640 con 0x56000d1bed30 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.800+0000 7fe5597fa640 1 -- 172.21.15.42:0/2277803849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0cb3e0 con 0x56000d1bed30 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 -- 172.21.15.42:0/2277803849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1bed30 msgr2=0x56000d1ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2277803849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1bed30 0x56000d1ce8e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe54c004630 tx=0x7fe54c000bc0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 -- 172.21.15.42:0/2277803849 shutdown_connections 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2277803849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 0x56000d1cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2277803849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1bed30 0x56000d1ce8e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 --2- 172.21.15.42:0/2277803849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1be380 0x56000d1ba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 -- 172.21.15.42:0/2277803849 >> 172.21.15.42:0/2277803849 conn(0x56000d1b8840 msgr2=0x56000d1b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 -- 172.21.15.42:0/2277803849 shutdown_connections 2024-03-12T04:24:50.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 -- 172.21.15.42:0/2277803849 wait complete. 2024-03-12T04:24:50.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 -- start start 2024-03-12T04:24:50.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 Processor -- start 2024-03-12T04:24:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.801+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d228710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1bed30 0x56000d228c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 0x56000d1cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d1ce2e0 con 0x56000d1be380 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d1ce450 con 0x56000d1bed30 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000d1ce610 con 0x56000d1c78b0 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 10 client.0 _ll_register_callbacks cb 0x7fe5540097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d228710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d228710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56182/0 (socket says 172.21.15.42:56182) 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 -- 172.21.15.42:0/3751547546 learned_addr learned my addr 172.21.15.42:0/3751547546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 -- 172.21.15.42:0/3751547546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 msgr2=0x56000d1cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 0x56000d1cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 -- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1bed30 msgr2=0x56000d228c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe562ffd640 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1bed30 0x56000d228c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1bed30 0x56000d228c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 -- 172.21.15.42:0/3751547546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5641a6930 con 0x56000d1be380 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe5637fe640 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d228710 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe554011940 tx=0x7fe554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f27c1f319bf7590 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe542ffd640 1 -- 172.21.15.42:0/3751547546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554021030 con 0x56000d1be380 2024-03-12T04:24:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe542ffd640 1 -- 172.21.15.42:0/3751547546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe554022070 con 0x56000d1be380 2024-03-12T04:24:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56000d14b900 con 0x56000d1be380 2024-03-12T04:24:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe542ffd640 1 -- 172.21.15.42:0/3751547546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5540098a0 con 0x56000d1be380 2024-03-12T04:24:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 10 client.18255 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.802+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56000d0e6910 con 0x56000d1be380 2024-03-12T04:24:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe542ffd640 1 -- 172.21.15.42:0/3751547546 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56000d14b900 con 0x56000d1be380 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe542ffd640 1 -- 172.21.15.42:0/3751547546 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe5540125a0 con 0x56000d1be380 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe56c9de740 10 client.18255 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe56c9de740 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe56c9de740 20 client.18255 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d230360 0x56000d232780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe56c9de740 4 client.18255 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56000ce25860 con 0x56000d230360 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe562ffd640 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d230360 0x56000d232780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe562ffd640 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d230360 0x56000d232780 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a1d0232248aea8 server_cookie=974045121bee93e4 in_seq=0 out_seq=0 2024-03-12T04:24:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.803+0000 7fe542ffd640 10 client.18255 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.804+0000 7fe542ffd640 1 -- 172.21.15.42:0/3751547546 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56000ce25860 con 0x56000d230360 2024-03-12T04:24:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.804+0000 7fe542ffd640 10 client.18255 handle_command_reply: tid=0 2024-03-12T04:24:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.804+0000 7fe56c9de740 1 client.18255 shutdown 2024-03-12T04:24:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 msgr2=0x56000d228710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d228710 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe554011940 tx=0x7fe554009bb0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 shutdown_connections 2024-03-12T04:24:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x56000d230360 0x56000d232780 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56000d1c78b0 0x56000d1cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe542ffd640 0 client.18255 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56000d1bed30 0x56000d228c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 --2- 172.21.15.42:0/3751547546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56000d1be380 0x56000d228710 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 >> 172.21.15.42:0/3751547546 conn(0x56000d1b8840 msgr2=0x56000d1b78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 shutdown_connections 2024-03-12T04:24:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 1 -- 172.21.15.42:0/3751547546 wait complete. 2024-03-12T04:24:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe56c9de740 20 client.18255 trim_cache size 0 max 16384 2024-03-12T04:24:50.819 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:24:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-03-12T04:24:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "739a63a6-3091-4701-81d0-540c06ab999c", 2024-03-12T04:24:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-03-12T04:24:50.819 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe5500a1ba0 msgr2=0x7fe5500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe5500a1ba0 0x7fe5500a4060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005c40 tx=0x7fe55c02c5d0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 msgr2=0x7fe56419fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56419fb30 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00c970 tx=0x7fe54c00ce30 comp rx=0 tx=0).stop 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 shutdown_connections 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe5500a1ba0 0x7fe5500a4060 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5641996b0 0x7fe564199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe564105330 0x7fe56419fb30 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 --2- 172.21.15.42:0/725933321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe564100260 0x7fe56419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.805+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 >> 172.21.15.42:0/725933321 conn(0x7fe564077180 msgr2=0x7fe56410c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.806+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 shutdown_connections 2024-03-12T04:24:50.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.806+0000 7fe569d2c640 1 -- 172.21.15.42:0/725933321 wait complete. 2024-03-12T04:24:50.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:24:50.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:24:50.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 -- 172.21.15.42:0/4230553551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730101d10 msgr2=0x7ff730105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 --2- 172.21.15.42:0/4230553551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730101d10 0x7ff730105d50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff724009930 tx=0x7ff72402bc30 comp rx=0 tx=0).stop 2024-03-12T04:24:50.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 -- 172.21.15.42:0/4230553551 shutdown_connections 2024-03-12T04:24:50.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 --2- 172.21.15.42:0/4230553551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff730106430 0x7ff73010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 --2- 172.21.15.42:0/4230553551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730101d10 0x7ff730105d50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 --2- 172.21.15.42:0/4230553551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff730101360 0x7ff730101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 -- 172.21.15.42:0/4230553551 >> 172.21.15.42:0/4230553551 conn(0x7ff7300fcc20 msgr2=0x7ff7300ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.924+0000 7ff737c15640 1 -- 172.21.15.42:0/4230553551 shutdown_connections 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 -- 172.21.15.42:0/4230553551 wait complete. 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 Processor -- start 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 -- start start 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff730101360 0x7ff730197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730106430 0x7ff730197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff73019bfe0 0x7ff73019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73019c9d0 con 0x7ff730106430 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73019cb40 con 0x7ff73019bfe0 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff737c15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73019ccb0 con 0x7ff730101360 2024-03-12T04:24:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff7351b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730106430 0x7ff730197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff7351b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730106430 0x7ff730197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56204/0 (socket says 172.21.15.42:56204) 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff7351b0640 1 -- 172.21.15.42:0/3602317243 learned_addr learned my addr 172.21.15.42:0/3602317243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.925+0000 7ff7359b1640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff730101360 0x7ff730197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7361b2640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff73019bfe0 0x7ff73019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7351b0640 1 -- 172.21.15.42:0/3602317243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff730101360 msgr2=0x7ff730197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7351b0640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff730101360 0x7ff730197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7351b0640 1 -- 172.21.15.42:0/3602317243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff73019bfe0 msgr2=0x7ff73019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7351b0640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff73019bfe0 0x7ff73019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7351b0640 1 -- 172.21.15.42:0/3602317243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724009560 con 0x7ff730106430 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7361b2640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff73019bfe0 0x7ff73019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff7351b0640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730106430 0x7ff730197c60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff724000c00 tx=0x7ff724004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff72effd640 1 -- 172.21.15.42:0/3602317243 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72402cb70 con 0x7ff730106430 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff72effd640 1 -- 172.21.15.42:0/3602317243 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff72402de90 con 0x7ff730106430 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7301a4290 con 0x7ff730106430 2024-03-12T04:24:50.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff72effd640 1 -- 172.21.15.42:0/3602317243 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724005380 con 0x7ff730106430 2024-03-12T04:24:50.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.926+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7301a4710 con 0x7ff730106430 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.928+0000 7ff72effd640 1 -- 172.21.15.42:0/3602317243 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7ff7240035f0 con 0x7ff730106430 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 Processor -- start 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 -- start start 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff72effd640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff7180a1ba0 0x7ff7180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7e3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af752280 con 0x5590af7e3bd0 2024-03-12T04:24:50.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af74f7b0 con 0x5590af7ec7b0 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af74af90 con 0x5590af7e32b0 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff72effd640 1 -- 172.21.15.42:0/3602317243 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff7240efa90 con 0x7ff730106430 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff7361b2640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff7180a1ba0 0x7ff7180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff7351b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff7351b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7ec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56210/0 (socket says 172.21.15.42:56210) 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff7351b0640 1 -- 172.21.15.42:0/2598099338 learned_addr learned my addr 172.21.15.42:0/2598099338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff7359b1640 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.929+0000 7ff7361b2640 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7351b0640 1 -- 172.21.15.42:0/2598099338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 msgr2=0x5590af7e3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7351b0640 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7e3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7351b0640 1 -- 172.21.15.42:0/2598099338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 msgr2=0x5590af7eeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7351b0640 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7eeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7351b0640 1 -- 172.21.15.42:0/2598099338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590af709570 con 0x5590af7e3bd0 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7359b1640 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7361b2640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff7180a1ba0 0x7ff7180a4060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff720005c40 tx=0x7ff720013a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff7351b0640 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7ec270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff724009900 tx=0x7ff724031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f54f27ca6a48156 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff717fff640 1 -- 172.21.15.42:0/2598099338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72403b2b0 con 0x5590af7e3bd0 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff717fff640 1 -- 172.21.15.42:0/2598099338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff72403b490 con 0x5590af7e3bd0 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.930+0000 7ff717fff640 1 -- 172.21.15.42:0/2598099338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240c0940 con 0x5590af7e3bd0 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 -- 172.21.15.42:0/2598099338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 msgr2=0x5590af7ec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7ec270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff724009900 tx=0x7ff724031f50 comp rx=0 tx=0).stop 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 -- 172.21.15.42:0/2598099338 shutdown_connections 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7eeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7ec270 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/2598099338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 -- 172.21.15.42:0/2598099338 >> 172.21.15.42:0/2598099338 conn(0x5590af7dd990 msgr2=0x5590af7ddd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 -- 172.21.15.42:0/2598099338 shutdown_connections 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 -- 172.21.15.42:0/2598099338 wait complete. 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 -- start start 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.931+0000 7ff73a7e1740 1 Processor -- start 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e32b0 0x5590af7ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e3bd0 0x5590af7e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7ec7b0 0x5590af7e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af41e430 con 0x5590af7e32b0 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af3dae00 con 0x5590af7e3bd0 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af6cffd0 con 0x5590af7ec7b0 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 10 client.0 _ll_register_callbacks cb 0x5590af725fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e32b0 0x5590af7ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e32b0 0x5590af7ec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56214/0 (socket says 172.21.15.42:56214) 2024-03-12T04:24:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 -- 172.21.15.42:0/1778415642 learned_addr learned my addr 172.21.15.42:0/1778415642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e3bd0 0x5590af7e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 -- 172.21.15.42:0/1778415642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7ec7b0 msgr2=0x5590af7e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7ec7b0 0x5590af7e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 -- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e3bd0 msgr2=0x5590af7e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e3bd0 0x5590af7e95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 -- 172.21.15.42:0/1778415642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7301a4710 con 0x5590af7e32b0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e32b0 0x5590af7ec240 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff72800baa0 tx=0x7ff72800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d46aabedfe364e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7157fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7157fa640 1 -- 172.21.15.42:0/1778415642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72801b640 con 0x5590af7e32b0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7157fa640 1 -- 172.21.15.42:0/1778415642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff728003e70 con 0x5590af7e32b0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590af44a7e0 con 0x5590af7e32b0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.932+0000 7ff7157fa640 1 -- 172.21.15.42:0/1778415642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff72801b7a0 con 0x5590af7e32b0 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 10 client.18267 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:50.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5590af749610 con 0x5590af7e32b0 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff7157fa640 1 -- 172.21.15.42:0/1778415642 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5590af44a7e0 con 0x5590af7e32b0 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff7157fa640 1 -- 172.21.15.42:0/1778415642 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff728011b30 con 0x5590af7e32b0 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 10 client.18267 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 20 client.18267 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af7fc820 0x5590af7fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 4 client.18267 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590af487720 con 0x5590af7fc820 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.933+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af7fc820 0x5590af7fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.934+0000 7ff7157fa640 10 client.18267 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.934+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af7fc820 0x5590af7fec40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd67ff9777dd37c4 server_cookie=888cbf277e82cf87 in_seq=0 out_seq=0 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.934+0000 7ff7157fa640 1 -- 172.21.15.42:0/1778415642 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590af487720 con 0x5590af7fc820 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.934+0000 7ff7157fa640 10 client.18267 handle_command_reply: tid=0 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.934+0000 7ff73a7e1740 1 client.18267 shutdown 2024-03-12T04:24:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e32b0 msgr2=0x5590af7ec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e32b0 0x5590af7ec240 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff72800baa0 tx=0x7ff72800bf60 comp rx=0 tx=0).stop 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 shutdown_connections 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af7fc820 0x5590af7fec40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7ec7b0 0x5590af7e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e3bd0 0x5590af7e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1778415642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e32b0 0x5590af7ec240 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff7157fa640 0 client.18267 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 >> 172.21.15.42:0/1778415642 conn(0x5590af7dd990 msgr2=0x5590af7df730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 shutdown_connections 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1778415642 wait complete. 2024-03-12T04:24:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.935+0000 7ff73a7e1740 20 client.18267 trim_cache size 0 max 16384 2024-03-12T04:24:50.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 Processor -- start 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 -- start start 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e32b0 0x5590af7e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e3bd0 0x5590af7ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7ec7b0 0x5590af7ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af7f8330 con 0x5590af7ec7b0 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af7e2f10 con 0x5590af7e32b0 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af7e51a0 con 0x5590af7e3bd0 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7ec7b0 0x5590af7ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7ec7b0 0x5590af7ec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56222/0 (socket says 172.21.15.42:56222) 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 -- 172.21.15.42:0/1499995282 learned_addr learned my addr 172.21.15.42:0/1499995282 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e3bd0 0x5590af7ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e32b0 0x5590af7e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 -- 172.21.15.42:0/1499995282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e3bd0 msgr2=0x5590af7ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e3bd0 0x5590af7ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 -- 172.21.15.42:0/1499995282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e32b0 msgr2=0x5590af7e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e32b0 0x5590af7e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.950+0000 7ff7361b2640 1 -- 172.21.15.42:0/1499995282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590af6e5ac0 con 0x5590af7ec7b0 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e3bd0 0x5590af7ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e32b0 0x5590af7e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff7361b2640 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7ec7b0 0x5590af7ec240 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff7200132e0 tx=0x7ff720019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaa698d5349af46 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff7177fe640 1 -- 172.21.15.42:0/1499995282 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720013310 con 0x5590af7ec7b0 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff7177fe640 1 -- 172.21.15.42:0/1499995282 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff720019070 con 0x5590af7ec7b0 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff7177fe640 1 -- 172.21.15.42:0/1499995282 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720005a10 con 0x5590af7ec7b0 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1499995282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7ec7b0 msgr2=0x5590af7ec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7ec7b0 0x5590af7ec240 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff7200132e0 tx=0x7ff720019000 comp rx=0 tx=0).stop 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1499995282 shutdown_connections 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7ec7b0 0x5590af7ec240 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e3bd0 0x5590af7ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1499995282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7e32b0 0x5590af7e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1499995282 >> 172.21.15.42:0/1499995282 conn(0x5590af7dd990 msgr2=0x5590af8035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1499995282 shutdown_connections 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.951+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1499995282 wait complete. 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 -- start start 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 Processor -- start 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af7f2060 con 0x5590af7e3bd0 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af7f21d0 con 0x5590af7ec7b0 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590af7f2390 con 0x5590af7e32b0 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff73a7e1740 10 client.0 _ll_register_callbacks cb 0x7ff724005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff7351b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff7351b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7f3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56234/0 (socket says 172.21.15.42:56234) 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff7351b0640 1 -- 172.21.15.42:0/1745920951 learned_addr learned my addr 172.21.15.42:0/1745920951 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.952+0000 7ff7361b2640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7351b0640 1 -- 172.21.15.42:0/1745920951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 msgr2=0x5590af7f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7351b0640 1 -- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 msgr2=0x5590af7f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7351b0640 1 -- 172.21.15.42:0/1745920951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff720013040 con 0x5590af7e3bd0 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7361b2640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7351b0640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7f3930 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff7240b91c0 tx=0x7ff72404dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20800fe0ed5073a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7157fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7157fa640 1 -- 172.21.15.42:0/1745920951 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240d4070 con 0x5590af7e3bd0 2024-03-12T04:24:50.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7157fa640 1 -- 172.21.15.42:0/1745920951 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff7240bcef0 con 0x5590af7e3bd0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff72403b490 con 0x5590af7e3bd0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7157fa640 1 -- 172.21.15.42:0/1745920951 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7240bc870 con 0x5590af7e3bd0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 10 client.18273 fetch_fsmap learned FSMap version 27 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5590af70b6a0 con 0x5590af7e3bd0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7157fa640 1 -- 172.21.15.42:0/1745920951 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff72403b490 con 0x5590af7e3bd0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff7157fa640 1 -- 172.21.15.42:0/1745920951 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff72403c070 con 0x5590af7e3bd0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 10 client.18273 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 20 client.18273 populate_metadata read hostname 'smithi042' 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af853da0 0x5590af8561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 4 client.18273 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.953+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590af4d6890 con 0x5590af853da0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af853da0 0x5590af8561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff7157fa640 10 client.18273 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff7359b1640 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af853da0 0x5590af8561c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9061b479bb9ab743 server_cookie=801ba0062ae1a2af in_seq=0 out_seq=0 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff7157fa640 1 -- 172.21.15.42:0/1745920951 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5590af4d6890 con 0x5590af853da0 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff7157fa640 10 client.18273 handle_command_reply: tid=0 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 client.18273 shutdown 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 msgr2=0x5590af7f3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7f3930 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff7240b91c0 tx=0x7ff72404dd40 comp rx=0 tx=0).stop 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 shutdown_connections 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x5590af853da0 0x5590af8561c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590af7ec7b0 0x5590af7f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590af7e3bd0 0x5590af7f3930 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff7157fa640 0 client.18273 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 --2- 172.21.15.42:0/1745920951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5590af7e32b0 0x5590af7f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 >> 172.21.15.42:0/1745920951 conn(0x5590af7dd990 msgr2=0x5590af7e8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 shutdown_connections 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 1 -- 172.21.15.42:0/1745920951 wait complete. 2024-03-12T04:24:50.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.954+0000 7ff73a7e1740 20 client.18273 trim_cache size 0 max 16384 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: "739a63a6-3091-4701-81d0-540c06ab999c": { 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "739a63a6-3091-4701-81d0-540c06ab999c", 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:24:50.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:24:50.966 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:24:50.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff7180a1ba0 msgr2=0x7ff7180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff7180a1ba0 0x7ff7180a4060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff720005c40 tx=0x7ff720013a40 comp rx=0 tx=0).stop 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730106430 msgr2=0x7ff730197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730106430 0x7ff730197c60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff724000c00 tx=0x7ff724004270 comp rx=0 tx=0).stop 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 shutdown_connections 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7ff7180a1ba0 0x7ff7180a4060 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff73019bfe0 0x7ff73019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff730106430 0x7ff730197c60 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 --2- 172.21.15.42:0/3602317243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff730101360 0x7ff730197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 >> 172.21.15.42:0/3602317243 conn(0x7ff7300fcc20 msgr2=0x7ff7300fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 shutdown_connections 2024-03-12T04:24:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:24:50.955+0000 7ff737c15640 1 -- 172.21.15.42:0/3602317243 wait complete. 2024-03-12T04:24:50.968 INFO:tasks.cephfs.filesystem:scrub status for tag:739a63a6-3091-4701-81d0-540c06ab999c - {'path': '/', 'tag': '739a63a6-3091-4701-81d0-540c06ab999c', 'options': 'recursive,force'} 2024-03-12T04:24:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:51 smithi042 ceph-mon[32260]: pgmap v298: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:51 smithi042 ceph-mon[32260]: scrub queued for path: / 2024-03-12T04:24:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:51 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:51 smithi042 ceph-mon[32260]: scrub summary: active paths [/] 2024-03-12T04:24:51.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:51 smithi042 ceph-mon[32260]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:51 smithi119 ceph-mon[36119]: pgmap v298: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:51 smithi119 ceph-mon[36119]: scrub queued for path: / 2024-03-12T04:24:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:51 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:51 smithi119 ceph-mon[36119]: scrub summary: active paths [/] 2024-03-12T04:24:51.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:51 smithi119 ceph-mon[36119]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:51 smithi136 ceph-mon[35571]: pgmap v298: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:51 smithi136 ceph-mon[35571]: scrub queued for path: / 2024-03-12T04:24:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:51 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:51 smithi136 ceph-mon[35571]: scrub summary: active paths [/] 2024-03-12T04:24:51.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:51 smithi136 ceph-mon[35571]: scrub summary: idle+waiting paths [/] 2024-03-12T04:24:53.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:53 smithi119 ceph-mon[36119]: pgmap v299: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:53 smithi136 ceph-mon[35571]: pgmap v299: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:53.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:53 smithi042 ceph-mon[32260]: pgmap v299: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:24:54.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:54 smithi119 ceph-mon[36119]: scrub summary: idle 2024-03-12T04:24:54.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:54 smithi136 ceph-mon[35571]: scrub summary: idle 2024-03-12T04:24:54.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:54 smithi042 ceph-mon[32260]: scrub summary: idle 2024-03-12T04:24:55.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:55 smithi119 ceph-mon[36119]: pgmap v300: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:55.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:55 smithi136 ceph-mon[35571]: pgmap v300: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:55.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:55 smithi042 ceph-mon[32260]: pgmap v300: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:56.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:56 smithi136 ceph-mon[35571]: pgmap v301: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:56.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:56 smithi042 ceph-mon[32260]: pgmap v301: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:56.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:56 smithi119 ceph-mon[36119]: pgmap v301: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:24:58.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:24:58 smithi119 ceph-mon[36119]: pgmap v302: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:58.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:24:58 smithi136 ceph-mon[35571]: pgmap v302: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:24:58.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:24:58 smithi042 ceph-mon[32260]: pgmap v302: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-12T04:25:00.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:00 smithi119 ceph-mon[36119]: pgmap v303: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:25:00.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:00 smithi136 ceph-mon[35571]: pgmap v303: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:25:00.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:00 smithi042 ceph-mon[32260]: pgmap v303: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:25:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:02 smithi119 ceph-mon[36119]: pgmap v304: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:25:02.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:02 smithi136 ceph-mon[35571]: pgmap v304: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:25:02.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:02 smithi042 ceph-mon[32260]: pgmap v304: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-12T04:25:04.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:04 smithi119 ceph-mon[36119]: pgmap v305: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:25:04.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:04 smithi136 ceph-mon[35571]: pgmap v305: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:25:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:04 smithi042 ceph-mon[32260]: pgmap v305: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:25:06.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:06 smithi119 ceph-mon[36119]: pgmap v306: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:25:06.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:06 smithi136 ceph-mon[35571]: pgmap v306: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:25:06.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:06 smithi042 ceph-mon[32260]: pgmap v306: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:25:08.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:08 smithi119 ceph-mon[36119]: pgmap v307: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:25:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:08 smithi136 ceph-mon[35571]: pgmap v307: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:25:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:08 smithi042 ceph-mon[32260]: pgmap v307: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:25:10.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:10 smithi119 ceph-mon[36119]: pgmap v308: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:25:10.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:10 smithi119 ceph-mon[36119]: osdmap e97: 12 total, 12 up, 12 in 2024-03-12T04:25:10.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:10 smithi136 ceph-mon[35571]: pgmap v308: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:25:10.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:10 smithi136 ceph-mon[35571]: osdmap e97: 12 total, 12 up, 12 in 2024-03-12T04:25:10.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:10 smithi042 ceph-mon[32260]: pgmap v308: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:25:10.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:10 smithi042 ceph-mon[32260]: osdmap e97: 12 total, 12 up, 12 in 2024-03-12T04:25:12.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:12 smithi119 ceph-mon[36119]: pgmap v310: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:25:12.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:12 smithi136 ceph-mon[35571]: pgmap v310: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:25:12.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:12 smithi042 ceph-mon[32260]: pgmap v310: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-12T04:25:13.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-12T04:25:13.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 -- 172.21.15.42:0/3919530587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081023f0 msgr2=0x7f07081027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 --2- 172.21.15.42:0/3919530587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081023f0 0x7f07081027d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009900 tx=0x7f06fc02bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:13.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 -- 172.21.15.42:0/3919530587 shutdown_connections 2024-03-12T04:25:13.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 --2- 172.21.15.42:0/3919530587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070810b900 0x7f070810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 --2- 172.21.15.42:0/3919530587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0708102da0 0x7f070810b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 --2- 172.21.15.42:0/3919530587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081023f0 0x7f07081027d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 -- 172.21.15.42:0/3919530587 >> 172.21.15.42:0/3919530587 conn(0x7f07080771a0 msgr2=0x7f07080775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 -- 172.21.15.42:0/3919530587 shutdown_connections 2024-03-12T04:25:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.255+0000 7f070e40f640 1 -- 172.21.15.42:0/3919530587 wait complete. 2024-03-12T04:25:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 Processor -- start 2024-03-12T04:25:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 -- start start 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07081023f0 0x7f070819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f070810b900 0x7f070819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081976a0 0x7f0708197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0708198170 con 0x7f07081976a0 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07081982e0 con 0x7f070810b900 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070e40f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07081984a0 con 0x7f07081023f0 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070c9ac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081976a0 0x7f0708197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070c9ac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081976a0 0x7f0708197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38192/0 (socket says 172.21.15.42:38192) 2024-03-12T04:25:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.256+0000 7f070c9ac640 1 -- 172.21.15.42:0/1992968711 learned_addr learned my addr 172.21.15.42:0/1992968711 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f0707fff640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07081023f0 0x7f070819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f07077fe640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f070810b900 0x7f070819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070c9ac640 1 -- 172.21.15.42:0/1992968711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07081023f0 msgr2=0x7f070819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070c9ac640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07081023f0 0x7f070819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070c9ac640 1 -- 172.21.15.42:0/1992968711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f070810b900 msgr2=0x7f070819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070c9ac640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f070810b900 0x7f070819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070c9ac640 1 -- 172.21.15.42:0/1992968711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc009560 con 0x7f07081976a0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070c9ac640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081976a0 0x7f0708197b20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0700009fd0 tx=0x7f070000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f07057fa640 1 -- 172.21.15.42:0/1992968711 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000bea0 con 0x7f07081976a0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07081a43f0 con 0x7f07081976a0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f07057fa640 1 -- 172.21.15.42:0/1992968711 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0700009940 con 0x7f07081976a0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f07057fa640 1 -- 172.21.15.42:0/1992968711 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000106c0 con 0x7f07081976a0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.257+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07081a4880 con 0x7f07081976a0 2024-03-12T04:25:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.258+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070806a1a0 con 0x7f07081976a0 2024-03-12T04:25:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.259+0000 7f07057fa640 1 -- 172.21.15.42:0/1992968711 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0700003970 con 0x7f07081976a0 2024-03-12T04:25:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.260+0000 7f07057fa640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f06f00a1c40 0x7f06f00a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.260+0000 7f0707fff640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f06f00a1c40 0x7f06f00a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.260+0000 7f07057fa640 1 -- 172.21.15.42:0/1992968711 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f07000cd4a0 con 0x7f07081976a0 2024-03-12T04:25:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.260+0000 7f0707fff640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f06f00a1c40 0x7f06f00a4100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006010 tx=0x7f06fc005df0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.264+0000 7f07057fa640 1 -- 172.21.15.42:0/1992968711 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0700096370 con 0x7f07081976a0 2024-03-12T04:25:13.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.469+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 --> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f0708102da0 con 0x7f06f00a1c40 2024-03-12T04:25:13.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.476+0000 7f07057fa640 1 -- 172.21.15.42:0/1992968711 <== mgr.15522 v2:172.21.15.42:6800/2974863619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f0708102da0 con 0x7f06f00a1c40 2024-03-12T04:25:13.478 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-12T00:00:00", "created": "2024-03-12T04:15:12", "first": "2024-03-12T04:16:00", "last": "2024-03-12T04:25:00", "last_pruned": "2024-03-12T04:25:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-12T04:25:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f06f00a1c40 msgr2=0x7f06f00a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f06f00a1c40 0x7f06f00a4100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006010 tx=0x7f06fc005df0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081976a0 msgr2=0x7f0708197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081976a0 0x7f0708197b20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0700009fd0 tx=0x7f070000b260 comp rx=0 tx=0).stop 2024-03-12T04:25:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 shutdown_connections 2024-03-12T04:25:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f06f00a1c40 0x7f06f00a4100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07081976a0 0x7f0708197b20 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f070810b900 0x7f070819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 --2- 172.21.15.42:0/1992968711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07081023f0 0x7f070819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.478+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 >> 172.21.15.42:0/1992968711 conn(0x7f07080771a0 msgr2=0x7f07080fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.479+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 shutdown_connections 2024-03-12T04:25:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.479+0000 7f070e40f640 1 -- 172.21.15.42:0/1992968711 wait complete. 2024-03-12T04:25:13.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-12T04:25:13.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 -- 172.21.15.42:0/1466102490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b8100c70 msgr2=0x7fe2b810ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/1466102490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b8100c70 0x7fe2b810ceb0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009ca0 tx=0x7fe2b002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 -- 172.21.15.42:0/1466102490 shutdown_connections 2024-03-12T04:25:13.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/1466102490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b8100c70 0x7fe2b810ceb0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/1466102490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b81002d0 0x7fe2b8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/1466102490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b8105000 0x7fe2b81053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 -- 172.21.15.42:0/1466102490 >> 172.21.15.42:0/1466102490 conn(0x7fe2b8076e50 msgr2=0x7fe2b8077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 -- 172.21.15.42:0/1466102490 shutdown_connections 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.593+0000 7fe2bfe17640 1 -- 172.21.15.42:0/1466102490 wait complete. 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 Processor -- start 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 -- start start 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b81002d0 0x7fe2b8197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b8100c70 0x7fe2b8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b8105000 0x7fe2b819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819c520 con 0x7fe2b81002d0 2024-03-12T04:25:13.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819c690 con 0x7fe2b8105000 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bfe17640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819c850 con 0x7fe2b8100c70 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bdbb3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b81002d0 0x7fe2b8197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bdbb3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b81002d0 0x7fe2b8197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38208/0 (socket says 172.21.15.42:38208) 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bdbb3640 1 -- 172.21.15.42:0/633224166 learned_addr learned my addr 172.21.15.42:0/633224166 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bd3b2640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b8100c70 0x7fe2b8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bdbb3640 1 -- 172.21.15.42:0/633224166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b8100c70 msgr2=0x7fe2b8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bdbb3640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b8100c70 0x7fe2b8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2bdbb3640 1 -- 172.21.15.42:0/633224166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b8105000 msgr2=0x7fe2b819bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.594+0000 7fe2be3b4640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b8105000 0x7fe2b819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2bdbb3640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b8105000 0x7fe2b819bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2bdbb3640 1 -- 172.21.15.42:0/633224166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0009900 con 0x7fe2b81002d0 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2bdbb3640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b81002d0 0x7fe2b8197660 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400d770 tx=0x7fe2b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2aaffd640 1 -- 172.21.15.42:0/633224166 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4009c30 con 0x7fe2b81002d0 2024-03-12T04:25:13.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2aaffd640 1 -- 172.21.15.42:0/633224166 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2b400de80 con 0x7fe2b81002d0 2024-03-12T04:25:13.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b819cb30 con 0x7fe2b81002d0 2024-03-12T04:25:13.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2aaffd640 1 -- 172.21.15.42:0/633224166 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004e00 con 0x7fe2b81002d0 2024-03-12T04:25:13.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.595+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81a43b0 con 0x7fe2b81002d0 2024-03-12T04:25:13.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.596+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b8101790 con 0x7fe2b81002d0 2024-03-12T04:25:13.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.597+0000 7fe2aaffd640 1 -- 172.21.15.42:0/633224166 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe2b4004f60 con 0x7fe2b81002d0 2024-03-12T04:25:13.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.598+0000 7fe2aaffd640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2a40a1cc0 0x7fe2a40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:13.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.598+0000 7fe2bd3b2640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2a40a1cc0 0x7fe2a40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:13.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.598+0000 7fe2aaffd640 1 -- 172.21.15.42:0/633224166 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fe2b40cd2a0 con 0x7fe2b81002d0 2024-03-12T04:25:13.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.598+0000 7fe2bd3b2640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2a40a1cc0 0x7fe2a40a4180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac005fd0 tx=0x7fe2ac005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:13.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.602+0000 7fe2aaffd640 1 -- 172.21.15.42:0/633224166 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2b4096170 con 0x7fe2b81002d0 2024-03-12T04:25:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.806+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 --> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe2b8111240 con 0x7fe2a40a1cc0 2024-03-12T04:25:13.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.813+0000 7fe2aaffd640 1 -- 172.21.15.42:0/633224166 <== mgr.15522 v2:172.21.15.42:6800/2974863619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fe2b8111240 con 0x7fe2a40a1cc0 2024-03-12T04:25:13.815 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2a40a1cc0 msgr2=0x7fe2a40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2a40a1cc0 0x7fe2a40a4180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac005fd0 tx=0x7fe2ac005850 comp rx=0 tx=0).stop 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b81002d0 msgr2=0x7fe2b8197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b81002d0 0x7fe2b8197660 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400d770 tx=0x7fe2b400dc30 comp rx=0 tx=0).stop 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 shutdown_connections 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2a40a1cc0 0x7fe2a40a4180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b8105000 0x7fe2b819bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b8100c70 0x7fe2b8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 --2- 172.21.15.42:0/633224166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2b81002d0 0x7fe2b8197660 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 >> 172.21.15.42:0/633224166 conn(0x7fe2b8076e50 msgr2=0x7fe2b80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 shutdown_connections 2024-03-12T04:25:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:13.815+0000 7fe2bfe17640 1 -- 172.21.15.42:0/633224166 wait complete. 2024-03-12T04:25:13.827 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-12T04:25:13.838 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-12T04:25:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:14 smithi119 ceph-mon[36119]: pgmap v311: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:25:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:14 smithi119 ceph-mon[36119]: from='client.18279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:25:14.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:14 smithi119 ceph-mon[36119]: from='client.18285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:25:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:14 smithi136 ceph-mon[35571]: pgmap v311: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:25:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:14 smithi136 ceph-mon[35571]: from='client.18279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:25:14.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:14 smithi136 ceph-mon[35571]: from='client.18285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:25:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:14 smithi042 ceph-mon[32260]: pgmap v311: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-12T04:25:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:14 smithi042 ceph-mon[32260]: from='client.18279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:25:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:14 smithi042 ceph-mon[32260]: from='client.18285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T04:25:16.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:16 smithi119 ceph-mon[36119]: osdmap e98: 12 total, 12 up, 12 in 2024-03-12T04:25:16.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:16 smithi136 ceph-mon[35571]: osdmap e98: 12 total, 12 up, 12 in 2024-03-12T04:25:16.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:16 smithi042 ceph-mon[32260]: osdmap e98: 12 total, 12 up, 12 in 2024-03-12T04:25:17.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:17 smithi119 ceph-mon[36119]: pgmap v313: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:25:17.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:17 smithi136 ceph-mon[35571]: pgmap v313: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:25:17.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:17 smithi042 ceph-mon[32260]: pgmap v313: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-12T04:25:19.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:19 smithi119 ceph-mon[36119]: pgmap v314: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-03-12T04:25:19.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:19 smithi136 ceph-mon[35571]: pgmap v314: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-03-12T04:25:19.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:19 smithi042 ceph-mon[32260]: pgmap v314: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-03-12T04:25:20.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:20 smithi042 ceph-mon[32260]: pgmap v315: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 206 B/s wr, 13 op/s 2024-03-12T04:25:20.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:20 smithi119 ceph-mon[36119]: pgmap v315: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 206 B/s wr, 13 op/s 2024-03-12T04:25:20.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:20 smithi136 ceph-mon[35571]: pgmap v315: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 206 B/s wr, 13 op/s 2024-03-12T04:25:20.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T04:25:20.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T04:25:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 -- 172.21.15.42:0/1516900885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 msgr2=0x7f430010ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 --2- 172.21.15.42:0/1516900885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f430010ce90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009ca0 tx=0x7f42f802c140 comp rx=0 tx=0).stop 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 -- 172.21.15.42:0/1516900885 shutdown_connections 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 --2- 172.21.15.42:0/1516900885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f430010ce90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 --2- 172.21.15.42:0/1516900885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43001002d0 0x7f4300100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 --2- 172.21.15.42:0/1516900885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4300106160 0x7f4300106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 -- 172.21.15.42:0/1516900885 >> 172.21.15.42:0/1516900885 conn(0x7f4300076e60 msgr2=0x7f4300077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 -- 172.21.15.42:0/1516900885 shutdown_connections 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.084+0000 7f430581b640 1 -- 172.21.15.42:0/1516900885 wait complete. 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 Processor -- start 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 -- start start 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43001002d0 0x7f43001976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f4300197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4300106160 0x7f430019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430019c580 con 0x7f4300100c90 2024-03-12T04:25:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430019c6f0 con 0x7f4300106160 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f430581b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430019c8b0 con 0x7f43001002d0 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f4300197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f4300197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38212/0 (socket says 172.21.15.42:38212) 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 -- 172.21.15.42:0/3158483046 learned_addr learned my addr 172.21.15.42:0/3158483046 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42ff7fe640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4300106160 0x7f430019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42feffd640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43001002d0 0x7f43001976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 -- 172.21.15.42:0/3158483046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43001002d0 msgr2=0x7f43001976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43001002d0 0x7f43001976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 -- 172.21.15.42:0/3158483046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4300106160 msgr2=0x7f430019bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4300106160 0x7f430019bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42f7fff640 1 -- 172.21.15.42:0/3158483046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f8009900 con 0x7f4300100c90 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42ff7fe640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4300106160 0x7f430019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.085+0000 7f42feffd640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43001002d0 0x7f43001976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.086+0000 7f42f7fff640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f4300197c00 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f42e800d950 tx=0x7f42e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.086+0000 7f4304819640 1 -- 172.21.15.42:0/3158483046 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8009c30 con 0x7f4300100c90 2024-03-12T04:25:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.086+0000 7f4304819640 1 -- 172.21.15.42:0/3158483046 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f42e8009d90 con 0x7f4300100c90 2024-03-12T04:25:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.086+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430019cb90 con 0x7f4300100c90 2024-03-12T04:25:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.086+0000 7f4304819640 1 -- 172.21.15.42:0/3158483046 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e800f6b0 con 0x7f4300100c90 2024-03-12T04:25:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.086+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43001a4410 con 0x7f4300100c90 2024-03-12T04:25:21.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.088+0000 7f4304819640 1 -- 172.21.15.42:0/3158483046 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f42e8010aa0 con 0x7f4300100c90 2024-03-12T04:25:21.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.088+0000 7f43083ea740 1 Processor -- start 2024-03-12T04:25:21.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.088+0000 7f43083ea740 1 -- start start 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f43083ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f4304819640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42ec0a1ba0 0x7f42ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f43083ea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 0x556cf1d34270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f43083ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 0x556cf1d36ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f43083ea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1c9a280 con 0x556cf1d2b2b0 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f43083ea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1c977b0 con 0x556cf1d2bbd0 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f43083ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1c92f90 con 0x556cf1d347b0 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42ff7fe640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42ec0a1ba0 0x7f42ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f4304819640 1 -- 172.21.15.42:0/3158483046 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f42e80cd560 con 0x7f4300100c90 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d2b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38224/0 (socket says 172.21.15.42:38224) 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 -- 172.21.15.42:0/902814581 learned_addr learned my addr 172.21.15.42:0/902814581 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42ff7fe640 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 0x556cf1d36ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42f7fff640 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 0x556cf1d34270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 -- 172.21.15.42:0/902814581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 msgr2=0x556cf1d36ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 0x556cf1d36ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 -- 172.21.15.42:0/902814581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 msgr2=0x556cf1d34270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 0x556cf1d34270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 -- 172.21.15.42:0/902814581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf1c51570 con 0x556cf1d2b2b0 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42ff7fe640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42ec0a1ba0 0x7f42ec0a4060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f42f8006010 tx=0x7f42f8034440 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42feffd640 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d2b690 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f42f000aa10 tx=0x7f42f000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dee92715be71bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.089+0000 7f42f5ffb640 1 -- 172.21.15.42:0/902814581 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000cc60 con 0x556cf1d2b2b0 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f42f5ffb640 1 -- 172.21.15.42:0/902814581 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f42f000cdc0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f42f5ffb640 1 -- 172.21.15.42:0/902814581 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00186d0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 -- 172.21.15.42:0/902814581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 msgr2=0x556cf1d2b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d2b690 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f42f000aa10 tx=0x7f42f000aed0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 -- 172.21.15.42:0/902814581 shutdown_connections 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 0x556cf1d36ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 0x556cf1d34270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 --2- 172.21.15.42:0/902814581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d2b690 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 -- 172.21.15.42:0/902814581 >> 172.21.15.42:0/902814581 conn(0x556cf1d25990 msgr2=0x556cf1d25d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 -- 172.21.15.42:0/902814581 shutdown_connections 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.090+0000 7f43083ea740 1 -- 172.21.15.42:0/902814581 wait complete. 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 -- start start 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 Processor -- start 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d34180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2bbd0 0x556cf1d31580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 0x556cf1d31c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1966430 con 0x556cf1d2b2b0 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1922e00 con 0x556cf1d347b0 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1c17fd0 con 0x556cf1d2bbd0 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f43083ea740 10 client.0 _ll_register_callbacks cb 0x556cf1c6dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f42feffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d34180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f42feffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d34180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38236/0 (socket says 172.21.15.42:38236) 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f42feffd640 1 -- 172.21.15.42:0/197703696 learned_addr learned my addr 172.21.15.42:0/197703696 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f42ff7fe640 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 0x556cf1d31c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.091+0000 7f42f7fff640 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2bbd0 0x556cf1d31580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42feffd640 1 -- 172.21.15.42:0/197703696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2bbd0 msgr2=0x556cf1d31580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42feffd640 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2bbd0 0x556cf1d31580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42feffd640 1 -- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 msgr2=0x556cf1d31c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42feffd640 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 0x556cf1d31c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42feffd640 1 -- 172.21.15.42:0/197703696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f000a640 con 0x556cf1d2b2b0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42feffd640 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d34180 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c520 tx=0x7f42f0011600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d59027a0834170 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42deffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42deffd640 1 -- 172.21.15.42:0/197703696 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f001a450 con 0x556cf1d2b2b0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42deffd640 1 -- 172.21.15.42:0/197703696 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f42f001a5b0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cf19927e0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42deffd640 1 -- 172.21.15.42:0/197703696 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42f001a770 con 0x556cf1d2b2b0 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f43083ea740 10 client.18297 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556cf1c91610 con 0x556cf1d2b2b0 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42deffd640 1 -- 172.21.15.42:0/197703696 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f42f0030080 con 0x556cf1d2b2b0 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f42deffd640 1 -- 172.21.15.42:0/197703696 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f42f002c9d0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f43083ea740 10 client.18297 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.092+0000 7f43083ea740 10 client.18297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.093+0000 7f43083ea740 20 client.18297 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.093+0000 7f43083ea740 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d44790 0x556cf1d46bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.093+0000 7f43083ea740 4 client.18297 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.093+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cf19cf720 con 0x556cf1d44790 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.093+0000 7f42f7fff640 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d44790 0x556cf1d46bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.093+0000 7f42deffd640 10 client.18297 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.093+0000 7f42f7fff640 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d44790 0x556cf1d46bb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5abd4cb798fbb8dd server_cookie=7e88168eb6e37c5f in_seq=0 out_seq=0 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f42deffd640 1 -- 172.21.15.42:0/197703696 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556cf19cf720 con 0x556cf1d44790 2024-03-12T04:25:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f42deffd640 10 client.18297 handle_command_reply: tid=0 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 client.18297 shutdown 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 msgr2=0x556cf1d34180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d34180 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c520 tx=0x7f42f0011600 comp rx=0 tx=0).stop 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 shutdown_connections 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d44790 0x556cf1d46bb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 0x556cf1d31c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2bbd0 0x556cf1d31580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f42deffd640 0 client.18297 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 --2- 172.21.15.42:0/197703696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d34180 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 >> 172.21.15.42:0/197703696 conn(0x556cf1d25990 msgr2=0x556cf1d27730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 shutdown_connections 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 1 -- 172.21.15.42:0/197703696 wait complete. 2024-03-12T04:25:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.094+0000 7f43083ea740 20 client.18297 trim_cache size 0 max 16384 2024-03-12T04:25:21.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.109+0000 7f43083ea740 1 Processor -- start 2024-03-12T04:25:21.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.109+0000 7f43083ea740 1 -- start start 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f43083ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d33cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f43083ea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 0x556cf1d3b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f43083ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 0x556cf1d3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f43083ea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1d4be60 con 0x556cf1d2b2b0 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f43083ea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1d2a600 con 0x556cf1d2bbd0 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f43083ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1d44540 con 0x556cf1d347b0 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d33cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d33cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38246/0 (socket says 172.21.15.42:38246) 2024-03-12T04:25:21.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 -- 172.21.15.42:0/3015106259 learned_addr learned my addr 172.21.15.42:0/3015106259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 -- 172.21.15.42:0/3015106259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 msgr2=0x556cf1d3bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 --2- 172.21.15.42:0/3015106259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 0x556cf1d3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 -- 172.21.15.42:0/3015106259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 msgr2=0x556cf1d3b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 --2- 172.21.15.42:0/3015106259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 0x556cf1d3b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 -- 172.21.15.42:0/3015106259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf1c2dac0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42feffd640 1 --2- 172.21.15.42:0/3015106259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d33cb0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006aa0 tx=0x7f42f000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c9c153de43ae96 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42f57fa640 1 -- 172.21.15.42:0/3015106259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0011de0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42f57fa640 1 -- 172.21.15.42:0/3015106259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x556cf19927e0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.110+0000 7f42f57fa640 1 -- 172.21.15.42:0/3015106259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0024600 con 0x556cf1d2b2b0 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 -- 172.21.15.42:0/3015106259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 msgr2=0x556cf1d33cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 --2- 172.21.15.42:0/3015106259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d33cb0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006aa0 tx=0x7f42f000a860 comp rx=0 tx=0).stop 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 -- 172.21.15.42:0/3015106259 shutdown_connections 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 --2- 172.21.15.42:0/3015106259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d347b0 0x556cf1d3bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 --2- 172.21.15.42:0/3015106259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d2bbd0 0x556cf1d3b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 --2- 172.21.15.42:0/3015106259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2b2b0 0x556cf1d33cb0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 -- 172.21.15.42:0/3015106259 >> 172.21.15.42:0/3015106259 conn(0x556cf1d25990 msgr2=0x556cf1d2c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 -- 172.21.15.42:0/3015106259 shutdown_connections 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 -- 172.21.15.42:0/3015106259 wait complete. 2024-03-12T04:25:21.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 -- start start 2024-03-12T04:25:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.111+0000 7f43083ea740 1 Processor -- start 2024-03-12T04:25:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2b2b0 0x556cf1d3b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2bbd0 0x556cf1d37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 0x556cf1d38170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1d386b0 con 0x556cf1d2bbd0 2024-03-12T04:25:21.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1d38820 con 0x556cf1d347b0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf1d389e0 con 0x556cf1d2b2b0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 10 client.0 _ll_register_callbacks cb 0x7f42f000a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2bbd0 0x556cf1d37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2bbd0 0x556cf1d37c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38254/0 (socket says 172.21.15.42:38254) 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 -- 172.21.15.42:0/4235477186 learned_addr learned my addr 172.21.15.42:0/4235477186 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 -- 172.21.15.42:0/4235477186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2b2b0 msgr2=0x556cf1d3b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2b2b0 0x556cf1d3b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 -- 172.21.15.42:0/4235477186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 msgr2=0x556cf1d38170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 0x556cf1d38170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 -- 172.21.15.42:0/4235477186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cf1c91610 con 0x556cf1d2bbd0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42f7fff640 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2bbd0 0x556cf1d37c30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f42e80cb2d0 tx=0x7f42e800b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf749a0a228b0a2d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42deffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42deffd640 1 -- 172.21.15.42:0/4235477186 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e809e120 con 0x556cf1d2bbd0 2024-03-12T04:25:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42deffd640 1 -- 172.21.15.42:0/4235477186 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f42e8096820 con 0x556cf1d2bbd0 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42f000cdc0 con 0x556cf1d2bbd0 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.112+0000 7f42deffd640 1 -- 172.21.15.42:0/4235477186 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e80969e0 con 0x556cf1d2bbd0 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 10 client.18303 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556cf1c536a0 con 0x556cf1d2bbd0 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f42deffd640 1 -- 172.21.15.42:0/4235477186 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f42e80a6080 con 0x556cf1d2bbd0 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f42deffd640 1 -- 172.21.15.42:0/4235477186 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f42e8094d20 con 0x556cf1d2bbd0 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 10 client.18303 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 20 client.18303 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d9c150 0x556cf1d9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 4 client.18303 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556cf1a1e890 con 0x556cf1d9c150 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.113+0000 7f42feffd640 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d9c150 0x556cf1d9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f42deffd640 10 client.18303 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f42feffd640 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d9c150 0x556cf1d9e570 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2268eca9a4fe90 server_cookie=48069e0034c8105 in_seq=0 out_seq=0 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f42deffd640 1 -- 172.21.15.42:0/4235477186 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556cf1a1e890 con 0x556cf1d9c150 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f42deffd640 10 client.18303 handle_command_reply: tid=0 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 client.18303 shutdown 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2bbd0 msgr2=0x556cf1d37c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2bbd0 0x556cf1d37c30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f42e80cb2d0 tx=0x7f42e800b530 comp rx=0 tx=0).stop 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 shutdown_connections 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x556cf1d9c150 0x556cf1d9e570 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cf1d347b0 0x556cf1d38170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556cf1d2bbd0 0x556cf1d37c30 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f42deffd640 0 client.18303 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 --2- 172.21.15.42:0/4235477186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cf1d2b2b0 0x556cf1d3b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 >> 172.21.15.42:0/4235477186 conn(0x556cf1d25990 msgr2=0x556cf1d241c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 shutdown_connections 2024-03-12T04:25:21.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 1 -- 172.21.15.42:0/4235477186 wait complete. 2024-03-12T04:25:21.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.114+0000 7f43083ea740 20 client.18303 trim_cache size 0 max 16384 2024-03-12T04:25:21.124 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:25:21.124 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-03-12T04:25:21.124 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-03-12T04:25:21.124 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42ec0a1ba0 msgr2=0x7f42ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42ec0a1ba0 0x7f42ec0a4060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f42f8006010 tx=0x7f42f8034440 comp rx=0 tx=0).stop 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 msgr2=0x7f4300197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f4300197c00 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f42e800d950 tx=0x7f42e800de10 comp rx=0 tx=0).stop 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 shutdown_connections 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f42ec0a1ba0 0x7f42ec0a4060 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4300106160 0x7f430019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4300100c90 0x7f4300197c00 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 --2- 172.21.15.42:0/3158483046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43001002d0 0x7f43001976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 >> 172.21.15.42:0/3158483046 conn(0x7f4300076e60 msgr2=0x7f4300069930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 shutdown_connections 2024-03-12T04:25:21.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.115+0000 7f430581b640 1 -- 172.21.15.42:0/3158483046 wait complete. 2024-03-12T04:25:21.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T04:25:21.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:25:21.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/3289794354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc106500 msgr2=0x7fd9dc10e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/3289794354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc106500 0x7fd9dc10e080 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009cd0 tx=0x7fd9cc02c0f0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/3289794354 shutdown_connections 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/3289794354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc106500 0x7fd9dc10e080 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/3289794354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc1002a0 0x7fd9dc105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/3289794354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9dc0ff8f0 0x7fd9dc0ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/3289794354 >> 172.21.15.42:0/3289794354 conn(0x7fd9dc076610 msgr2=0x7fd9dc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/3289794354 shutdown_connections 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/3289794354 wait complete. 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.229+0000 7fd9e3aa7640 1 Processor -- start 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 -- start start 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9dc0ff8f0 0x7fd9dc100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc106500 0x7fd9dc100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc1012e0 0x7fd9dc1a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a5e70 con 0x7fd9dc1012e0 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a5fe0 con 0x7fd9dc106500 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a61a0 con 0x7fd9dc0ff8f0 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc1012e0 0x7fd9dc1a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc1012e0 0x7fd9dc1a58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38266/0 (socket says 172.21.15.42:38266) 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 -- 172.21.15.42:0/496276357 learned_addr learned my addr 172.21.15.42:0/496276357 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e1843640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9dc0ff8f0 0x7fd9dc100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 -- 172.21.15.42:0/496276357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9dc0ff8f0 msgr2=0x7fd9dc100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9dc0ff8f0 0x7fd9dc100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 -- 172.21.15.42:0/496276357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc106500 msgr2=0x7fd9dc100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc106500 0x7fd9dc100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 -- 172.21.15.42:0/496276357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc009900 con 0x7fd9dc1012e0 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e2044640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc1012e0 0x7fd9dc1a58a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc004af0 tx=0x7fd9cc035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9daffd640 1 -- 172.21.15.42:0/496276357 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0329e0 con 0x7fd9dc1012e0 2024-03-12T04:25:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9daffd640 1 -- 172.21.15.42:0/496276357 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd9cc035e90 con 0x7fd9dc1012e0 2024-03-12T04:25:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.230+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc1a6420 con 0x7fd9dc1012e0 2024-03-12T04:25:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.231+0000 7fd9daffd640 1 -- 172.21.15.42:0/496276357 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004720 con 0x7fd9dc1012e0 2024-03-12T04:25:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.231+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc1a68d0 con 0x7fd9dc1012e0 2024-03-12T04:25:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.232+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc101d90 con 0x7fd9dc1012e0 2024-03-12T04:25:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.232+0000 7fd9daffd640 1 -- 172.21.15.42:0/496276357 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fd9cc03b040 con 0x7fd9dc1012e0 2024-03-12T04:25:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.233+0000 7fd9daffd640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9c40a1bd0 0x7fd9c40a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.233+0000 7fd9daffd640 1 -- 172.21.15.42:0/496276357 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd9cc0f0240 con 0x7fd9dc1012e0 2024-03-12T04:25:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.233+0000 7fd9e1843640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9c40a1bd0 0x7fd9c40a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.234+0000 7fd9e1843640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9c40a1bd0 0x7fd9c40a4090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005fd0 tx=0x7fd9d4009470 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.238+0000 7fd9daffd640 1 -- 172.21.15.42:0/496276357 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd9cc0b9240 con 0x7fd9dc1012e0 2024-03-12T04:25:21.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.459+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9dc109700 con 0x7fd9dc1012e0 2024-03-12T04:25:21.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.460+0000 7fd9daffd640 1 -- 172.21.15.42:0/496276357 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7fd9cc0bc900 con 0x7fd9dc1012e0 2024-03-12T04:25:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:25:21.463 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:25:21.463 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:25:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9c40a1bd0 msgr2=0x7fd9c40a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9c40a1bd0 0x7fd9c40a4090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005fd0 tx=0x7fd9d4009470 comp rx=0 tx=0).stop 2024-03-12T04:25:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc1012e0 msgr2=0x7fd9dc1a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc1012e0 0x7fd9dc1a58a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc004af0 tx=0x7fd9cc035ca0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 shutdown_connections 2024-03-12T04:25:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9c40a1bd0 0x7fd9c40a4090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9dc1012e0 0x7fd9dc1a58a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc106500 0x7fd9dc100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 --2- 172.21.15.42:0/496276357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9dc0ff8f0 0x7fd9dc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 >> 172.21.15.42:0/496276357 conn(0x7fd9dc076610 msgr2=0x7fd9dc1047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 shutdown_connections 2024-03-12T04:25:21.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.462+0000 7fd9e3aa7640 1 -- 172.21.15.42:0/496276357 wait complete. 2024-03-12T04:25:21.476 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:25:21.476 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 -- 172.21.15.42:0/1836430263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101d30 msgr2=0x7febc0105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 --2- 172.21.15.42:0/1836430263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101d30 0x7febc0105d70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7febb0009930 tx=0x7febb002bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 -- 172.21.15.42:0/1836430263 shutdown_connections 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 --2- 172.21.15.42:0/1836430263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc0106450 0x7febc010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 --2- 172.21.15.42:0/1836430263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101d30 0x7febc0105d70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 --2- 172.21.15.42:0/1836430263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febc0101380 0x7febc0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 -- 172.21.15.42:0/1836430263 >> 172.21.15.42:0/1836430263 conn(0x7febc0078580 msgr2=0x7febc0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 -- 172.21.15.42:0/1836430263 shutdown_connections 2024-03-12T04:25:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.578+0000 7febc84f4640 1 -- 172.21.15.42:0/1836430263 wait complete. 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 Processor -- start 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 -- start start 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101380 0x7febc01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febc0106450 0x7febc0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc019bfa0 0x7febc019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019c990 con 0x7febc0101380 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019cb00 con 0x7febc019bfa0 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019cc70 con 0x7febc0106450 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101380 0x7febc01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101380 0x7febc01976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40458/0 (socket says 172.21.15.42:40458) 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 -- 172.21.15.42:0/2735008012 learned_addr learned my addr 172.21.15.42:0/2735008012 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc5a8f640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febc0106450 0x7febc0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6a91640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc019bfa0 0x7febc019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 -- 172.21.15.42:0/2735008012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febc0106450 msgr2=0x7febc0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febc0106450 0x7febc0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 -- 172.21.15.42:0/2735008012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc019bfa0 msgr2=0x7febc019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc019bfa0 0x7febc019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 -- 172.21.15.42:0/2735008012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb0009560 con 0x7febc0101380 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc6290640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101380 0x7febc01976e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c920 tx=0x7febbc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febb77fe640 1 -- 172.21.15.42:0/2735008012 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007c70 con 0x7febc0101380 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febb77fe640 1 -- 172.21.15.42:0/2735008012 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febbc007dd0 con 0x7febc0101380 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.579+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc01a4250 con 0x7febc0101380 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.580+0000 7febb77fe640 1 -- 172.21.15.42:0/2735008012 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc0187c0 con 0x7febc0101380 2024-03-12T04:25:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.580+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc01a4720 con 0x7febc0101380 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.581+0000 7febb77fe640 1 -- 172.21.15.42:0/2735008012 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7febbc00f450 con 0x7febc0101380 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.582+0000 7febcb1a1740 1 Processor -- start 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.582+0000 7febb77fe640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7febac0a1c20 0x7febac0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.582+0000 7febcb1a1740 1 -- start start 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.582+0000 7febb77fe640 1 -- 172.21.15.42:0/2735008012 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7febbc0cd9e0 con 0x7febc0101380 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.582+0000 7febc6a91640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7febac0a1c20 0x7febac0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8267630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8270210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8272b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febcb1a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc8222960 con 0x558fc8270750 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febcb1a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc81d6310 con 0x558fc8267b70 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febcb1a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc81d3840 con 0x558fc8267250 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8272b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8272b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40464/0 (socket says 172.21.15.42:40464) 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 -- 172.21.15.42:0/1325907327 learned_addr learned my addr 172.21.15.42:0/1325907327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7febac0a1c20 0x7febac0a40e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febb8009370 tx=0x7febb800f730 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6290640 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8267630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc5a8f640 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8270210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 -- 172.21.15.42:0/1325907327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 msgr2=0x558fc8267630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8267630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 -- 172.21.15.42:0/1325907327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 msgr2=0x558fc8270210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8270210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 -- 172.21.15.42:0/1325907327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fc8191060 con 0x558fc8270750 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febc6a91640 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8272b40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7febb80126f0 tx=0x7febb801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc77e96c772f05b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.583+0000 7febb4bfe640 1 -- 172.21.15.42:0/1325907327 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb800d850 con 0x558fc8270750 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febb4bfe640 1 -- 172.21.15.42:0/1325907327 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febb800de60 con 0x558fc8270750 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febb4bfe640 1 -- 172.21.15.42:0/1325907327 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb801de40 con 0x558fc8270750 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 -- 172.21.15.42:0/1325907327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 msgr2=0x558fc8272b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8272b40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7febb80126f0 tx=0x7febb801a000 comp rx=0 tx=0).stop 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 -- 172.21.15.42:0/1325907327 shutdown_connections 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8272b40 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8270210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1325907327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8267630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 -- 172.21.15.42:0/1325907327 >> 172.21.15.42:0/1325907327 conn(0x558fc8261980 msgr2=0x558fc8261d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 -- 172.21.15.42:0/1325907327 shutdown_connections 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.584+0000 7febcb1a1740 1 -- 172.21.15.42:0/1325907327 wait complete. 2024-03-12T04:25:21.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 -- start start 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 Processor -- start 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267250 0x558fc826d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267b70 0x558fc826dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 0x558fc826e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc81cf020 con 0x558fc8267b70 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc7ea2430 con 0x558fc8267250 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc7e5ee00 con 0x558fc8270750 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febcb1a1740 10 client.0 _ll_register_callbacks cb 0x558fc8176cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267b70 0x558fc826dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267b70 0x558fc826dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40470/0 (socket says 172.21.15.42:40470) 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 -- 172.21.15.42:0/3140774427 learned_addr learned my addr 172.21.15.42:0/3140774427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc6a91640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 0x558fc826e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc6290640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267250 0x558fc826d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 -- 172.21.15.42:0/3140774427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 msgr2=0x558fc826e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 0x558fc826e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 -- 172.21.15.42:0/3140774427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267250 msgr2=0x558fc826d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267250 0x558fc826d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc5a8f640 1 -- 172.21.15.42:0/3140774427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb80123f0 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.585+0000 7febc6290640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267250 0x558fc826d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febc5a8f640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267b70 0x558fc826dc20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7febc006aa20 tx=0x7febb002c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874fb0e96492f656 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7feba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7feba9ffb640 1 -- 172.21.15.42:0/3140774427 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb002cb70 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7feba9ffb640 1 -- 172.21.15.42:0/3140774427 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7febb0004260 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fc7ece7e0 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7feba9ffb640 1 -- 172.21.15.42:0/3140774427 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb0004420 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 10 client.18321 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558fc81cd6a0 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7feba9ffb640 1 -- 172.21.15.42:0/3140774427 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558fc7ece7e0 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7feba9ffb640 1 -- 172.21.15.42:0/3140774427 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7febb003c040 con 0x558fc8267b70 2024-03-12T04:25:21.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 10 client.18321 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 10 client.18321 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 20 client.18321 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc8280a20 0x558fc8282e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 4 client.18321 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fc7f0b720 con 0x558fc8280a20 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.586+0000 7febc6290640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc8280a20 0x558fc8282e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7feba9ffb640 10 client.18321 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febc6290640 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc8280a20 0x558fc8282e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3baa26e85118f935 server_cookie=7d23928be047c913 in_seq=0 out_seq=0 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7feba9ffb640 1 -- 172.21.15.42:0/3140774427 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558fc7f0b720 con 0x558fc8280a20 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7feba9ffb640 10 client.18321 handle_command_reply: tid=0 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 client.18321 shutdown 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267b70 msgr2=0x558fc826dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267b70 0x558fc826dc20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7febc006aa20 tx=0x7febb002c980 comp rx=0 tx=0).stop 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 shutdown_connections 2024-03-12T04:25:21.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 0x558fc826e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc8280a20 0x558fc8282e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267b70 0x558fc826dc20 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 --2- 172.21.15.42:0/3140774427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267250 0x558fc826d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7feba9ffb640 0 client.18321 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 >> 172.21.15.42:0/3140774427 conn(0x558fc8261980 msgr2=0x558fc82636d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 shutdown_connections 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.587+0000 7febcb1a1740 1 -- 172.21.15.42:0/3140774427 wait complete. 2024-03-12T04:25:21.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.588+0000 7febcb1a1740 20 client.18321 trim_cache size 0 max 16384 2024-03-12T04:25:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 Processor -- start 2024-03-12T04:25:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 -- start start 2024-03-12T04:25:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267250 0x558fc826fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc826df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 0x558fc826a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc8154130 con 0x558fc8267250 2024-03-12T04:25:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc8262780 con 0x558fc8267b70 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febcb1a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc8261840 con 0x558fc8270750 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.599+0000 7febc6290640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267250 0x558fc826fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267250 0x558fc826fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40480/0 (socket says 172.21.15.42:40480) 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 -- 172.21.15.42:0/594383437 learned_addr learned my addr 172.21.15.42:0/594383437 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc5a8f640 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc826df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 -- 172.21.15.42:0/594383437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 msgr2=0x558fc826a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 0x558fc826a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 -- 172.21.15.42:0/594383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 msgr2=0x558fc826df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc826df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 -- 172.21.15.42:0/594383437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fc8169c20 con 0x558fc8267250 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febc6290640 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267250 0x558fc826fe10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7febbc00fd30 tx=0x7febbc00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aa7e3e49e3695 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febabfff640 1 -- 172.21.15.42:0/594383437 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc09e720 con 0x558fc8267250 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febabfff640 1 -- 172.21.15.42:0/594383437 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febbc096040 con 0x558fc8267250 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.600+0000 7febabfff640 1 -- 172.21.15.42:0/594383437 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc0850b0 con 0x558fc8267250 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- 172.21.15.42:0/594383437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267250 msgr2=0x558fc826fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267250 0x558fc826fe10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7febbc00fd30 tx=0x7febbc00fc80 comp rx=0 tx=0).stop 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- 172.21.15.42:0/594383437 shutdown_connections 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8270750 0x558fc826a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc826df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 --2- 172.21.15.42:0/594383437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8267250 0x558fc826fe10 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- 172.21.15.42:0/594383437 >> 172.21.15.42:0/594383437 conn(0x558fc8261980 msgr2=0x558fc82689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- 172.21.15.42:0/594383437 shutdown_connections 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- 172.21.15.42:0/594383437 wait complete. 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- start start 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 Processor -- start 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8275fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8276500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8276a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc8277f10 con 0x558fc8270750 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc82db380 con 0x558fc8267b70 2024-03-12T04:25:21.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc82db540 con 0x558fc8267250 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febcb1a1740 10 client.0 _ll_register_callbacks cb 0x7febb8009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febc6a91640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8276a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febc6a91640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8276a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40490/0 (socket says 172.21.15.42:40490) 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febc6a91640 1 -- 172.21.15.42:0/1955424723 learned_addr learned my addr 172.21.15.42:0/1955424723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.601+0000 7febc5a8f640 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8276500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febc6290640 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8275fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febc6a91640 1 -- 172.21.15.42:0/1955424723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 msgr2=0x558fc8275fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febc6a91640 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8275fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febc6a91640 1 -- 172.21.15.42:0/1955424723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 msgr2=0x558fc8276500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febc6a91640 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8276500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febc6a91640 1 -- 172.21.15.42:0/1955424723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc01a4720 con 0x558fc8270750 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febc6a91640 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8276a40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7febb80126c0 tx=0x7febb801de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26337d4de4018b62 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7feba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7feba9ffb640 1 -- 172.21.15.42:0/1955424723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb801d6f0 con 0x558fc8270750 2024-03-12T04:25:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7feba9ffb640 1 -- 172.21.15.42:0/1955424723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7febb8018050 con 0x558fc8270750 2024-03-12T04:25:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fc8262300 con 0x558fc8270750 2024-03-12T04:25:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7feba9ffb640 1 -- 172.21.15.42:0/1955424723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb801d850 con 0x558fc8270750 2024-03-12T04:25:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 10 client.18327 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558fc818f8c0 con 0x558fc8270750 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7feba9ffb640 1 -- 172.21.15.42:0/1955424723 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558fc8262300 con 0x558fc8270750 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7feba9ffb640 1 -- 172.21.15.42:0/1955424723 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7febb802c070 con 0x558fc8270750 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 10 client.18327 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 10 client.18327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 20 client.18327 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc82dcd50 0x558fc82df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 4 client.18327 mds_command: new command op to 14634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:25:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.602+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 --> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7febb800de60 con 0x558fc82dcd50 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febc6290640 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc82dcd50 0x558fc82df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7feba9ffb640 10 client.18327 ms_handle_connect on v2:172.21.15.42:6840/799836757 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febc6290640 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc82dcd50 0x558fc82df170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63d93acc05e311a2 server_cookie=4c0ab88af789cdc7 in_seq=0 out_seq=0 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7feba9ffb640 1 -- 172.21.15.42:0/1955424723 <== mds.4 v2:172.21.15.42:6840/799836757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7febb800de60 con 0x558fc82dcd50 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7feba9ffb640 10 client.18327 handle_command_reply: tid=0 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 client.18327 shutdown 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 msgr2=0x558fc8276a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8276a40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7febb80126c0 tx=0x7febb801de70 comp rx=0 tx=0).stop 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 shutdown_connections 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:6840/799836757,v1:172.21.15.42:6841/799836757] conn(0x558fc82dcd50 0x558fc82df170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558fc8270750 0x558fc8276a40 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fc8267b70 0x558fc8276500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 --2- 172.21.15.42:0/1955424723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fc8267250 0x558fc8275fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7feba9ffb640 0 client.18327 ms_handle_reset on v2:172.21.15.42:6840/799836757 2024-03-12T04:25:21.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 >> 172.21.15.42:0/1955424723 conn(0x558fc8261980 msgr2=0x558fc82602f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 shutdown_connections 2024-03-12T04:25:21.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 1 -- 172.21.15.42:0/1955424723 wait complete. 2024-03-12T04:25:21.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.603+0000 7febcb1a1740 20 client.18327 trim_cache size 0 max 16384 2024-03-12T04:25:21.615 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7febac0a1c20 msgr2=0x7febac0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7febac0a1c20 0x7febac0a40e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febb8009370 tx=0x7febb800f730 comp rx=0 tx=0).stop 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101380 msgr2=0x7febc01976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101380 0x7febc01976e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c920 tx=0x7febbc00cde0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 shutdown_connections 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7febac0a1c20 0x7febac0a40e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc019bfa0 0x7febc019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febc0106450 0x7febc0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 --2- 172.21.15.42:0/2735008012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101380 0x7febc01976e0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 >> 172.21.15.42:0/2735008012 conn(0x7febc0078580 msgr2=0x7febc00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 shutdown_connections 2024-03-12T04:25:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.604+0000 7febc84f4640 1 -- 172.21.15.42:0/2735008012 wait complete. 2024-03-12T04:25:21.618 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:25:21.618 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 -- 172.21.15.42:0/209452679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4100260 msgr2=0x7efca4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 --2- 172.21.15.42:0/209452679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4100260 0x7efca4100640 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7efc94009930 tx=0x7efc9402bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 -- 172.21.15.42:0/209452679 shutdown_connections 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 --2- 172.21.15.42:0/209452679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca4105330 0x7efca410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 --2- 172.21.15.42:0/209452679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca4100c10 0x7efca4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 --2- 172.21.15.42:0/209452679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4100260 0x7efca4100640 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 -- 172.21.15.42:0/209452679 >> 172.21.15.42:0/209452679 conn(0x7efca4077180 msgr2=0x7efca4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 -- 172.21.15.42:0/209452679 shutdown_connections 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.719+0000 7efcaa84e640 1 -- 172.21.15.42:0/209452679 wait complete. 2024-03-12T04:25:21.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 Processor -- start 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 -- start start 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca4100260 0x7efca4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca4105330 0x7efca4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4198fd0 0x7efca41a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41a3c20 con 0x7efca4198fd0 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41a3d90 con 0x7efca4100260 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.720+0000 7efcaa84e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41a3f50 con 0x7efca4105330 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4198fd0 0x7efca41a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4198fd0 0x7efca41a3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40510/0 (socket says 172.21.15.42:40510) 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 -- 172.21.15.42:0/381261430 learned_addr learned my addr 172.21.15.42:0/381261430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca3fff640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca4100260 0x7efca4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 -- 172.21.15.42:0/381261430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca4105330 msgr2=0x7efca4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca4105330 0x7efca4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 -- 172.21.15.42:0/381261430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca4100260 msgr2=0x7efca4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca4100260 0x7efca4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 -- 172.21.15.42:0/381261430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94009560 con 0x7efca4198fd0 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca3fff640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca4100260 0x7efca4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca8deb640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4198fd0 0x7efca41a3580 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00eb10 tx=0x7efc9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca17fa640 1 -- 172.21.15.42:0/381261430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c004b20 con 0x7efca4198fd0 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca17fa640 1 -- 172.21.15.42:0/381261430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efc9c004c80 con 0x7efca4198fd0 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca41a4230 con 0x7efca4198fd0 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efca17fa640 1 -- 172.21.15.42:0/381261430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c010690 con 0x7efca4198fd0 2024-03-12T04:25:21.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.721+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca41a4740 con 0x7efca4198fd0 2024-03-12T04:25:21.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.723+0000 7efca17fa640 1 -- 172.21.15.42:0/381261430 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7efc9c005020 con 0x7efca4198fd0 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 Processor -- start 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 -- start start 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da10630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da19210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 0x56235da1bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235d9cb960 con 0x56235da10250 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235d97f310 con 0x56235da19750 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efcad428740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235d97c840 con 0x56235da10b70 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca17fa640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7efc8c0a1c20 0x7efc8c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da10630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da10630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40522/0 (socket says 172.21.15.42:40522) 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 -- 172.21.15.42:0/2569601876 learned_addr learned my addr 172.21.15.42:0/2569601876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca8deb640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7efc8c0a1c20 0x7efc8c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca17fa640 1 -- 172.21.15.42:0/381261430 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efc9c014070 con 0x7efca4198fd0 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca3fff640 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da19210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 -- 172.21.15.42:0/2569601876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 msgr2=0x56235da19210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da19210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 -- 172.21.15.42:0/2569601876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 msgr2=0x56235da1bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 0x56235da1bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.724+0000 7efca37fe640 1 -- 172.21.15.42:0/2569601876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56235d93a060 con 0x56235da10250 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.725+0000 7efca37fe640 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da10630 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7efc9800ab20 tx=0x7efc9800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3341323d2d8e7481 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.725+0000 7efca8deb640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7efc8c0a1c20 0x7efc8c0a40e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc9c09a770 tx=0x7efc9c00efd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.725+0000 7efc8a7fc640 1 -- 172.21.15.42:0/2569601876 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98005020 con 0x56235da10250 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.725+0000 7efc8a7fc640 1 -- 172.21.15.42:0/2569601876 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efc98005630 con 0x56235da10250 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.725+0000 7efc8a7fc640 1 -- 172.21.15.42:0/2569601876 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98010600 con 0x56235da10250 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.725+0000 7efcad428740 1 -- 172.21.15.42:0/2569601876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 msgr2=0x56235da10630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.725+0000 7efcad428740 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da10630 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7efc9800ab20 tx=0x7efc9800e2f0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- 172.21.15.42:0/2569601876 shutdown_connections 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 0x56235da1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da19210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 --2- 172.21.15.42:0/2569601876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da10630 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- 172.21.15.42:0/2569601876 >> 172.21.15.42:0/2569601876 conn(0x56235da0a980 msgr2=0x56235da0ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- 172.21.15.42:0/2569601876 shutdown_connections 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- 172.21.15.42:0/2569601876 wait complete. 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- start start 2024-03-12T04:25:21.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 Processor -- start 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da16700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da16c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 0x56235da17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235d978020 con 0x56235da10250 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235d64b430 con 0x56235da19750 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235d607e00 con 0x56235da10b70 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efcad428740 10 client.0 _ll_register_callbacks cb 0x56235d91fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da16700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.726+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da16700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40534/0 (socket says 172.21.15.42:40534) 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 -- 172.21.15.42:0/249716435 learned_addr learned my addr 172.21.15.42:0/249716435 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca3fff640 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da16c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 0x56235da17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 -- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 msgr2=0x56235da16c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da16c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 -- 172.21.15.42:0/249716435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 msgr2=0x56235da17180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 0x56235da17180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 -- 172.21.15.42:0/249716435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9800a7d0 con 0x56235da10250 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efca37fe640 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da16700 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7efc9800a2a0 tx=0x7efc9800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50650e14736be87d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efc837fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efc837fe640 1 -- 172.21.15.42:0/249716435 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98028430 con 0x56235da10250 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efc837fe640 1 -- 172.21.15.42:0/249716435 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efc98028590 con 0x56235da10250 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56235d6777e0 con 0x56235da10250 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efc837fe640 1 -- 172.21.15.42:0/249716435 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc98028750 con 0x56235da10250 2024-03-12T04:25:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efcad428740 10 client.18339 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56235d9766a0 con 0x56235da10250 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.727+0000 7efc837fe640 1 -- 172.21.15.42:0/249716435 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56235d6777e0 con 0x56235da10250 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efc837fe640 1 -- 172.21.15.42:0/249716435 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efc98016030 con 0x56235da10250 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efcad428740 10 client.18339 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efcad428740 10 client.18339 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efcad428740 20 client.18339 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efcad428740 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da299d0 0x56235da2bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efcad428740 4 client.18339 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56235d6b4720 con 0x56235da299d0 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efca3fff640 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da299d0 0x56235da2bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efc837fe640 10 client.18339 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:25:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.728+0000 7efca3fff640 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da299d0 0x56235da2bdc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86f935f60aa8d11a server_cookie=a9047e50a7f26f54 in_seq=0 out_seq=0 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efc837fe640 1 -- 172.21.15.42:0/249716435 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56235d6b4720 con 0x56235da299d0 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efc837fe640 10 client.18339 handle_command_reply: tid=0 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 client.18339 shutdown 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 msgr2=0x56235da16700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da16700 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7efc9800a2a0 tx=0x7efc9800a710 comp rx=0 tx=0).stop 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 shutdown_connections 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da299d0 0x56235da2bdc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da19750 0x56235da17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efc837fe640 0 client.18339 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da16c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 --2- 172.21.15.42:0/249716435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da10250 0x56235da16700 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 >> 172.21.15.42:0/249716435 conn(0x56235da0a980 msgr2=0x56235da0c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 shutdown_connections 2024-03-12T04:25:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 1 -- 172.21.15.42:0/249716435 wait complete. 2024-03-12T04:25:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.729+0000 7efcad428740 20 client.18339 trim_cache size 0 max 16384 2024-03-12T04:25:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 Processor -- start 2024-03-12T04:25:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 -- start start 2024-03-12T04:25:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10250 0x56235da18e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da13960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da13ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235d8fd130 con 0x56235da19750 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235da1c110 con 0x56235da10250 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efcad428740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235da265e0 con 0x56235da10b70 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da13ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da13ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40550/0 (socket says 172.21.15.42:40550) 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.741+0000 7efca37fe640 1 -- 172.21.15.42:0/1175840630 learned_addr learned my addr 172.21.15.42:0/1175840630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efca37fe640 1 -- 172.21.15.42:0/1175840630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 msgr2=0x56235da13960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efca37fe640 1 --2- 172.21.15.42:0/1175840630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da13960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efca37fe640 1 -- 172.21.15.42:0/1175840630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10250 msgr2=0x56235da18e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efca37fe640 1 --2- 172.21.15.42:0/1175840630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10250 0x56235da18e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efca37fe640 1 -- 172.21.15.42:0/1175840630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56235d912c20 con 0x56235da19750 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efca37fe640 1 --2- 172.21.15.42:0/1175840630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da13ea0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7efc98001eb0 tx=0x7efc98004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91393c8044be170 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efc89ffb640 1 -- 172.21.15.42:0/1175840630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98036070 con 0x56235da19750 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efc89ffb640 1 -- 172.21.15.42:0/1175840630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efc98004e50 con 0x56235da19750 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efc89ffb640 1 -- 172.21.15.42:0/1175840630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc980054b0 con 0x56235da19750 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efcad428740 1 -- 172.21.15.42:0/1175840630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 msgr2=0x56235da13ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efcad428740 1 --2- 172.21.15.42:0/1175840630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da13ea0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7efc98001eb0 tx=0x7efc98004be0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efcad428740 1 -- 172.21.15.42:0/1175840630 shutdown_connections 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efcad428740 1 --2- 172.21.15.42:0/1175840630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da13ea0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efcad428740 1 --2- 172.21.15.42:0/1175840630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10b70 0x56235da13960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efcad428740 1 --2- 172.21.15.42:0/1175840630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10250 0x56235da18e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.742+0000 7efcad428740 1 -- 172.21.15.42:0/1175840630 >> 172.21.15.42:0/1175840630 conn(0x56235da0a980 msgr2=0x56235da119f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 -- 172.21.15.42:0/1175840630 shutdown_connections 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 -- 172.21.15.42:0/1175840630 wait complete. 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 -- start start 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 Processor -- start 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10250 0x56235da20b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10b70 0x56235da1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235da1fcf0 con 0x56235da19750 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235da7feb0 con 0x56235da10b70 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235da7fff0 con 0x56235da10250 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efcad428740 10 client.0 _ll_register_callbacks cb 0x7efc9800a2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efca37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da1f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40560/0 (socket says 172.21.15.42:40560) 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efca37fe640 1 -- 172.21.15.42:0/2846608836 learned_addr learned my addr 172.21.15.42:0/2846608836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.743+0000 7efca3fff640 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10b70 0x56235da1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efca37fe640 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10250 0x56235da20b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efca37fe640 1 -- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10250 msgr2=0x56235da20b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efca37fe640 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10250 0x56235da20b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efca37fe640 1 -- 172.21.15.42:0/2846608836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10b70 msgr2=0x56235da1eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efca37fe640 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10b70 0x56235da1eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efca37fe640 1 -- 172.21.15.42:0/2846608836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56235d9766a0 con 0x56235da19750 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efca37fe640 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da1f5f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7efc9800a4f0 tx=0x7efc98033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8316815e6b8207 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efc837fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efc837fe640 1 -- 172.21.15.42:0/2846608836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98040070 con 0x56235da19750 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efc837fe640 1 -- 172.21.15.42:0/2846608836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efc98000e40 con 0x56235da19750 2024-03-12T04:25:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc98005630 con 0x56235da19750 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efc837fe640 1 -- 172.21.15.42:0/2846608836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc98036040 con 0x56235da19750 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efcad428740 10 client.18345 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56235d9388c0 con 0x56235da19750 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.744+0000 7efc837fe640 1 -- 172.21.15.42:0/2846608836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efc98005630 con 0x56235da19750 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efc837fe640 1 -- 172.21.15.42:0/2846608836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efc98043020 con 0x56235da19750 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efcad428740 10 client.18345 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efcad428740 10 client.18345 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efcad428740 20 client.18345 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efcad428740 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da81320 0x56235da83740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efcad428740 4 client.18345 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 --> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56235d703890 con 0x56235da81320 2024-03-12T04:25:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efca3fff640 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da81320 0x56235da83740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efc837fe640 10 client.18345 ms_handle_connect on v2:172.21.15.136:6836/23397667 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efca3fff640 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da81320 0x56235da83740 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=560f6019bea07464 server_cookie=87cd4305caa386ee in_seq=0 out_seq=0 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efc837fe640 1 -- 172.21.15.42:0/2846608836 <== mds.3 v2:172.21.15.136:6836/23397667 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56235d703890 con 0x56235da81320 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efc837fe640 10 client.18345 handle_command_reply: tid=0 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.745+0000 7efcad428740 1 client.18345 shutdown 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 msgr2=0x56235da1f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da1f5f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7efc9800a4f0 tx=0x7efc98033040 comp rx=0 tx=0).stop 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 shutdown_connections 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:6836/23397667,v1:172.21.15.136:6837/23397667] conn(0x56235da81320 0x56235da83740 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235da19750 0x56235da1f5f0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56235da10b70 0x56235da1eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efc837fe640 0 client.18345 ms_handle_reset on v2:172.21.15.136:6836/23397667 2024-03-12T04:25:21.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 --2- 172.21.15.42:0/2846608836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56235da10250 0x56235da20b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 >> 172.21.15.42:0/2846608836 conn(0x56235da0a980 msgr2=0x56235da092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 shutdown_connections 2024-03-12T04:25:21.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 1 -- 172.21.15.42:0/2846608836 wait complete. 2024-03-12T04:25:21.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcad428740 20 client.18345 trim_cache size 0 max 16384 2024-03-12T04:25:21.756 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:25:21.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7efc8c0a1c20 msgr2=0x7efc8c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7efc8c0a1c20 0x7efc8c0a40e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc9c09a770 tx=0x7efc9c00efd0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4198fd0 msgr2=0x7efca41a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4198fd0 0x7efca41a3580 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00eb10 tx=0x7efc9c00c2d0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 shutdown_connections 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7efc8c0a1c20 0x7efc8c0a40e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4198fd0 0x7efca41a3580 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca4105330 0x7efca4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 --2- 172.21.15.42:0/381261430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca4100260 0x7efca4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 >> 172.21.15.42:0/381261430 conn(0x7efca4077180 msgr2=0x7efca40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 shutdown_connections 2024-03-12T04:25:21.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.746+0000 7efcaa84e640 1 -- 172.21.15.42:0/381261430 wait complete. 2024-03-12T04:25:21.760 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:25:21.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T04:25:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.861+0000 7fda1433a640 1 -- 172.21.15.42:0/3760535613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c10b7c0 msgr2=0x7fda0c10dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.861+0000 7fda1433a640 1 --2- 172.21.15.42:0/3760535613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c10b7c0 0x7fda0c10dc30 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fda08009ca0 tx=0x7fda0802c180 comp rx=0 tx=0).stop 2024-03-12T04:25:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.861+0000 7fda1433a640 1 -- 172.21.15.42:0/3760535613 shutdown_connections 2024-03-12T04:25:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.861+0000 7fda1433a640 1 --2- 172.21.15.42:0/3760535613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c10b7c0 0x7fda0c10dc30 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.861+0000 7fda1433a640 1 --2- 172.21.15.42:0/3760535613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda0c102d10 0x7fda0c10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.861+0000 7fda1433a640 1 --2- 172.21.15.42:0/3760535613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda0c1023f0 0x7fda0c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.861+0000 7fda1433a640 1 -- 172.21.15.42:0/3760535613 >> 172.21.15.42:0/3760535613 conn(0x7fda0c0771e0 msgr2=0x7fda0c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 -- 172.21.15.42:0/3760535613 shutdown_connections 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 -- 172.21.15.42:0/3760535613 wait complete. 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 Processor -- start 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 -- start start 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1023f0 0x7fda0c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda0c102d10 0x7fda0c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda0c10b7c0 0x7fda0c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c19c500 con 0x7fda0c1023f0 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c19c670 con 0x7fda0c10b7c0 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda1433a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c19c830 con 0x7fda0c102d10 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda120d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1023f0 0x7fda0c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda120d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1023f0 0x7fda0c197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40582/0 (socket says 172.21.15.42:40582) 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda120d6640 1 -- 172.21.15.42:0/3096497924 learned_addr learned my addr 172.21.15.42:0/3096497924 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.862+0000 7fda118d5640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda0c102d10 0x7fda0c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda128d7640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda0c10b7c0 0x7fda0c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda120d6640 1 -- 172.21.15.42:0/3096497924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda0c102d10 msgr2=0x7fda0c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda120d6640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda0c102d10 0x7fda0c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda120d6640 1 -- 172.21.15.42:0/3096497924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda0c10b7c0 msgr2=0x7fda0c19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda120d6640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda0c10b7c0 0x7fda0c19bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda120d6640 1 -- 172.21.15.42:0/3096497924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08009900 con 0x7fda0c1023f0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda120d6640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1023f0 0x7fda0c197640 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e9f0 tx=0x7fda0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda037fe640 1 -- 172.21.15.42:0/3096497924 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04009720 con 0x7fda0c1023f0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda037fe640 1 -- 172.21.15.42:0/3096497924 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fda040054d0 con 0x7fda0c1023f0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c19cb10 con 0x7fda0c1023f0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda037fe640 1 -- 172.21.15.42:0/3096497924 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04010430 con 0x7fda0c1023f0 2024-03-12T04:25:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.863+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c1a4390 con 0x7fda0c1023f0 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.866+0000 7fda037fe640 1 -- 172.21.15.42:0/3096497924 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fda04004fb0 con 0x7fda0c1023f0 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.866+0000 7fda16fe3740 1 Processor -- start 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.866+0000 7fda16fe3740 1 -- start start 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.866+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f2d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f2db70 0x55f7a9f36210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f36750 0x55f7a9f38b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda16fe3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9ee8960 con 0x55f7a9f2d250 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda16fe3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9e9c310 con 0x55f7a9f2db70 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda16fe3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9e99840 con 0x55f7a9f36750 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f2d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f2d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40598/0 (socket says 172.21.15.42:40598) 2024-03-12T04:25:21.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 -- 172.21.15.42:0/1921235859 learned_addr learned my addr 172.21.15.42:0/1921235859 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda037fe640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9f80a1ba0 0x7fd9f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda120d6640 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f2db70 0x55f7a9f36210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda118d5640 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f36750 0x55f7a9f38b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9f80a1ba0 0x7fd9f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda037fe640 1 -- 172.21.15.42:0/3096497924 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda04014070 con 0x7fda0c1023f0 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 -- 172.21.15.42:0/1921235859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f36750 msgr2=0x55f7a9f38b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f36750 0x55f7a9f38b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 -- 172.21.15.42:0/1921235859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f2db70 msgr2=0x55f7a9f36210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f2db70 0x55f7a9f36210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 -- 172.21.15.42:0/1921235859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7a9e57060 con 0x55f7a9f2d250 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.867+0000 7fda128d7640 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f2d630 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fda080098d0 tx=0x7fda08034e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13cf659cdb2bb8b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda00bfe640 1 -- 172.21.15.42:0/1921235859 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080346a0 con 0x55f7a9f2d250 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda00bfe640 1 -- 172.21.15.42:0/1921235859 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fda08034800 con 0x55f7a9f2d250 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda00bfe640 1 -- 172.21.15.42:0/1921235859 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080042a0 con 0x55f7a9f2d250 2024-03-12T04:25:21.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda16fe3740 1 -- 172.21.15.42:0/1921235859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 msgr2=0x55f7a9f2d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f2d630 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fda080098d0 tx=0x7fda08034e70 comp rx=0 tx=0).stop 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda128d7640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9f80a1ba0 0x7fd9f80a4060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda08005e30 tx=0x7fda0804b000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda16fe3740 1 -- 172.21.15.42:0/1921235859 shutdown_connections 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f36750 0x55f7a9f38b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f2db70 0x55f7a9f36210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1921235859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f2d630 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.868+0000 7fda16fe3740 1 -- 172.21.15.42:0/1921235859 >> 172.21.15.42:0/1921235859 conn(0x55f7a9f27980 msgr2=0x55f7a9f27d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 -- 172.21.15.42:0/1921235859 shutdown_connections 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 -- 172.21.15.42:0/1921235859 wait complete. 2024-03-12T04:25:21.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 -- start start 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 Processor -- start 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9e95020 con 0x55f7a9f2db70 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9b68430 con 0x55f7a9f36750 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9b24e00 con 0x55f7a9f2d250 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda16fe3740 10 client.0 _ll_register_callbacks cb 0x55f7a9e3ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda118d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda118d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40604/0 (socket says 172.21.15.42:40604) 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda118d5640 1 -- 172.21.15.42:0/2201422666 learned_addr learned my addr 172.21.15.42:0/2201422666 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda128d7640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.869+0000 7fda120d6640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda118d5640 1 -- 172.21.15.42:0/2201422666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 msgr2=0x55f7a9f33740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda118d5640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f33740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda118d5640 1 -- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 msgr2=0x55f7a9f341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda118d5640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda118d5640 1 -- 172.21.15.42:0/2201422666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c10e8b0 con 0x55f7a9f2db70 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda128d7640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda120d6640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda118d5640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc010820 tx=0x7fd9fc010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89591560c9b2e7d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fd9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/2201422666 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc015070 con 0x55f7a9f2db70 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/2201422666 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd9fc00fb90 con 0x55f7a9f2db70 2024-03-12T04:25:21.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7a9b947e0 con 0x55f7a9f2db70 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/2201422666 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9fc00fd90 con 0x55f7a9f2db70 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 10 client.18357 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f7a9e936a0 con 0x55f7a9f2db70 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/2201422666 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f7a9b947e0 con 0x55f7a9f2db70 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/2201422666 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd9fc00e440 con 0x55f7a9f2db70 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 10 client.18357 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 10 client.18357 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 20 client.18357 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9f48ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 4 client.18357 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:25:21.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.870+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7a9bd1720 con 0x55f7a9f46bb0 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.871+0000 7fda120d6640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9f48ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.871+0000 7fd9f5ffb640 10 client.18357 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.871+0000 7fda120d6640 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9f48ff0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6b8f0bbd4a9477f server_cookie=e8bd28ff92ac5939 in_seq=0 out_seq=0 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/2201422666 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7a9bd1720 con 0x55f7a9f46bb0 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fd9f5ffb640 10 client.18357 handle_command_reply: tid=0 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 client.18357 shutdown 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 msgr2=0x55f7a9f33c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc010820 tx=0x7fd9fc010ce0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 shutdown_connections 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9f48ff0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33c80 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fd9f5ffb640 0 client.18357 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:25:21.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2201422666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 >> 172.21.15.42:0/2201422666 conn(0x55f7a9f27980 msgr2=0x55f7a9f296d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 shutdown_connections 2024-03-12T04:25:21.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 1 -- 172.21.15.42:0/2201422666 wait complete. 2024-03-12T04:25:21.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.872+0000 7fda16fe3740 20 client.18357 trim_cache size 0 max 16384 2024-03-12T04:25:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 Processor -- start 2024-03-12T04:25:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 -- start start 2024-03-12T04:25:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9e1a130 con 0x55f7a9f2db70 2024-03-12T04:25:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9f2f340 con 0x55f7a9f36750 2024-03-12T04:25:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda16fe3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9f28a40 con 0x55f7a9f2d250 2024-03-12T04:25:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda118d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda118d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40608/0 (socket says 172.21.15.42:40608) 2024-03-12T04:25:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda118d5640 1 -- 172.21.15.42:0/2098939169 learned_addr learned my addr 172.21.15.42:0/2098939169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda120d6640 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda118d5640 1 -- 172.21.15.42:0/2098939169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 msgr2=0x55f7a9f35e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda118d5640 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f35e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda128d7640 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda118d5640 1 -- 172.21.15.42:0/2098939169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 msgr2=0x55f7a9f309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.884+0000 7fda118d5640 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda118d5640 1 -- 172.21.15.42:0/2098939169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7a9e2fc20 con 0x55f7a9f2db70 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda120d6640 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f35e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda128d7640 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda118d5640 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33fe0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e770 tx=0x7fd9fc005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83ab5b7cbf8795a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fd9f7fff640 1 -- 172.21.15.42:0/2098939169 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc015070 con 0x55f7a9f2db70 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fd9f7fff640 1 -- 172.21.15.42:0/2098939169 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd9fc016040 con 0x55f7a9f2db70 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fd9f7fff640 1 -- 172.21.15.42:0/2098939169 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0226a0 con 0x55f7a9f2db70 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 -- 172.21.15.42:0/2098939169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 msgr2=0x55f7a9f33fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33fe0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e770 tx=0x7fd9fc005670 comp rx=0 tx=0).stop 2024-03-12T04:25:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 -- 172.21.15.42:0/2098939169 shutdown_connections 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2db70 0x55f7a9f33fe0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 --2- 172.21.15.42:0/2098939169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2d250 0x55f7a9f35e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 -- 172.21.15.42:0/2098939169 >> 172.21.15.42:0/2098939169 conn(0x55f7a9f27980 msgr2=0x55f7a9f32010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 -- 172.21.15.42:0/2098939169 shutdown_connections 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.885+0000 7fda16fe3740 1 -- 172.21.15.42:0/2098939169 wait complete. 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 -- start start 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 Processor -- start 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f3c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2db70 0x55f7a9f3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f3cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9f9d120 con 0x55f7a9f2d250 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9f9d290 con 0x55f7a9f36750 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a9f9d450 con 0x55f7a9f2db70 2024-03-12T04:25:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda16fe3740 10 client.0 _ll_register_callbacks cb 0x7fda08037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f3c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f3c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40612/0 (socket says 172.21.15.42:40612) 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 -- 172.21.15.42:0/1130410414 learned_addr learned my addr 172.21.15.42:0/1130410414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda118d5640 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2db70 0x55f7a9f3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda128d7640 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f3cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 -- 172.21.15.42:0/1130410414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2db70 msgr2=0x55f7a9f3c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2db70 0x55f7a9f3c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 -- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 msgr2=0x55f7a9f3cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f3cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.886+0000 7fda120d6640 1 -- 172.21.15.42:0/1130410414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7a9e936a0 con 0x55f7a9f2d250 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda120d6640 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f3c0f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fda040cb480 tx=0x7fda040c4f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08b4071ca3ebfd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fd9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/1130410414 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040a3050 con 0x55f7a9f2d250 2024-03-12T04:25:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/1130410414 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda04005260 con 0x55f7a9f2d250 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda08034800 con 0x55f7a9f2d250 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/1130410414 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda04098bb0 con 0x55f7a9f2d250 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 10 client.18363 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f7a9e558c0 con 0x55f7a9f2d250 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/1130410414 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fda08034800 con 0x55f7a9f2d250 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/1130410414 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fda0401f060 con 0x55f7a9f2d250 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 10 client.18363 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 10 client.18363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 20 client.18363 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9fa0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 4 client.18363 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 --> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7a9c20890 con 0x55f7a9f46bb0 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.887+0000 7fda118d5640 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9fa0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda118d5640 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9fa0bf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91223fe6fa9c0e30 server_cookie=c255f8b52b21db26 in_seq=0 out_seq=0 2024-03-12T04:25:21.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fd9f5ffb640 10 client.18363 ms_handle_connect on v2:172.21.15.119:6836/3839383373 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fd9f5ffb640 1 -- 172.21.15.42:0/1130410414 <== mds.2 v2:172.21.15.119:6836/3839383373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7a9c20890 con 0x55f7a9f46bb0 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fd9f5ffb640 10 client.18363 handle_command_reply: tid=0 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 client.18363 shutdown 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 msgr2=0x55f7a9f3c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f3c0f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fda040cb480 tx=0x7fda040c4f70 comp rx=0 tx=0).stop 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 shutdown_connections 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:6836/3839383373,v1:172.21.15.119:6837/3839383373] conn(0x55f7a9f46bb0 0x55f7a9fa0bf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7a9f36750 0x55f7a9f3cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fd9f5ffb640 0 client.18363 ms_handle_reset on v2:172.21.15.119:6836/3839383373 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7a9f2db70 0x55f7a9f3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 --2- 172.21.15.42:0/1130410414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7a9f2d250 0x55f7a9f3c0f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 >> 172.21.15.42:0/1130410414 conn(0x55f7a9f27980 msgr2=0x55f7a9f262f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 shutdown_connections 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 1 -- 172.21.15.42:0/1130410414 wait complete. 2024-03-12T04:25:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.888+0000 7fda16fe3740 20 client.18363 trim_cache size 0 max 16384 2024-03-12T04:25:21.902 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:25:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9f80a1ba0 msgr2=0x7fd9f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9f80a1ba0 0x7fd9f80a4060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda08005e30 tx=0x7fda0804b000 comp rx=0 tx=0).stop 2024-03-12T04:25:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1023f0 msgr2=0x7fda0c197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1023f0 0x7fda0c197640 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e9f0 tx=0x7fda0400edf0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 shutdown_connections 2024-03-12T04:25:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fd9f80a1ba0 0x7fd9f80a4060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda0c10b7c0 0x7fda0c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda0c102d10 0x7fda0c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 --2- 172.21.15.42:0/3096497924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1023f0 0x7fda0c197640 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:21.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 >> 172.21.15.42:0/3096497924 conn(0x7fda0c0771e0 msgr2=0x7fda0c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:21.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 shutdown_connections 2024-03-12T04:25:21.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:21.889+0000 7fda1433a640 1 -- 172.21.15.42:0/3096497924 wait complete. 2024-03-12T04:25:21.904 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:25:21.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T04:25:22.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 -- 172.21.15.42:0/1390233148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 msgr2=0x7f99a0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 --2- 172.21.15.42:0/1390233148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a0101760 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f999c009930 tx=0x7f999c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 -- 172.21.15.42:0/1390233148 shutdown_connections 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 --2- 172.21.15.42:0/1390233148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0106450 0x7f99a010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 --2- 172.21.15.42:0/1390233148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a0101d30 0x7f99a0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 --2- 172.21.15.42:0/1390233148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a0101760 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 -- 172.21.15.42:0/1390233148 >> 172.21.15.42:0/1390233148 conn(0x7f99a0078580 msgr2=0x7f99a0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 -- 172.21.15.42:0/1390233148 shutdown_connections 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.006+0000 7f99a84c0640 1 -- 172.21.15.42:0/1390233148 wait complete. 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 Processor -- start 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 -- start start 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0106450 0x7f99a019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a01975a0 0x7f99a0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0197f90 con 0x7f99a0101380 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0198100 con 0x7f99a01975a0 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a84c0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a01982c0 con 0x7f99a0106450 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40630/0 (socket says 172.21.15.42:40630) 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 -- 172.21.15.42:0/353695054 learned_addr learned my addr 172.21.15.42:0/353695054 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0106450 0x7f99a019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 -- 172.21.15.42:0/353695054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0106450 msgr2=0x7f99a019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0106450 0x7f99a019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 -- 172.21.15.42:0/353695054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a01975a0 msgr2=0x7f99a0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a01975a0 0x7f99a0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a01975a0 0x7f99a0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.007+0000 7f99a625c640 1 -- 172.21.15.42:0/353695054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c009560 con 0x7f99a0101380 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.008+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a01975a0 0x7f99a0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:22.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.008+0000 7f99a625c640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a019d3f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f999c009230 tx=0x7f999c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.008+0000 7f99977fe640 1 -- 172.21.15.42:0/353695054 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c0044c0 con 0x7f99a0101380 2024-03-12T04:25:22.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.008+0000 7f99977fe640 1 -- 172.21.15.42:0/353695054 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f999c02de90 con 0x7f99a0101380 2024-03-12T04:25:22.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.008+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a01a42b0 con 0x7f99a0101380 2024-03-12T04:25:22.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.008+0000 7f99977fe640 1 -- 172.21.15.42:0/353695054 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c005400 con 0x7f99a0101380 2024-03-12T04:25:22.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.008+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a01a4760 con 0x7f99a0101380 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.010+0000 7f99977fe640 1 -- 172.21.15.42:0/353695054 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f999c005560 con 0x7f99a0101380 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 Processor -- start 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 -- start start 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0250 0x55b2f55f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0b70 0x55b2f55f9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f55ab960 con 0x55b2f55f0250 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f555f310 con 0x55b2f55f9750 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99ab16d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f555c840 con 0x55b2f55f0b70 2024-03-12T04:25:22.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99977fe640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f998c0a1ba0 0x7f998c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99a6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0250 0x55b2f55f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99a6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0250 0x55b2f55f0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40640/0 (socket says 172.21.15.42:40640) 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99a6a5d640 1 -- 172.21.15.42:0/1159154729 learned_addr learned my addr 172.21.15.42:0/1159154729 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f998c0a1ba0 0x7f998c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99977fe640 1 -- 172.21.15.42:0/353695054 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f999c0ef860 con 0x7f99a0101380 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99a625c640 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0b70 0x55b2f55f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.011+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f99a6a5d640 1 -- 172.21.15.42:0/1159154729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0b70 msgr2=0x55b2f55f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0b70 0x55b2f55f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f99a6a5d640 1 -- 172.21.15.42:0/1159154729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 msgr2=0x55b2f55fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55fbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f99a6a5d640 1 -- 172.21.15.42:0/1159154729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f551a060 con 0x55b2f55f0250 2024-03-12T04:25:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0250 0x55b2f55f0630 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f999800ed70 tx=0x7f999800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ac7829ad589621 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f998c0a1ba0 0x7f998c0a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f999800cc40 tx=0x7f9998019000 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f9994bfe640 1 -- 172.21.15.42:0/1159154729 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998018b70 con 0x55b2f55f0250 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f9994bfe640 1 -- 172.21.15.42:0/1159154729 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9998018cd0 con 0x55b2f55f0250 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.012+0000 7f9994bfe640 1 -- 172.21.15.42:0/1159154729 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999801aea0 con 0x55b2f55f0250 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- 172.21.15.42:0/1159154729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0250 msgr2=0x55b2f55f0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0250 0x55b2f55f0630 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f999800ed70 tx=0x7f999800c740 comp rx=0 tx=0).stop 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- 172.21.15.42:0/1159154729 shutdown_connections 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0b70 0x55b2f55f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 --2- 172.21.15.42:0/1159154729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0250 0x55b2f55f0630 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- 172.21.15.42:0/1159154729 >> 172.21.15.42:0/1159154729 conn(0x55b2f55ea980 msgr2=0x55b2f55ead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- 172.21.15.42:0/1159154729 shutdown_connections 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- 172.21.15.42:0/1159154729 wait complete. 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- start start 2024-03-12T04:25:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 Processor -- start 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f5558020 con 0x55b2f55f9750 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f522b430 con 0x55b2f55f0b70 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f51e7e00 con 0x55b2f55f0250 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.013+0000 7f99ab16d740 10 client.0 _ll_register_callbacks cb 0x55b2f54ffcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40656/0 (socket says 172.21.15.42:40656) 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 -- 172.21.15.42:0/2230688954 learned_addr learned my addr 172.21.15.42:0/2230688954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a625c640 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 -- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 msgr2=0x55b2f55f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 -- 172.21.15.42:0/2230688954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 msgr2=0x55b2f55f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 -- 172.21.15.42:0/2230688954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9998016040 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f7210 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9998004a80 tx=0x7f999801adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b02c6db3aca1fe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f9989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f9989ffb640 1 -- 172.21.15.42:0/2230688954 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998018b40 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f9989ffb640 1 -- 172.21.15.42:0/2230688954 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9998018e40 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2f52577e0 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f9989ffb640 1 -- 172.21.15.42:0/2230688954 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999801a630 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99ab16d740 10 client.18375 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2f55566a0 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.014+0000 7f9989ffb640 1 -- 172.21.15.42:0/2230688954 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b2f52577e0 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f9989ffb640 1 -- 172.21.15.42:0/2230688954 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9998009030 con 0x55b2f55f9750 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99ab16d740 10 client.18375 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99ab16d740 10 client.18375 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99ab16d740 20 client.18375 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99ab16d740 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5609ae0 0x55b2f560bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99ab16d740 4 client.18375 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2f5294720 con 0x55b2f5609ae0 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99a625c640 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5609ae0 0x55b2f560bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f99a625c640 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5609ae0 0x55b2f560bf00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16a8fd8a8697826e server_cookie=1bde2de43fb58fca in_seq=0 out_seq=0 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.015+0000 7f9989ffb640 10 client.18375 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f9989ffb640 1 -- 172.21.15.42:0/2230688954 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2f5294720 con 0x55b2f5609ae0 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f9989ffb640 10 client.18375 handle_command_reply: tid=0 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 client.18375 shutdown 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 msgr2=0x55b2f55f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f7210 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9998004a80 tx=0x7f999801adf0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 shutdown_connections 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5609ae0 0x55b2f560bf00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f7210 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 --2- 172.21.15.42:0/2230688954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f9989ffb640 0 client.18375 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:25:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 >> 172.21.15.42:0/2230688954 conn(0x55b2f55ea980 msgr2=0x55b2f55ec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 shutdown_connections 2024-03-12T04:25:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.016+0000 7f99ab16d740 1 -- 172.21.15.42:0/2230688954 wait complete. 2024-03-12T04:25:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.017+0000 7f99ab16d740 20 client.18375 trim_cache size 0 max 16384 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 Processor -- start 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 -- start start 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f54dd130 con 0x55b2f55f9750 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f5606620 con 0x55b2f55f0b70 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.028+0000 7f99ab16d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f55eb980 con 0x55b2f55f0250 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40660/0 (socket says 172.21.15.42:40660) 2024-03-12T04:25:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 -- 172.21.15.42:0/3253082108 learned_addr learned my addr 172.21.15.42:0/3253082108 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a625c640 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 -- 172.21.15.42:0/3253082108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 msgr2=0x55b2f55f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 -- 172.21.15.42:0/3253082108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 msgr2=0x55b2f55f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 -- 172.21.15.42:0/3253082108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f54f2c20 con 0x55b2f55f9750 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f3a10 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9998009780 tx=0x7f99980092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a959b53b5bae909a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f998bfff640 1 -- 172.21.15.42:0/3253082108 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998009820 con 0x55b2f55f9750 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f998bfff640 1 -- 172.21.15.42:0/3253082108 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f999801ae60 con 0x55b2f55f9750 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.029+0000 7f998bfff640 1 -- 172.21.15.42:0/3253082108 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998036a20 con 0x55b2f55f9750 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- 172.21.15.42:0/3253082108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 msgr2=0x55b2f55f3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f3a10 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9998009780 tx=0x7f99980092b0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- 172.21.15.42:0/3253082108 shutdown_connections 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f9750 0x55b2f55f3a10 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f0b70 0x55b2f55f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3253082108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f55f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- 172.21.15.42:0/3253082108 >> 172.21.15.42:0/3253082108 conn(0x55b2f55ea980 msgr2=0x55b2f5610860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- 172.21.15.42:0/3253082108 shutdown_connections 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- 172.21.15.42:0/3253082108 wait complete. 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- start start 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 Processor -- start 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f5600830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0b70 0x55b2f55fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f55ff6d0 con 0x55b2f55f0b70 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f55ff840 con 0x55b2f55f9750 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f55ffa00 con 0x55b2f55f0250 2024-03-12T04:25:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99ab16d740 10 client.0 _ll_register_callbacks cb 0x7f999800e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99a5a5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0b70 0x55b2f55fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99a5a5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0b70 0x55b2f55fec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40676/0 (socket says 172.21.15.42:40676) 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99a5a5b640 1 -- 172.21.15.42:0/3835802351 learned_addr learned my addr 172.21.15.42:0/3835802351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.030+0000 7f99a625c640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f5600830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a5a5b640 1 -- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 msgr2=0x55b2f5600830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f5600830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a5a5b640 1 -- 172.21.15.42:0/3835802351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 msgr2=0x55b2f55ff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55ff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a5a5b640 1 -- 172.21.15.42:0/3835802351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2f55566a0 con 0x55b2f55f0b70 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a6a5d640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99a5a5b640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0b70 0x55b2f55fec00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f99a01986e0 tx=0x7f9990013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d454269670b0bb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f9989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f9989ffb640 1 -- 172.21.15.42:0/3835802351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990018070 con 0x55b2f55f0b70 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9998018cd0 con 0x55b2f55f0b70 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f9989ffb640 1 -- 172.21.15.42:0/3835802351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9990004240 con 0x55b2f55f0b70 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f9989ffb640 1 -- 172.21.15.42:0/3835802351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9990004400 con 0x55b2f55f0b70 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 10 client.18381 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2f55188c0 con 0x55b2f55f0b70 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f9989ffb640 1 -- 172.21.15.42:0/3835802351 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9998018cd0 con 0x55b2f55f0b70 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f9989ffb640 1 -- 172.21.15.42:0/3835802351 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f999001d750 con 0x55b2f55f0b70 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 10 client.18381 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 10 client.18381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 20 client.18381 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5661650 0x55b2f5663a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 4 client.18381 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.031+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 --> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b2f52e3890 con 0x55b2f5661650 2024-03-12T04:25:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f99a625c640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5661650 0x55b2f5663a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f99a625c640 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5661650 0x55b2f5663a70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4626a0b30b675b4c server_cookie=29ec62cf14e52b23 in_seq=0 out_seq=0 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f9989ffb640 10 client.18381 ms_handle_connect on v2:172.21.15.136:6838/3211189908 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f9989ffb640 1 -- 172.21.15.42:0/3835802351 <== mds.1 v2:172.21.15.136:6838/3211189908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b2f52e3890 con 0x55b2f5661650 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f9989ffb640 10 client.18381 handle_command_reply: tid=0 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f99ab16d740 1 client.18381 shutdown 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0b70 msgr2=0x55b2f55fec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.032+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0b70 0x55b2f55fec00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f99a01986e0 tx=0x7f9990013830 comp rx=0 tx=0).stop 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 shutdown_connections 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:6838/3211189908,v1:172.21.15.136:6839/3211189908] conn(0x55b2f5661650 0x55b2f5663a70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2f55f9750 0x55b2f55ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2f55f0b70 0x55b2f55fec00 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f9989ffb640 0 client.18381 ms_handle_reset on v2:172.21.15.136:6838/3211189908 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 --2- 172.21.15.42:0/3835802351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2f55f0250 0x55b2f5600830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 >> 172.21.15.42:0/3835802351 conn(0x55b2f55ea980 msgr2=0x55b2f55e92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 shutdown_connections 2024-03-12T04:25:22.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 1 -- 172.21.15.42:0/3835802351 wait complete. 2024-03-12T04:25:22.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99ab16d740 20 client.18381 trim_cache size 0 max 16384 2024-03-12T04:25:22.043 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f998c0a1ba0 msgr2=0x7f998c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f998c0a1ba0 0x7f998c0a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f999800cc40 tx=0x7f9998019000 comp rx=0 tx=0).stop 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 msgr2=0x7f99a019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a019d3f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f999c009230 tx=0x7f999c0040c0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 shutdown_connections 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f998c0a1ba0 0x7f998c0a4060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a01975a0 0x7f99a0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0106450 0x7f99a019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 --2- 172.21.15.42:0/353695054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0101380 0x7f99a019d3f0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 >> 172.21.15.42:0/353695054 conn(0x7f99a0078580 msgr2=0x7f99a00ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 shutdown_connections 2024-03-12T04:25:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.033+0000 7f99a84c0640 1 -- 172.21.15.42:0/353695054 wait complete. 2024-03-12T04:25:22.046 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T04:25:22.046 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T04:25:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 -- 172.21.15.42:0/2754378311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 msgr2=0x7f59f810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 --2- 172.21.15.42:0/2754378311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f810dc30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009ca0 tx=0x7f59f002c180 comp rx=0 tx=0).stop 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 -- 172.21.15.42:0/2754378311 shutdown_connections 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 --2- 172.21.15.42:0/2754378311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f810dc30 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 --2- 172.21.15.42:0/2754378311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f8102d10 0x7f59f810b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 --2- 172.21.15.42:0/2754378311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f81023f0 0x7f59f81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 -- 172.21.15.42:0/2754378311 >> 172.21.15.42:0/2754378311 conn(0x7f59f80771e0 msgr2=0x7f59f80775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 -- 172.21.15.42:0/2754378311 shutdown_connections 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.148+0000 7f59fd348640 1 -- 172.21.15.42:0/2754378311 wait complete. 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 Processor -- start 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 -- start start 2024-03-12T04:25:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f81023f0 0x7f59f81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f8102d10 0x7f59f8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819c550 con 0x7f59f810b7c0 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819c6c0 con 0x7f59f81023f0 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59fd348640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819c880 con 0x7f59f8102d10 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40696/0 (socket says 172.21.15.42:40696) 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 -- 172.21.15.42:0/1458199506 learned_addr learned my addr 172.21.15.42:0/1458199506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f81023f0 0x7f59f81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59effff640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f8102d10 0x7f59f8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 -- 172.21.15.42:0/1458199506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f8102d10 msgr2=0x7f59f8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f8102d10 0x7f59f8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 -- 172.21.15.42:0/1458199506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f81023f0 msgr2=0x7f59f81976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f81023f0 0x7f59f81976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59f77fe640 1 -- 172.21.15.42:0/1458199506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0009900 con 0x7f59f810b7c0 2024-03-12T04:25:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.149+0000 7f59effff640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f8102d10 0x7f59f8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.150+0000 7f59f77fe640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f819bf50 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f59f002c790 tx=0x7f59f00349e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.150+0000 7f59f4ff9640 1 -- 172.21.15.42:0/1458199506 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0032680 con 0x7f59f810b7c0 2024-03-12T04:25:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.150+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f819cb00 con 0x7f59f810b7c0 2024-03-12T04:25:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.150+0000 7f59f4ff9640 1 -- 172.21.15.42:0/1458199506 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59f0032c90 con 0x7f59f810b7c0 2024-03-12T04:25:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.150+0000 7f59f4ff9640 1 -- 172.21.15.42:0/1458199506 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f003f440 con 0x7f59f810b7c0 2024-03-12T04:25:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.150+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f81a43e0 con 0x7f59f810b7c0 2024-03-12T04:25:22.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.152+0000 7f59f4ff9640 1 -- 172.21.15.42:0/1458199506 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f59f003b040 con 0x7f59f810b7c0 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.152+0000 7f59ffffa740 1 Processor -- start 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.152+0000 7f59ffffa740 1 -- start start 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b3ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b3ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59ffffa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b39c960 con 0x564c2b3e1b70 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59ffffa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b350310 con 0x564c2b3ea750 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59ffffa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b34d840 con 0x564c2b3e1250 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b3ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b3ea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40712/0 (socket says 172.21.15.42:40712) 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 -- 172.21.15.42:0/1291760265 learned_addr learned my addr 172.21.15.42:0/1291760265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f4ff9640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59e40a1ba0 0x7f59e40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59effff640 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b3ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 -- 172.21.15.42:0/1291760265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 msgr2=0x564c2b3e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 -- 172.21.15.42:0/1291760265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 msgr2=0x564c2b3ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f77fe640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59e40a1ba0 0x7f59e40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b3ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 -- 172.21.15.42:0/1291760265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2b30b060 con 0x564c2b3e1b70 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f4ff9640 1 -- 172.21.15.42:0/1458199506 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f59f00f0430 con 0x7f59f810b7c0 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.153+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b3ea210 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009a70 tx=0x7f59e8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ca52f6f6f0b553 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59f77fe640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59e40a1ba0 0x7f59e40a4060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f59f00bd120 tx=0x7f59f008f390 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ed7fa640 1 -- 172.21.15.42:0/1291760265 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8009570 con 0x564c2b3e1b70 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ed7fa640 1 -- 172.21.15.42:0/1291760265 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59e8013470 con 0x564c2b3e1b70 2024-03-12T04:25:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ed7fa640 1 -- 172.21.15.42:0/1291760265 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8012630 con 0x564c2b3e1b70 2024-03-12T04:25:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 -- 172.21.15.42:0/1291760265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 msgr2=0x564c2b3ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b3ea210 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009a70 tx=0x7f59e8009f30 comp rx=0 tx=0).stop 2024-03-12T04:25:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 -- 172.21.15.42:0/1291760265 shutdown_connections 2024-03-12T04:25:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b3ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b3ea210 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 --2- 172.21.15.42:0/1291760265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 -- 172.21.15.42:0/1291760265 >> 172.21.15.42:0/1291760265 conn(0x564c2b3db980 msgr2=0x564c2b3dbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 -- 172.21.15.42:0/1291760265 shutdown_connections 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.154+0000 7f59ffffa740 1 -- 172.21.15.42:0/1291760265 wait complete. 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 -- start start 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 Processor -- start 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b349020 con 0x564c2b3ea750 2024-03-12T04:25:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b01c430 con 0x564c2b3e1b70 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2afd8e00 con 0x564c2b3e1250 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59ffffa740 10 client.0 _ll_register_callbacks cb 0x564c2b2f0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40718/0 (socket says 172.21.15.42:40718) 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 -- 172.21.15.42:0/3651655443 learned_addr learned my addr 172.21.15.42:0/3651655443 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 -- 172.21.15.42:0/3651655443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 msgr2=0x564c2b3e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 -- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 msgr2=0x564c2b3e7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3e7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.155+0000 7f59effff640 1 -- 172.21.15.42:0/3651655443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e80096d0 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59effff640 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e81e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005da0 tx=0x7f59e000d540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825597f2f981967a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59de7fc640 1 -- 172.21.15.42:0/3651655443 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e00140d0 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59de7fc640 1 -- 172.21.15.42:0/3651655443 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f59e0014230 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2b0487e0 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59de7fc640 1 -- 172.21.15.42:0/3651655443 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59e0005020 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 10 client.18393 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564c2b3476a0 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59de7fc640 1 -- 172.21.15.42:0/3651655443 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564c2b0487e0 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59de7fc640 1 -- 172.21.15.42:0/3651655443 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59e0004c60 con 0x564c2b3ea750 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 10 client.18393 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 10 client.18393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 20 client.18393 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b3faa90 0x564c2b3fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 4 client.18393 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.156+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2b085720 con 0x564c2b3faa90 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.157+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b3faa90 0x564c2b3fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.157+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b3faa90 0x564c2b3fceb0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a1d819805eccc4 server_cookie=badeda5b0e2a7c2d in_seq=0 out_seq=0 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.157+0000 7f59de7fc640 10 client.18393 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59de7fc640 1 -- 172.21.15.42:0/3651655443 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c2b085720 con 0x564c2b3faa90 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59de7fc640 10 client.18393 handle_command_reply: tid=0 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 client.18393 shutdown 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 msgr2=0x564c2b3e81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e81e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005da0 tx=0x7f59e000d540 comp rx=0 tx=0).stop 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 shutdown_connections 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b3faa90 0x564c2b3fceb0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e81e0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59de7fc640 0 client.18393 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 --2- 172.21.15.42:0/3651655443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 >> 172.21.15.42:0/3651655443 conn(0x564c2b3db980 msgr2=0x564c2b3dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 shutdown_connections 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 1 -- 172.21.15.42:0/3651655443 wait complete. 2024-03-12T04:25:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.158+0000 7f59ffffa740 20 client.18393 trim_cache size 0 max 16384 2024-03-12T04:25:22.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 Processor -- start 2024-03-12T04:25:22.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 -- start start 2024-03-12T04:25:22.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b2ce130 con 0x564c2b3ea750 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b0487e0 con 0x564c2b3e1b70 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59ffffa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b3cda40 con 0x564c2b3e1250 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40728/0 (socket says 172.21.15.42:40728) 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 -- 172.21.15.42:0/518839028 learned_addr learned my addr 172.21.15.42:0/518839028 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 -- 172.21.15.42:0/518839028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 msgr2=0x564c2b3e9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 -- 172.21.15.42:0/518839028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 msgr2=0x564c2b3f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.170+0000 7f59effff640 1 -- 172.21.15.42:0/518839028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2b2e3c20 con 0x564c2b3ea750 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59effff640 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e49e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f59e000db00 tx=0x7f59e0012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8221dd89f229b745 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ecff9640 1 -- 172.21.15.42:0/518839028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0014a40 con 0x564c2b3ea750 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ecff9640 1 -- 172.21.15.42:0/518839028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59e001a030 con 0x564c2b3ea750 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ecff9640 1 -- 172.21.15.42:0/518839028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0014330 con 0x564c2b3ea750 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 -- 172.21.15.42:0/518839028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 msgr2=0x564c2b3e49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e49e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f59e000db00 tx=0x7f59e0012620 comp rx=0 tx=0).stop 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 -- 172.21.15.42:0/518839028 shutdown_connections 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3ea750 0x564c2b3e49e0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3e1b70 0x564c2b3f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 --2- 172.21.15.42:0/518839028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b3e9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 -- 172.21.15.42:0/518839028 >> 172.21.15.42:0/518839028 conn(0x564c2b3db980 msgr2=0x564c2b3e2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 -- 172.21.15.42:0/518839028 shutdown_connections 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.171+0000 7f59ffffa740 1 -- 172.21.15.42:0/518839028 wait complete. 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 -- start start 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 Processor -- start 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b44f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b44fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b449b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b44a130 con 0x564c2b3e1b70 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b44a2a0 con 0x564c2b3ea750 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b44a460 con 0x564c2b3e1250 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59ffffa740 10 client.0 _ll_register_callbacks cb 0x7f59e80094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T04:25:22.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b44fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b44fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40732/0 (socket says 172.21.15.42:40732) 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 -- 172.21.15.42:0/303377719 learned_addr learned my addr 172.21.15.42:0/303377719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59effff640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b44f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59effff640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b449b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 -- 172.21.15.42:0/303377719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 msgr2=0x564c2b44f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b44f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 -- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 msgr2=0x564c2b449b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b449b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59f6ffd640 1 -- 172.21.15.42:0/303377719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2b3476a0 con 0x564c2b3e1b70 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.172+0000 7f59effff640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b44f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59f6ffd640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b44fca0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f59e8008010 tx=0x7f59e8013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0aa401ad00dd68 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-03-12T04:25:22.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59de7fc640 1 -- 172.21.15.42:0/303377719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8021040 con 0x564c2b3e1b70 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59e8013470 con 0x564c2b3e1b70 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59de7fc640 1 -- 172.21.15.42:0/303377719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f59e80137e0 con 0x564c2b3e1b70 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59de7fc640 1 -- 172.21.15.42:0/303377719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59e800ee90 con 0x564c2b3e1b70 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 10 client.18399 fetch_fsmap learned FSMap version 27 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564c2b3098c0 con 0x564c2b3e1b70 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59de7fc640 1 -- 172.21.15.42:0/303377719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59e8013470 con 0x564c2b3e1b70 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59de7fc640 1 -- 172.21.15.42:0/303377719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59e8011070 con 0x564c2b3e1b70 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 10 client.18399 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 20 client.18399 populate_metadata read hostname 'smithi042' 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b457610 0x564c2b459a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 4 client.18399 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T04:25:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.173+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 --> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c2b0d4890 con 0x564c2b457610 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59effff640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b457610 0x564c2b459a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59effff640 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b457610 0x564c2b459a30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31aeaad2bde81041 server_cookie=9f02c674625d69d3 in_seq=0 out_seq=0 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59de7fc640 10 client.18399 ms_handle_connect on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59de7fc640 1 -- 172.21.15.42:0/303377719 <== mds.0 v2:172.21.15.119:6838/1451526204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c2b0d4890 con 0x564c2b457610 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59de7fc640 10 client.18399 handle_command_reply: tid=0 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 client.18399 shutdown 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 msgr2=0x564c2b44fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b44fca0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f59e8008010 tx=0x7f59e8013dd0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 shutdown_connections 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:6838/1451526204,v1:172.21.15.119:6839/1451526204] conn(0x564c2b457610 0x564c2b459a30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c2b3ea750 0x564c2b449b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2b3e1b70 0x564c2b44fca0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59de7fc640 0 client.18399 ms_handle_reset on v2:172.21.15.119:6838/1451526204 2024-03-12T04:25:22.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 --2- 172.21.15.42:0/303377719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2b3e1250 0x564c2b44f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 >> 172.21.15.42:0/303377719 conn(0x564c2b3db980 msgr2=0x564c2b3da2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 shutdown_connections 2024-03-12T04:25:22.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 1 -- 172.21.15.42:0/303377719 wait complete. 2024-03-12T04:25:22.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.174+0000 7f59ffffa740 20 client.18399 trim_cache size 0 max 16384 2024-03-12T04:25:22.187 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59e40a1ba0 msgr2=0x7f59e40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59e40a1ba0 0x7f59e40a4060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f59f00bd120 tx=0x7f59f008f390 comp rx=0 tx=0).stop 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 msgr2=0x7f59f819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f819bf50 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f59f002c790 tx=0x7f59f00349e0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 shutdown_connections 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59e40a1ba0 0x7f59e40a4060 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59f810b7c0 0x7f59f819bf50 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f8102d10 0x7f59f8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 --2- 172.21.15.42:0/1458199506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f81023f0 0x7f59f81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 >> 172.21.15.42:0/1458199506 conn(0x7f59f80771e0 msgr2=0x7f59f80fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:22.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 shutdown_connections 2024-03-12T04:25:22.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:22.175+0000 7f59fd348640 1 -- 172.21.15.42:0/1458199506 wait complete. 2024-03-12T04:25:22.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:22 smithi119 ceph-mon[36119]: pgmap v316: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-03-12T04:25:22.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:22 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/496276357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:25:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:22 smithi136 ceph-mon[35571]: pgmap v316: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-03-12T04:25:22.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:22 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/496276357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:25:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:22 smithi042 ceph-mon[32260]: pgmap v316: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-03-12T04:25:22.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:22 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/496276357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:25:23.190 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-12T04:25:23.190 INFO:tasks.fwd_scrub:done joining 2024-03-12T04:25:23.191 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-12T04:25:23.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:25:23.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 -- 172.21.15.42:0/3939197239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398100c60 msgr2=0x7f4398104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 --2- 172.21.15.42:0/3939197239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398100c60 0x7f4398104ca0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f438c009930 tx=0x7f438c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 -- 172.21.15.42:0/3939197239 shutdown_connections 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 --2- 172.21.15.42:0/3939197239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4398105380 0x7f439810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 --2- 172.21.15.42:0/3939197239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398100c60 0x7f4398104ca0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 --2- 172.21.15.42:0/3939197239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43981002b0 0x7f4398100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 -- 172.21.15.42:0/3939197239 >> 172.21.15.42:0/3939197239 conn(0x7f4398077120 msgr2=0x7f4398077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 -- 172.21.15.42:0/3939197239 shutdown_connections 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.315+0000 7f439d01b640 1 -- 172.21.15.42:0/3939197239 wait complete. 2024-03-12T04:25:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.316+0000 7f439d01b640 1 Processor -- start 2024-03-12T04:25:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.316+0000 7f439d01b640 1 -- start start 2024-03-12T04:25:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439d01b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43981002b0 0x7f439819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439d01b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398105380 0x7f439819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439d01b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4398199740 0x7f4398199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439d01b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439819a210 con 0x7f4398105380 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439d01b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439819a380 con 0x7f43981002b0 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439d01b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439819a540 con 0x7f4398199740 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398105380 0x7f439819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398105380 0x7f439819fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40748/0 (socket says 172.21.15.42:40748) 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 -- 172.21.15.42:0/1449858679 learned_addr learned my addr 172.21.15.42:0/1449858679 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f4396d9d640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43981002b0 0x7f439819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439759e640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4398199740 0x7f4398199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 -- 172.21.15.42:0/1449858679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4398199740 msgr2=0x7f4398199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4398199740 0x7f4398199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 -- 172.21.15.42:0/1449858679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43981002b0 msgr2=0x7f439819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43981002b0 0x7f439819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 -- 172.21.15.42:0/1449858679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c009560 con 0x7f4398105380 2024-03-12T04:25:23.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f4396d9d640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43981002b0 0x7f439819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:25:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f439659c640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398105380 0x7f439819fbc0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f438c000c00 tx=0x7f438c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f437ffff640 1 -- 172.21.15.42:0/1449858679 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c02cb70 con 0x7f4398105380 2024-03-12T04:25:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.317+0000 7f437ffff640 1 -- 172.21.15.42:0/1449858679 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f438c02de90 con 0x7f4398105380 2024-03-12T04:25:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.318+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43981a6480 con 0x7f4398105380 2024-03-12T04:25:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.318+0000 7f437ffff640 1 -- 172.21.15.42:0/1449858679 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c0053c0 con 0x7f4398105380 2024-03-12T04:25:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.318+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43981a69b0 con 0x7f4398105380 2024-03-12T04:25:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.319+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4398101780 con 0x7f4398105380 2024-03-12T04:25:23.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.320+0000 7f437ffff640 1 -- 172.21.15.42:0/1449858679 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f438c0035f0 con 0x7f4398105380 2024-03-12T04:25:23.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.320+0000 7f437ffff640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f43800a1c20 0x7f43800a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.321+0000 7f4396d9d640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f43800a1c20 0x7f43800a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.321+0000 7f437ffff640 1 -- 172.21.15.42:0/1449858679 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f438c0efac0 con 0x7f4398105380 2024-03-12T04:25:23.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.321+0000 7f4396d9d640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f43800a1c20 0x7f43800a40e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43880058c0 tx=0x7f4388005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:23.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.325+0000 7f437ffff640 1 -- 172.21.15.42:0/1449858679 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f438c0b8ac0 con 0x7f4398105380 2024-03-12T04:25:23.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.508+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43980683c0 con 0x7f4398105380 2024-03-12T04:25:23.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.509+0000 7f437ffff640 1 -- 172.21.15.42:0/1449858679 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+23076 (secure 0 0 0) 0x7f438c0bc180 con 0x7f4398105380 2024-03-12T04:25:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:25:23.513 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:25:15.177682+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1061357309":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/2954927866":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6801/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/3686333836":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/4126408073":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:25:23.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.511+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f43800a1c20 msgr2=0x7f43800a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.511+0000 7f439d01b640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f43800a1c20 0x7f43800a40e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43880058c0 tx=0x7f4388005850 comp rx=0 tx=0).stop 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398105380 msgr2=0x7f439819fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398105380 0x7f439819fbc0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f438c000c00 tx=0x7f438c004270 comp rx=0 tx=0).stop 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 shutdown_connections 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f43800a1c20 0x7f43800a40e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4398199740 0x7f4398199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4398105380 0x7f439819fbc0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 --2- 172.21.15.42:0/1449858679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43981002b0 0x7f439819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 >> 172.21.15.42:0/1449858679 conn(0x7f4398077120 msgr2=0x7f43980ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 shutdown_connections 2024-03-12T04:25:23.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.512+0000 7f439d01b640 1 -- 172.21.15.42:0/1449858679 wait complete. 2024-03-12T04:25:23.525 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:25:23.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:25:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.628+0000 7fe3bab56640 1 -- 172.21.15.42:0/3910185389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 msgr2=0x7fe3b4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.628+0000 7fe3bab56640 1 --2- 172.21.15.42:0/3910185389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4100640 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009930 tx=0x7fe3a802bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- 172.21.15.42:0/3910185389 shutdown_connections 2024-03-12T04:25:23.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 --2- 172.21.15.42:0/3910185389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b4105330 0x7fe3b410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 --2- 172.21.15.42:0/3910185389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3b4100c10 0x7fe3b4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 --2- 172.21.15.42:0/3910185389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4100640 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- 172.21.15.42:0/3910185389 >> 172.21.15.42:0/3910185389 conn(0x7fe3b4077180 msgr2=0x7fe3b4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- 172.21.15.42:0/3910185389 shutdown_connections 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- 172.21.15.42:0/3910185389 wait complete. 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 Processor -- start 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- start start 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b4105330 0x7fe3b4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3b419c010 0x7fe3b419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b419ca00 con 0x7fe3b4100260 2024-03-12T04:25:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b419cb70 con 0x7fe3b419c010 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3bab56640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b419cce0 con 0x7fe3b4105330 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3b88f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3b88f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40766/0 (socket says 172.21.15.42:40766) 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.629+0000 7fe3b88f2640 1 -- 172.21.15.42:0/2024395174 learned_addr learned my addr 172.21.15.42:0/2024395174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b90f3640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3b419c010 0x7fe3b419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b4105330 0x7fe3b4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b88f2640 1 -- 172.21.15.42:0/2024395174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b4105330 msgr2=0x7fe3b4197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b88f2640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b4105330 0x7fe3b4197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b88f2640 1 -- 172.21.15.42:0/2024395174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3b419c010 msgr2=0x7fe3b419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b88f2640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3b419c010 0x7fe3b419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b88f2640 1 -- 172.21.15.42:0/2024395174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8009560 con 0x7fe3b4100260 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b88f2640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4197750 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009230 tx=0x7fe3a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b1ffb640 1 -- 172.21.15.42:0/2024395174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80044c0 con 0x7fe3b4100260 2024-03-12T04:25:23.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b1ffb640 1 -- 172.21.15.42:0/2024395174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe3a802de90 con 0x7fe3b4100260 2024-03-12T04:25:23.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b41a42c0 con 0x7fe3b4100260 2024-03-12T04:25:23.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3b1ffb640 1 -- 172.21.15.42:0/2024395174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8005400 con 0x7fe3b4100260 2024-03-12T04:25:23.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.630+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b41a4740 con 0x7fe3b4100260 2024-03-12T04:25:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.631+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b4067f70 con 0x7fe3b4100260 2024-03-12T04:25:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.633+0000 7fe3b1ffb640 1 -- 172.21.15.42:0/2024395174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe3a8005560 con 0x7fe3b4100260 2024-03-12T04:25:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.633+0000 7fe3b1ffb640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe3a40a1ba0 0x7fe3a40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.633+0000 7fe3b1ffb640 1 -- 172.21.15.42:0/2024395174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe3a80ef6e0 con 0x7fe3b4100260 2024-03-12T04:25:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.633+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe3a40a1ba0 0x7fe3a40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.634+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe3a40a1ba0 0x7fe3a40a4060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4068d30 tx=0x7fe3a000b410 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:23.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.638+0000 7fe3b1ffb640 1 -- 172.21.15.42:0/2024395174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3a80b86e0 con 0x7fe3b4100260 2024-03-12T04:25:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.820+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3b41095d0 con 0x7fe3b4100260 2024-03-12T04:25:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.820+0000 7fe3b1ffb640 1 -- 172.21.15.42:0/2024395174 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3a80bbda0 con 0x7fe3b4100260 2024-03-12T04:25:23.823 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:25:23.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe3a40a1ba0 msgr2=0x7fe3a40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe3a40a1ba0 0x7fe3a40a4060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4068d30 tx=0x7fe3a000b410 comp rx=0 tx=0).stop 2024-03-12T04:25:23.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 msgr2=0x7fe3b4197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4197750 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009230 tx=0x7fe3a80040c0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 shutdown_connections 2024-03-12T04:25:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe3a40a1ba0 0x7fe3a40a4060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3b419c010 0x7fe3b419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b4105330 0x7fe3b4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 --2- 172.21.15.42:0/2024395174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3b4100260 0x7fe3b4197750 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 >> 172.21.15.42:0/2024395174 conn(0x7fe3b4077180 msgr2=0x7fe3b40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 shutdown_connections 2024-03-12T04:25:23.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.822+0000 7fe3bab56640 1 -- 172.21.15.42:0/2024395174 wait complete. 2024-03-12T04:25:23.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:25:23.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.938+0000 7f4cc256b640 1 -- 172.21.15.42:0/2290135739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc0685f0 msgr2=0x7f4cbc0689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.938+0000 7f4cc256b640 1 --2- 172.21.15.42:0/2290135739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc0685f0 0x7f4cbc0689d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009930 tx=0x7f4cac02bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.938+0000 7f4cc256b640 1 -- 172.21.15.42:0/2290135739 shutdown_connections 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.938+0000 7f4cc256b640 1 --2- 172.21.15.42:0/2290135739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cbc1064c0 0x7f4cbc10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.938+0000 7f4cc256b640 1 --2- 172.21.15.42:0/2290135739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cbc068fa0 0x7f4cbc105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.938+0000 7f4cc256b640 1 --2- 172.21.15.42:0/2290135739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc0685f0 0x7f4cbc0689d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.938+0000 7f4cc256b640 1 -- 172.21.15.42:0/2290135739 >> 172.21.15.42:0/2290135739 conn(0x7f4cbc077180 msgr2=0x7f4cbc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 -- 172.21.15.42:0/2290135739 shutdown_connections 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 -- 172.21.15.42:0/2290135739 wait complete. 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 Processor -- start 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 -- start start 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cbc0685f0 0x7f4cbc197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc1064c0 0x7f4cbc197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cbc19bfd0 0x7f4cbc19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19c9c0 con 0x7f4cbc1064c0 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19cb30 con 0x7f4cbc0685f0 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.939+0000 7f4cc256b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19cca0 con 0x7f4cbc19bfd0 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc1064c0 0x7f4cbc197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc1064c0 0x7f4cbc197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40788/0 (socket says 172.21.15.42:40788) 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 -- 172.21.15.42:0/1343888791 learned_addr learned my addr 172.21.15.42:0/1343888791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbbfff640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cbc0685f0 0x7f4cbc197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 -- 172.21.15.42:0/1343888791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cbc19bfd0 msgr2=0x7f4cbc19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cbc19bfd0 0x7f4cbc19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 -- 172.21.15.42:0/1343888791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cbc0685f0 msgr2=0x7f4cbc197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cbc0685f0 0x7f4cbc197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 -- 172.21.15.42:0/1343888791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac009560 con 0x7f4cbc1064c0 2024-03-12T04:25:23.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cbb7fe640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc1064c0 0x7f4cbc197c50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000d750 tx=0x7f4cb000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cb97fa640 1 -- 172.21.15.42:0/1343888791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0009c30 con 0x7f4cbc1064c0 2024-03-12T04:25:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cb97fa640 1 -- 172.21.15.42:0/1343888791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4cb000de60 con 0x7f4cbc1064c0 2024-03-12T04:25:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc1a4280 con 0x7f4cbc1064c0 2024-03-12T04:25:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cb97fa640 1 -- 172.21.15.42:0/1343888791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0004e00 con 0x7f4cbc1064c0 2024-03-12T04:25:23.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.940+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc1a4730 con 0x7f4cbc1064c0 2024-03-12T04:25:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.941+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc067f70 con 0x7f4cbc1064c0 2024-03-12T04:25:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.942+0000 7f4cb97fa640 1 -- 172.21.15.42:0/1343888791 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f4cb0004f60 con 0x7f4cbc1064c0 2024-03-12T04:25:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.943+0000 7f4cb97fa640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4ca40a1c20 0x7f4ca40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.943+0000 7f4cbbfff640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4ca40a1c20 0x7f4ca40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.943+0000 7f4cb97fa640 1 -- 172.21.15.42:0/1343888791 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4cb00cd130 con 0x7f4cbc1064c0 2024-03-12T04:25:23.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.943+0000 7f4cbbfff640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4ca40a1c20 0x7f4ca40a40e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4cac005ab0 tx=0x7f4cac005fd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:23.947+0000 7f4cb97fa640 1 -- 172.21.15.42:0/1343888791 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4cb0095fc0 con 0x7f4cbc1064c0 2024-03-12T04:25:24.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.128+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4cbc109710 con 0x7f4cbc1064c0 2024-03-12T04:25:24.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.129+0000 7f4cb97fa640 1 -- 172.21.15.42:0/1343888791 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4cb0020020 con 0x7f4cbc1064c0 2024-03-12T04:25:24.131 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:25:24.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.130+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4ca40a1c20 msgr2=0x7f4ca40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.130+0000 7f4cc256b640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4ca40a1c20 0x7f4ca40a40e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4cac005ab0 tx=0x7f4cac005fd0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.130+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc1064c0 msgr2=0x7f4cbc197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.130+0000 7f4cc256b640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc1064c0 0x7f4cbc197c50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000d750 tx=0x7f4cb000dc10 comp rx=0 tx=0).stop 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 shutdown_connections 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f4ca40a1c20 0x7f4ca40a40e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cbc19bfd0 0x7f4cbc19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cbc1064c0 0x7f4cbc197c50 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 --2- 172.21.15.42:0/1343888791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cbc0685f0 0x7f4cbc197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 >> 172.21.15.42:0/1343888791 conn(0x7f4cbc077180 msgr2=0x7f4cbc0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 shutdown_connections 2024-03-12T04:25:24.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.131+0000 7f4cc256b640 1 -- 172.21.15.42:0/1343888791 wait complete. 2024-03-12T04:25:24.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:25:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- 172.21.15.42:0/808534734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d5810b5d0 msgr2=0x7f8d5810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 --2- 172.21.15.42:0/808534734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d5810b5d0 0x7f8d5810da70 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009ca0 tx=0x7f8d5002c180 comp rx=0 tx=0).stop 2024-03-12T04:25:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- 172.21.15.42:0/808534734 shutdown_connections 2024-03-12T04:25:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 --2- 172.21.15.42:0/808534734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d5810b5d0 0x7f8d5810da70 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 --2- 172.21.15.42:0/808534734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d58102ad0 0x7f8d5810afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 --2- 172.21.15.42:0/808534734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d581021b0 0x7f8d58102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- 172.21.15.42:0/808534734 >> 172.21.15.42:0/808534734 conn(0x7f8d580fb9d0 msgr2=0x7f8d580fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- 172.21.15.42:0/808534734 shutdown_connections 2024-03-12T04:25:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- 172.21.15.42:0/808534734 wait complete. 2024-03-12T04:25:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 Processor -- start 2024-03-12T04:25:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- start start 2024-03-12T04:25:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d581021b0 0x7f8d581975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d58102ad0 0x7f8d58197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d5810b5d0 0x7f8d5819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5819c410 con 0x7f8d581021b0 2024-03-12T04:25:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5819c580 con 0x7f8d5810b5d0 2024-03-12T04:25:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.247+0000 7f8d5f331640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5819c740 con 0x7f8d58102ad0 2024-03-12T04:25:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d581021b0 0x7f8d581975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d581021b0 0x7f8d581975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40802/0 (socket says 172.21.15.42:40802) 2024-03-12T04:25:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 -- 172.21.15.42:0/1457571983 learned_addr learned my addr 172.21.15.42:0/1457571983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5eb30640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d5810b5d0 0x7f8d5819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5db2e640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d58102ad0 0x7f8d58197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 -- 172.21.15.42:0/1457571983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d58102ad0 msgr2=0x7f8d58197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d58102ad0 0x7f8d58197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 -- 172.21.15.42:0/1457571983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d5810b5d0 msgr2=0x7f8d5819be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d5810b5d0 0x7f8d5819be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 -- 172.21.15.42:0/1457571983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50009900 con 0x7f8d581021b0 2024-03-12T04:25:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5db2e640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d58102ad0 0x7f8d58197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5eb30640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d5810b5d0 0x7f8d5819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:25:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5e32f640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d581021b0 0x7f8d581975a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400d950 tx=0x7f8d5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d4f7fe640 1 -- 172.21.15.42:0/1457571983 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54009c30 con 0x7f8d581021b0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d4f7fe640 1 -- 172.21.15.42:0/1457571983 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d54009d90 con 0x7f8d581021b0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5819ca20 con 0x7f8d581021b0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d4f7fe640 1 -- 172.21.15.42:0/1457571983 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5400f6b0 con 0x7f8d581021b0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.248+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d581a42a0 con 0x7f8d581021b0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.249+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d58067600 con 0x7f8d581021b0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.251+0000 7f8d4f7fe640 1 -- 172.21.15.42:0/1457571983 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8d54010aa0 con 0x7f8d581021b0 2024-03-12T04:25:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.251+0000 7f8d4f7fe640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8d440a1ba0 0x7f8d440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.251+0000 7f8d5db2e640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8d440a1ba0 0x7f8d440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.251+0000 7f8d4f7fe640 1 -- 172.21.15.42:0/1457571983 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8d540cd590 con 0x7f8d581021b0 2024-03-12T04:25:24.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.252+0000 7f8d5db2e640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8d440a1ba0 0x7f8d440a4060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005fd0 tx=0x7f8d48005dc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:24.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.255+0000 7f8d4f7fe640 1 -- 172.21.15.42:0/1457571983 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d54096420 con 0x7f8d581021b0 2024-03-12T04:25:24.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.437+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d58106670 con 0x7f8d581021b0 2024-03-12T04:25:24.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.437+0000 7f8d4f7fe640 1 -- 172.21.15.42:0/1457571983 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d540200a0 con 0x7f8d581021b0 2024-03-12T04:25:24.440 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:25:24.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8d440a1ba0 msgr2=0x7f8d440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8d440a1ba0 0x7f8d440a4060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005fd0 tx=0x7f8d48005dc0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d581021b0 msgr2=0x7f8d581975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d581021b0 0x7f8d581975a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400d950 tx=0x7f8d5400de10 comp rx=0 tx=0).stop 2024-03-12T04:25:24.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 shutdown_connections 2024-03-12T04:25:24.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f8d440a1ba0 0x7f8d440a4060 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d5810b5d0 0x7f8d5819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d58102ad0 0x7f8d58197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 --2- 172.21.15.42:0/1457571983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d581021b0 0x7f8d581975a0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 >> 172.21.15.42:0/1457571983 conn(0x7f8d580fb9d0 msgr2=0x7f8d580fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:24.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 shutdown_connections 2024-03-12T04:25:24.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.439+0000 7f8d5f331640 1 -- 172.21.15.42:0/1457571983 wait complete. 2024-03-12T04:25:24.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:24 smithi042 ceph-mon[32260]: pgmap v317: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-03-12T04:25:24.452 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:24 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1449858679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:25:24.452 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:24 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2024395174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:25:24.452 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:24 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1343888791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:25:24.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-12T04:25:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 -- 172.21.15.42:0/1080410100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101d30 msgr2=0x7f9d38105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1080410100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101d30 0x7f9d38105d70 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009930 tx=0x7f9d2c02bc30 comp rx=0 tx=0).stop 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 -- 172.21.15.42:0/1080410100 shutdown_connections 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1080410100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d38106450 0x7f9d3810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1080410100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101d30 0x7f9d38105d70 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1080410100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d38101380 0x7f9d38101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 -- 172.21.15.42:0/1080410100 >> 172.21.15.42:0/1080410100 conn(0x7f9d380fcc20 msgr2=0x7f9d380ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 -- 172.21.15.42:0/1080410100 shutdown_connections 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.553+0000 7f9d3db65640 1 -- 172.21.15.42:0/1080410100 wait complete. 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 Processor -- start 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 -- start start 2024-03-12T04:25:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101380 0x7f9d3819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d38106450 0x7f9d3819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d38197700 0x7f9d38197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381981d0 con 0x7f9d38101380 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38198340 con 0x7f9d38106450 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d3db65640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38198500 con 0x7f9d38197700 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101380 0x7f9d3819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101380 0x7f9d3819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40818/0 (socket says 172.21.15.42:40818) 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 -- 172.21.15.42:0/1960266399 learned_addr learned my addr 172.21.15.42:0/1960266399 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d37fff640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d38197700 0x7f9d38197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d36ffd640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d38106450 0x7f9d3819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 -- 172.21.15.42:0/1960266399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d38197700 msgr2=0x7f9d38197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d38197700 0x7f9d38197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 -- 172.21.15.42:0/1960266399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d38106450 msgr2=0x7f9d3819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d38106450 0x7f9d3819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.554+0000 7f9d377fe640 1 -- 172.21.15.42:0/1960266399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009560 con 0x7f9d38101380 2024-03-12T04:25:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.555+0000 7f9d377fe640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101380 0x7f9d3819d640 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800b4a0 tx=0x7f9d2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.555+0000 7f9d34ff9640 1 -- 172.21.15.42:0/1960266399 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28010610 con 0x7f9d38101380 2024-03-12T04:25:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.555+0000 7f9d34ff9640 1 -- 172.21.15.42:0/1960266399 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d28010c20 con 0x7f9d38101380 2024-03-12T04:25:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.555+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d381a44a0 con 0x7f9d38101380 2024-03-12T04:25:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.555+0000 7f9d34ff9640 1 -- 172.21.15.42:0/1960266399 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28017d00 con 0x7f9d38101380 2024-03-12T04:25:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.555+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d381a4930 con 0x7f9d38101380 2024-03-12T04:25:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.556+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38102850 con 0x7f9d38101380 2024-03-12T04:25:24.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.557+0000 7f9d34ff9640 1 -- 172.21.15.42:0/1960266399 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f9d28010770 con 0x7f9d38101380 2024-03-12T04:25:24.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.558+0000 7f9d34ff9640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9d200a1ba0 0x7f9d200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:25:24.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.558+0000 7f9d36ffd640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9d200a1ba0 0x7f9d200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:25:24.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.558+0000 7f9d34ff9640 1 -- 172.21.15.42:0/1960266399 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9d280ccdf0 con 0x7f9d38101380 2024-03-12T04:25:24.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.558+0000 7f9d36ffd640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9d200a1ba0 0x7f9d200a4060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009900 tx=0x7f9d2c005fd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:25:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.562+0000 7f9d34ff9640 1 -- 172.21.15.42:0/1960266399 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d28095d90 con 0x7f9d38101380 2024-03-12T04:25:24.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:24 smithi119 ceph-mon[36119]: pgmap v317: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-03-12T04:25:24.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:24 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1449858679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:25:24.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:24 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2024395174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:25:24.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:24 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1343888791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:25:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:24 smithi136 ceph-mon[35571]: pgmap v317: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-03-12T04:25:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:24 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1449858679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:25:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:24 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2024395174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:25:24.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:24 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1343888791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:25:24.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.801+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f9d38109680 con 0x7f9d38101380 2024-03-12T04:25:24.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.809+0000 7f9d34ff9640 1 -- 172.21.15.42:0/1960266399 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+735001 (secure 0 0 0) 0x7f9d28099450 con 0x7f9d38101380 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 26, 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: "active_gid": 15522, 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: "active_name": "x", 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addrs": { 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-03-12T04:25:24.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2974863619 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2974863619 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addr": "172.21.15.42:6801/2974863619", 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "active_change": "2024-03-12T04:15:11.144287+0000", 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "active_mgr_features": 4540701547738038271, 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "standbys": [ 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 24976, 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "z", 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-03-12T04:25:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-03-12T04:25:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-03-12T04:25:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-03-12T04:25:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-03-12T04:25:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-03-12T04:25:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-03-12T04:25:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-03-12T04:25:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-03-12T04:25:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-03-12T04:25:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-03-12T04:25:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-03-12T04:25:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T04:25:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-03-12T04:25:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-03-12T04:25:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-03-12T04:25:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-03-12T04:25:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-12T04:25:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-03-12T04:25:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-03-12T04:25:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-12T04:25:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-03-12T04:25:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-03-12T04:25:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-03-12T04:25:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-03-12T04:25:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-03-12T04:25:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-03-12T04:25:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-03-12T04:25:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-03-12T04:25:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-03-12T04:25:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-03-12T04:25:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.919 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:24.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.921 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.923 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-12T04:25:24.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.926 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.932 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-12T04:25:24.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.936 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.941 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-12T04:25:24.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.951 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-12T04:25:24.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.953 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-03-12T04:25:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.959 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.963 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-03-12T04:25:24.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-03-12T04:25:24.967 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.969 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:24.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.972 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.974 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:24.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-03-12T04:25:24.978 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-03-12T04:25:24.980 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:24.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:24.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.985 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.986 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-03-12T04:25:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-03-12T04:25:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-03-12T04:25:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-03-12T04:25:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-03-12T04:25:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-12T04:25:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-03-12T04:25:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-03-12T04:25:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-03-12T04:25:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-03-12T04:25:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.070 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.073 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.077 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.078 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.079 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.082 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.083 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.085 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-03-12T04:25:25.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.088 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.090 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.092 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.093 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.094 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.097 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.098 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.099 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.104 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-03-12T04:25:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-03-12T04:25:25.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.110 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-03-12T04:25:25.111 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.112 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.113 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.114 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-03-12T04:25:25.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.118 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.119 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.120 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.122 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.123 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.124 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.128 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.129 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.131 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.132 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-03-12T04:25:25.133 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.135 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.136 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-03-12T04:25:25.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.139 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.140 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.142 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.146 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.147 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.152 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-03-12T04:25:25.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-03-12T04:25:25.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.155 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.156 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.158 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.160 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.162 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.163 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.164 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.165 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.167 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.170 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-03-12T04:25:25.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.173 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.177 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.178 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.179 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.180 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.182 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.183 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.185 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.186 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.187 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-03-12T04:25:25.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.189 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.191 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.192 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.194 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.197 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.199 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.204 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.205 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.206 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-03-12T04:25:25.208 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.209 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.211 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.215 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.217 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.218 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 25061, 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "y", 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.222 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.226 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.228 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.229 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.230 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-03-12T04:25:25.233 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-03-12T04:25:25.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-03-12T04:25:25.235 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.236 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.239 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-03-12T04:25:25.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-03-12T04:25:25.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.242 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.244 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.245 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.246 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.249 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.249 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.249 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.249 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.251 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.252 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.255 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-03-12T04:25:25.257 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-03-12T04:25:25.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.259 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.261 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-03-12T04:25:25.262 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.263 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.264 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.266 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-12T04:25:25.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.273 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.274 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.275 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-03-12T04:25:25.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-03-12T04:25:25.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-12T04:25:25.280 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.282 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.283 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-12T04:25:25.284 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.285 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.286 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.291 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.294 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.299 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-03-12T04:25:25.301 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.302 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.303 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-12T04:25:25.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.305 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.307 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-03-12T04:25:25.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-03-12T04:25:25.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.310 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.311 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.316 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.318 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.319 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.320 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.322 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-03-12T04:25:25.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-12T04:25:25.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-03-12T04:25:25.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.327 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.329 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.331 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.334 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.338 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-12T04:25:25.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-12T04:25:25.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-12T04:25:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-12T04:25:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-12T04:25:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-03-12T04:25:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-12T04:25:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-03-12T04:25:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-03-12T04:25:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-03-12T04:25:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-03-12T04:25:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-03-12T04:25:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-03-12T04:25:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-03-12T04:25:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-12T04:25:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.398 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-12T04:25:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-03-12T04:25:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-03-12T04:25:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-03-12T04:25:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-03-12T04:25:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-03-12T04:25:25.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-03-12T04:25:25.442 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-03-12T04:25:25.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.446 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.447 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.449 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.453 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.453 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.453 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.453 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.454 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.456 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.457 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.458 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.461 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.471 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.473 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.475 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.476 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.478 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.479 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.480 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.482 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.483 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-12T04:25:25.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.498 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.499 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.500 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.502 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.504 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.505 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.507 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-03-12T04:25:25.508 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.512 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.513 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.514 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.523 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.527 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-03-12T04:25:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-03-12T04:25:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-03-12T04:25:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-03-12T04:25:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-03-12T04:25:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-03-12T04:25:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.559 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-12T04:25:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-03-12T04:25:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-03-12T04:25:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-03-12T04:25:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-03-12T04:25:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard", 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "restful", 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_schedule" 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.622 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.623 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.624 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.626 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-03-12T04:25:25.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.632 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.634 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-03-12T04:25:25.635 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-03-12T04:25:25.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-03-12T04:25:25.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-03-12T04:25:25.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.642 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.644 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.645 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.646 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-12T04:25:25.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-03-12T04:25:25.649 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-12T04:25:25.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-12T04:25:25.657 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-12T04:25:25.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-03-12T04:25:25.660 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-12T04:25:25.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-03-12T04:25:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-03-12T04:25:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-12T04:25:25.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-03-12T04:25:25.674 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-03-12T04:25:25.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.678 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-03-12T04:25:25.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-03-12T04:25:25.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.682 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.684 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.687 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.688 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.691 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-12T04:25:25.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-12T04:25:25.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.702 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.705 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-12T04:25:25.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-03-12T04:25:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-03-12T04:25:25.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.714 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.715 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-03-12T04:25:25.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-12T04:25:25.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.727 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.728 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-12T04:25:25.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-12T04:25:25.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-03-12T04:25:25.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-12T04:25:25.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.750 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-03-12T04:25:25.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-12T04:25:25.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.761 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.763 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.764 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.766 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.773 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.774 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.777 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.781 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.786 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-03-12T04:25:25.789 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.790 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.791 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.792 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-03-12T04:25:25.797 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.799 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.804 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.809 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.813 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.815 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.816 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.818 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.820 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.822 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-03-12T04:25:25.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.827 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-03-12T04:25:25.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.833 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.834 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-03-12T04:25:25.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.840 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.847 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.853 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.857 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-03-12T04:25:25.861 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.864 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.866 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.871 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.872 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.875 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.877 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.883 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.884 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.886 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-03-12T04:25:25.887 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.888 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-03-12T04:25:25.889 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.890 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.892 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.894 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.895 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-03-12T04:25:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-03-12T04:25:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-03-12T04:25:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-12T04:25:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-03-12T04:25:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-03-12T04:25:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-12T04:25:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.990 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.994 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:25.995 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.996 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.997 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:25.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:25.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-03-12T04:25:26.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-03-12T04:25:26.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-03-12T04:25:26.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:26.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.007 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-03-12T04:25:26.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:26.009 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard": "https://172.21.15.42:8443/" 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "always_on_modules": { 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "octopus": [ 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-03-12T04:25:26.014 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "pacific": [ 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:26.015 INFO:teuthology.orchestra.run.smithi042.stdout: "quincy": [ 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "reef": [ 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-03-12T04:25:26.016 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "squid": [ 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-03-12T04:25:26.017 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "last_failure_osd_epoch": 90, 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "active_clients": [ 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1268597852 2024-03-12T04:25:26.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "libcephsqlite", 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3920806808 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-03-12T04:25:26.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3060865637 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 605415832 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2006438442 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-03-12T04:25:26.021 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-03-12T04:25:26.022 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4015940271 2024-03-12T04:25:26.022 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.022 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.022 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:26.022 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-03-12T04:25:26.022 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:25:26.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:25 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1457571983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:25:26.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:25 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:25:26.024 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:25 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1960266399' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-12T04:25:26.024 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:25 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1457571983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:25:26.024 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:25 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:25:26.024 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:25 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1960266399' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-12T04:25:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9d200a1ba0 msgr2=0x7f9d200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9d200a1ba0 0x7f9d200a4060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009900 tx=0x7f9d2c005fd0 comp rx=0 tx=0).stop 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101380 msgr2=0x7f9d3819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101380 0x7f9d3819d640 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800b4a0 tx=0x7f9d2800b960 comp rx=0 tx=0).stop 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 shutdown_connections 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9d200a1ba0 0x7f9d200a4060 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d38197700 0x7f9d38197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d38106450 0x7f9d3819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 --2- 172.21.15.42:0/1960266399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d38101380 0x7f9d3819d640 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 >> 172.21.15.42:0/1960266399 conn(0x7f9d380fcc20 msgr2=0x7f9d380fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 shutdown_connections 2024-03-12T04:25:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:25:24.814+0000 7f9d3db65640 1 -- 172.21.15.42:0/1960266399 wait complete. 2024-03-12T04:25:26.031 DEBUG:tasks.check_counter:Getting stats from a 2024-03-12T04:25:26.031 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-12T04:25:26.032 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:25 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1457571983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:25:26.032 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:25 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T04:25:26.032 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:25 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1960266399' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-12T04:25:26.320 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:25:26.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:26 smithi042 ceph-mon[32260]: pgmap v318: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 100 B/s wr, 13 op/s 2024-03-12T04:25:26.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:26 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:26 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:26 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:26 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:26 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.690 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:26 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:26 smithi119 ceph-mon[36119]: pgmap v318: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 100 B/s wr, 13 op/s 2024-03-12T04:25:26.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:26 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:26 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:26 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:26 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:26 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:26 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:26 smithi136 ceph-mon[35571]: pgmap v318: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 100 B/s wr, 13 op/s 2024-03-12T04:25:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:26 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:26 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:26 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:26 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:26 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:26.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:26 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:27.692 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: Unable to set osd_memory_target on smithi136 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:25:27.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:25:27.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:27 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: Unable to set osd_memory_target on smithi136 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:25:27.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:27 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: Unable to set osd_memory_target on smithi136 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T04:25:27.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:27 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 82, 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 116, 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 17347, 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 387424, 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 52, 2024-03-12T04:25:28.272 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.025794500, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003688075, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.015887889, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000410532, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 110, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.015508715, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000140988 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 108, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000021144, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000195 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.273 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 5488, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 6576 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 796, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 873, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1773825, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 511149, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 12, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.107170150, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.040887707, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.068059719, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.008879882, 2024-03-12T04:25:28.274 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 874, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.055697252, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000063726 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1114, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000301400, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000270 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1607008, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 131104 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:28.275 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 117, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 269, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 45235, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 487038, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 9, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.020913797, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.011641210, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.008559361, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000265512, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 269, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.052818089, 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000196349 2024-03-12T04:25:28.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 193, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000038269, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000198 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 438944 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 1, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-MDSRank": { 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2024-03-12T04:25:28.277 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 58, 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001512884, 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000026084 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 2, 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000006336, 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000003168 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-03-12T04:25:28.278 INFO:teuthology.orchestra.run.smithi042.stdout: "request": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "reply": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "reply_latency": { 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_reply": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "forward": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_commit": 11, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_split": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_merge": 0, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes": 15, 2024-03-12T04:25:28.279 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_top": 0, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_bottom": 15, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pinned": 13, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_expired": 4, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_with_caps": 0, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": 0, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "subtrees": 3, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse": 6, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_hit": 3, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_forward": 0, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_discover": 3, 2024-03-12T04:25:28.280 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_lock": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "load_cent": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "q": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "exported": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "exported_inodes": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "imported": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "imported_inodes": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rfiles": 4, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rbytes": 234, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rsnaps": 6, 2024-03-12T04:25:28.281 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps": 0, 2024-03-12T04:25:28.282 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "process_request_cap_release": 0 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache": { 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update_receipt": 9, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_try_discover": 6, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_send_discover": 7, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_created": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_enqueued": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_migrated": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_started": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_completed": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_flush": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_inodestats": 0 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log": { 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "evlrg": 0, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "evadd": 6, 2024-03-12T04:25:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "evex": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "evtrm": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "ev": 6, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "evexg": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "evexd": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "segadd": 2, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "segex": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "segtrm": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "seg": 2, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "segmjr": 2, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "segexg": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "segexd": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "expos": 4194304, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "wrpos": 4196308, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rdpos": 0, 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "jlat": { 2024-03-12T04:25:28.285 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 5, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.010753297, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.002150659 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "replayed": 0 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mem": { 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 19, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "ino+": 21, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "ino-": 2, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "dir": 18, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "dir+": 21, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "dir-": 3, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "dn": 15, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "dn+": 19, 2024-03-12T04:25:28.286 INFO:teuthology.orchestra.run.smithi042.stdout: "dn-": 4, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "cap": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "cap+": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "cap-": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rss": 51236, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "heap": 207228 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_server": { 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_client_request": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_request": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_session": 21, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_peer_request": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "req_create_latency": { 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.287 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getattr_latency": { 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "req_link_latency": { 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.288 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookup_latency": { 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupino_latency": { 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupname_latency": { 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lssnap_latency": { 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mkdir_latency": { 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.291 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mknod_latency": { 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mksnap_latency": { 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.292 INFO:teuthology.orchestra.run.smithi042.stdout: "req_open_latency": { 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "req_readdir_latency": { 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rename_latency": { 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:28.293 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmdir_latency": { 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:28.294 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setattr_latency": { 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:28.295 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setlayout_latency": { 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setxattr_latency": { 2024-03-12T04:25:28.296 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "req_symlink_latency": { 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "req_unlink_latency": { 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:28.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessions": { 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "session_count": 2, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "session_add": 3, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "session_remove": 1, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_open": 2, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_stale": 0, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "total_load": 0, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "average_load": 0, 2024-03-12T04:25:28.298 INFO:teuthology.orchestra.run.smithi042.stdout: "avg_session_uptime": 598, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:28.300 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 246263, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 83, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:28.301 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 165, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 78749, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 312, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-03-12T04:25:28.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": { 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 43, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 12486, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 33, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 43, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 43, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.267997550, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.006232501 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-03-12T04:25:28.303 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 43, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 65 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 43, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 43, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 10, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 5, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 9, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-03-12T04:25:28.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 6, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 20, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:28.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-03-12T04:25:28.306 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 98, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 11, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 9, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 9, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 14, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 1 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "oft": { 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_objs": 1, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_updates": 0, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_removes": 0 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "osd-slow-ops": { 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_ops_count": 0 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "purge_queue": { 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing": 0, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed": 0, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:28.308 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 991, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1730282, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 991, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 991, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1730282, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 43, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 12486, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 43, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 12486, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 43, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 43, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 43, 2024-03-12T04:25:28.311 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 43, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:28.312 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle-0x564268dda6e0": { 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 6, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2004, 2024-03-12T04:25:28.313 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 6, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 5, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2004, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:25:28.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:28 smithi042 ceph-mon[32260]: Adjusting osd_memory_target on smithi042 to 177.2M 2024-03-12T04:25:28.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:28 smithi042 ceph-mon[32260]: Unable to set osd_memory_target on smithi042 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-12T04:25:28.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:28 smithi042 ceph-mon[32260]: pgmap v319: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-03-12T04:25:28.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:28 smithi119 ceph-mon[36119]: Adjusting osd_memory_target on smithi042 to 177.2M 2024-03-12T04:25:28.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:28 smithi119 ceph-mon[36119]: Unable to set osd_memory_target on smithi042 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-12T04:25:28.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:28 smithi119 ceph-mon[36119]: pgmap v319: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-03-12T04:25:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:28 smithi136 ceph-mon[35571]: Adjusting osd_memory_target on smithi042 to 177.2M 2024-03-12T04:25:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:28 smithi136 ceph-mon[35571]: Unable to set osd_memory_target on smithi042 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-12T04:25:28.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:28 smithi136 ceph-mon[35571]: pgmap v319: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-03-12T04:25:29.079 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-12T04:25:29.079 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-12T04:25:29.079 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-12T04:25:29.079 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=9 2024-03-12T04:25:29.079 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-03-12T04:25:29.080 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:29.080 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:29.080 DEBUG:tasks.check_counter:Getting stats from d 2024-03-12T04:25:29.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-12T04:25:29.332 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:25:30.468 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:30 smithi042 ceph-mon[32260]: pgmap v320: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:30.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:30 smithi119 ceph-mon[36119]: pgmap v320: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:30.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:30 smithi136 ceph-mon[35571]: pgmap v320: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.001554404, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000037180, 2024-03-12T04:25:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000650580, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000764678, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000254892 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-03-12T04:25:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 161, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 183040, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.014672042, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.010433273, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001797923, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 158, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.012486080, 2024-03-12T04:25:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000079025 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 183040 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 223, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 212, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1563936, 2024-03-12T04:25:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 126816, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.046334917, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.014176879, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.037986814, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.004425069, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 211, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.009822324, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000046551 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1563936, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 126816 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148977, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 225, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1532507, 2024-03-12T04:25:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 225, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 225, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1532507, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:31.041 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:25:31.750 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-12T04:25:31.750 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-12T04:25:31.751 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-12T04:25:31.751 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-12T04:25:31.751 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-12T04:25:31.751 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-12T04:25:31.751 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-12T04:25:31.751 DEBUG:tasks.check_counter:Getting stats from g 2024-03-12T04:25:31.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-12T04:25:32.001 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:25:32.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:32 smithi119 ceph-mon[36119]: pgmap v321: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:32.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:32 smithi136 ceph-mon[35571]: pgmap v321: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:32.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:32 smithi042 ceph-mon[32260]: pgmap v321: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 988, 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 976, 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1843904, 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 360960, 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-03-12T04:25:33.842 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.128979101, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.049487422, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.054470874, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.027220462, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 975, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.047663098, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000048885 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.843 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1591232, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 127088 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 1402, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 1557, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 360828, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 875557, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 7, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.101345717, 2024-03-12T04:25:33.844 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.033442268, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.024987164, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.034605805, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1555, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.052472104, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000033744 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.845 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 441344 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1629, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1256, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 4, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.001065221, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000058052, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000847695, 2024-03-12T04:25:33.846 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000065449, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001862242, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000465560 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-03-12T04:25:33.847 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 1, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-MDSRank": { 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3636, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.092504016, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000025441 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.848 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "request": 0, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "reply": 0, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "reply_latency": { 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_reply": 0, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "forward": 0, 2024-03-12T04:25:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_commit": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_split": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_merge": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_top": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_bottom": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pinned": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_expired": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_with_caps": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": 0, 2024-03-12T04:25:33.850 INFO:teuthology.orchestra.run.smithi042.stdout: "subtrees": 1, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_hit": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_forward": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_discover": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_lock": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "load_cent": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "q": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "exported": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "exported_inodes": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "imported": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "imported_inodes": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:33.851 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rfiles": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rbytes": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rsnaps": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:33.852 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "process_request_cap_release": 0 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache": { 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update_receipt": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_try_discover": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_send_discover": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:33.853 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_created": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_enqueued": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_migrated": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_started": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_completed": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_flush": 0, 2024-03-12T04:25:33.854 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_inodestats": 0 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log": { 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "evlrg": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "evadd": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "evex": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "evtrm": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "ev": 6, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "evexg": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "evexd": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "segadd": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "segex": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "segtrm": 0, 2024-03-12T04:25:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: "seg": 2, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "segmjr": 2, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "segexg": 0, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "segexd": 0, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "expos": 4194304, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "wrpos": 4196308, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "rdpos": 4196167, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "jlat": { 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "replayed": 6 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mem": { 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 3, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "ino+": 3, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "ino-": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "dir": 1, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "dir+": 1, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "dir-": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "dn": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "dn+": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "dn-": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "cap": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "cap+": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "cap-": 0, 2024-03-12T04:25:33.857 INFO:teuthology.orchestra.run.smithi042.stdout: "rss": 49448, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "heap": 215420 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_server": { 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_client_request": 0, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_request": 0, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_session": 0, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_peer_request": 0, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "req_create_latency": { 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.858 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getattr_latency": { 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "req_link_latency": { 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookup_latency": { 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupino_latency": { 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.860 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupname_latency": { 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lssnap_latency": { 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mkdir_latency": { 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.865 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mknod_latency": { 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mksnap_latency": { 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "req_open_latency": { 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.866 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "req_readdir_latency": { 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rename_latency": { 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.867 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmdir_latency": { 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.868 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setattr_latency": { 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:33.869 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setlayout_latency": { 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.870 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setxattr_latency": { 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "req_symlink_latency": { 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "req_unlink_latency": { 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessions": { 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "session_count": 2, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "session_add": 3, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "session_remove": 1, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_open": 3, 2024-03-12T04:25:33.872 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_stale": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "total_load": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "average_load": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "avg_session_uptime": 2531, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:33.873 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:33.874 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:33.875 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 197490, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 56, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 165, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 8320, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 13, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": { 2024-03-12T04:25:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 2152, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 21, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 3, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 2152, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 2152, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 1.318988400, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000612912 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-03-12T04:25:33.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 2152, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 2153 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 2152, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 2152, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 1390, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 760, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-03-12T04:25:33.879 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 1, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-03-12T04:25:33.880 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-03-12T04:25:33.881 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 98, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 11, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 6, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 6, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 2, 2024-03-12T04:25:33.882 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "oft": { 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_objs": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_updates": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_removes": 0 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "osd-slow-ops": { 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_ops_count": 0 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "purge_queue": { 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed": 0, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 2390, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1976190, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 2390, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2390, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1976190, 2024-03-12T04:25:33.884 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 2152, 2024-03-12T04:25:33.885 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 2009, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2152, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2009, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-03-12T04:25:33.886 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 2152, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 2152, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2152, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2152, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:33.887 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle-0x564918932960": { 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:33.889 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:33.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:33.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:33.890 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:25:34.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:34 smithi042 ceph-mon[32260]: pgmap v322: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:34.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:34 smithi119 ceph-mon[36119]: pgmap v322: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:34.704 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-12T04:25:34.704 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-12T04:25:34.704 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-12T04:25:34.704 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-03-12T04:25:34.704 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-12T04:25:34.704 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:34.704 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:34.704 DEBUG:tasks.check_counter:Getting stats from j 2024-03-12T04:25:34.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-12T04:25:34.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:34 smithi136 ceph-mon[35571]: pgmap v322: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:34.951 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T04:25:36.682 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:36 smithi042 ceph-mon[32260]: pgmap v323: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-03-12T04:25:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.001048953, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000225800, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T04:25:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 225, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 214, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1564480, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 128064, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.044519903, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.014161021, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.033884874, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.005234441, 2024-03-12T04:25:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 213, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.010304791, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000048379 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1564480, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 128064 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 6, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 165, 2024-03-12T04:25:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 3312, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 183840, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.014542474, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.009911998, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002456385, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 163, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.015640034, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000095951 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 183840 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148995, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 31, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-03-12T04:25:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 227, 2024-03-12T04:25:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1532767, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 227, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 227, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1532767, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-03-12T04:25:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-03-12T04:25:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-03-12T04:25:36.705 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-03-12T04:25:36.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:36 smithi119 ceph-mon[36119]: pgmap v323: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:36.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:36 smithi136 ceph-mon[35571]: pgmap v323: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:37.519 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-12T04:25:37.519 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-12T04:25:37.519 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-12T04:25:37.520 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-12T04:25:37.520 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-12T04:25:37.520 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-12T04:25:37.520 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-12T04:25:37.520 DEBUG:tasks.check_counter:Getting stats from b 2024-03-12T04:25:37.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-12T04:25:37.772 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:25:38.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:38 smithi119 ceph-mon[36119]: pgmap v324: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:38.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:38 smithi136 ceph-mon[35571]: pgmap v324: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:38.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:38 smithi042 ceph-mon[32260]: pgmap v324: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-12T04:25:39.717 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:39.717 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1218, 2024-03-12T04:25:39.717 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1245, 2024-03-12T04:25:39.717 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 2053910, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 733633, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 12, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.169565506, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.055657231, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.078243614, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.027134898, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1242, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.284519870, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000229082 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:39.718 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1059, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000241140, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000227 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1637392, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 468880 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 2486, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1136, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1084483, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 654244, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 26, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.302905537, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.065715290, 2024-03-12T04:25:39.719 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.118560000, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.111881326, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1133, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.379986918, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000335381 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3082, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000607106, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000196 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.720 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 398048, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 250368 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1944, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1691, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 876459, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 2847832, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 153, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.233305699, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.055779967, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.109409967, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.058032812, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1692, 2024-03-12T04:25:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.286881348, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000169551 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2351, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000425958, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000181 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 45216, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 1424 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-03-12T04:25:39.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 559, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.191945533, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000343373 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 82, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.003223787, 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000039314 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 159, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 159, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 159, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.157299722, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000989306 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 12, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-03-12T04:25:39.724 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 38, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 1, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 37, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 36, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 3, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 3, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 2, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 219, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 207, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-03-12T04:25:39.725 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 223, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 4, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 234, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 6, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 223, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:39.726 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 139, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 19, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 120, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 84, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 21, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 31, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 102, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 77, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 2, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 34 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.727 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 17, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 36, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 2, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 6, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 4, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:39.728 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 19, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 217, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 217, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-03-12T04:25:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 2, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 5150935, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 177, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.688954412, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003892397 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.730 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 29, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 52, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 23, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 20, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 20, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 0, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 38, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 42, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 4, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 3, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 22, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 19, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 59004, 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 215420 2024-03-12T04:25:39.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 215, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 1, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 159, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 32, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 2, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 8, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.009441861, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001180232 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 99, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.018973862, 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000191655 2024-03-12T04:25:39.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000529639, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000264819 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.733 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.734 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001453491, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000145349 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-03-12T04:25:39.735 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 8, 2024-03-12T04:25:39.736 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.012169670, 2024-03-12T04:25:39.736 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001521208 2024-03-12T04:25:39.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.736 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-03-12T04:25:39.736 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.736 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.736 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.049376459, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004937645 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-03-12T04:25:39.737 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.002769186, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001384593 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.003710906, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001855453 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:39.738 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.014364954, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.007182477 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.017302413, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004325603 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:39.739 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001038047, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001038047 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 7, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.018744491, 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002677784 2024-03-12T04:25:39.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.007424743, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001856185 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:39.742 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 5, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 3, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 27, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 13, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 611, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.743 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:39.744 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:39.745 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 363665, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 145, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 440, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 5, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-03-12T04:25:39.746 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 132588, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 1110, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 1091, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 974970, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 37, 2024-03-12T04:25:39.747 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 1091, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1091, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 24.333393752, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.022303752 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1091, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1138 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 1091, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 262, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 829, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-03-12T04:25:39.748 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 11, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 20, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 178, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 28, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 579, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 242, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 22, 2024-03-12T04:25:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-03-12T04:25:39.750 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 98, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 12, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-03-12T04:25:39.751 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 49, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 0, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 9 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 67, 2024-03-12T04:25:39.752 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 4, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 67, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 4, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 5644, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 3441944, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 5644, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:39.753 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 5644, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 3441944, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:39.754 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 1091, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 975406, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 1091, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 975406, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:39.755 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 1091, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 1091, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 1091, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1091, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 4, 2024-03-12T04:25:39.756 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 436, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 4, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 1, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 436, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x56055f7a66e0": { 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:39.757 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 217, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 956631, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 217, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 177, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 956631, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-12T04:25:40.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:40 smithi119 ceph-mon[36119]: pgmap v325: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:40.491 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-12T04:25:40.491 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-12T04:25:40.491 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=17 2024-03-12T04:25:40.491 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-12T04:25:40.491 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-12T04:25:40.491 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-12T04:25:40.491 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-12T04:25:40.491 DEBUG:tasks.check_counter:Getting stats from e 2024-03-12T04:25:40.492 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-12T04:25:40.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:40 smithi136 ceph-mon[35571]: pgmap v325: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:25:40.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:40 smithi042 ceph-mon[32260]: pgmap v325: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:42.485 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1007, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 992, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1865712, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 367814, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 5, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 1, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.130908397, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.049742956, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.053004913, 2024-03-12T04:25:42.486 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.027713643, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 991, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.050800342, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000051261 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1609680, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 130832 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:42.487 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 8, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 167, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 3868, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 444332, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 6, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 4, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.011114061, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.006629066, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.002481215, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000072835, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 165, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.015774348, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000095602 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:42.488 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 443648 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1458, 2024-03-12T04:25:42.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1460, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 374041, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 453445, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.093854689, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.021728004, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.025260204, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.040662278, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1458, 2024-03-12T04:25:42.490 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.045101833, 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000030934 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.491 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3730, 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.091703883, 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000024585 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:42.492 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 0, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 0, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-03-12T04:25:42.493 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2024-03-12T04:25:42.494 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 1, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:42.495 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:42.496 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-03-12T04:25:42.497 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:42.498 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 8, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 0, 2024-03-12T04:25:42.499 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 4196840, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 4196699, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.500 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 8 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 3, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 3, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 1, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 1, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 0, 2024-03-12T04:25:42.501 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 52544, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 215420 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 0, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 0, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 0, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.502 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.503 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.504 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.507 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-03-12T04:25:42.507 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.507 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.507 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.507 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-03-12T04:25:42.507 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-03-12T04:25:42.508 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.509 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.510 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-03-12T04:25:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 4, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 3, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 0, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 0, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 2580, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:42.514 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:42.515 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:42.516 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 181353, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 48, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:42.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 8320, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 13, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-03-12T04:25:42.518 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 2222, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 21, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 3, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 2222, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2222, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.233968985, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000555341 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2222, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2223 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.519 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 2222, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 2222, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 1449, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 771, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-03-12T04:25:42.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 1, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-03-12T04:25:42.521 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-03-12T04:25:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 98, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 11, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 4, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 4, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 0, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 2, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 0 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 0, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:42.523 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.524 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 2469, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 2005899, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 2469, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2469, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2005899, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.525 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 2222, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 2541, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2222, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2541, 2024-03-12T04:25:42.526 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 2222, 2024-03-12T04:25:42.527 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 2222, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2222, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2222, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:42.528 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x55bac50ea960": { 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:42.529 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:42.531 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-12T04:25:42.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:42 smithi119 ceph-mon[36119]: pgmap v326: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:42.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:42 smithi136 ceph-mon[35571]: pgmap v326: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:42.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:42 smithi042 ceph-mon[32260]: pgmap v326: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:43.186 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-12T04:25:43.186 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-12T04:25:43.186 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-12T04:25:43.186 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-12T04:25:43.186 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-12T04:25:43.186 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:43.186 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:43.186 DEBUG:tasks.check_counter:Getting stats from h 2024-03-12T04:25:43.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-12T04:25:43.425 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:25:44.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:44 smithi119 ceph-mon[36119]: pgmap v327: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:44.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:44 smithi136 ceph-mon[35571]: pgmap v327: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:44.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:44 smithi042 ceph-mon[32260]: pgmap v327: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 900, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 958, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1827766, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 524369, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 9, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.123867421, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.048646917, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.072718167, 2024-03-12T04:25:45.118 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.009728583, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 962, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.064341699, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000066883 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1255, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000280536, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000223 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1625120, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 135456 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.119 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 122, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 185, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 35140, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 506894, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 45, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.025546146, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.005859294, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.018325423, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000281786, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 182, 2024-03-12T04:25:45.120 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.015169978, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000083351 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 192, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000034784, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000181 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 5440, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 146112 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 38, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 199, 2024-03-12T04:25:45.121 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 7977, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 405929, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 19, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.014846306, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.006212596, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.006717494, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000348783, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 193, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.022457215, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000116358 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:45.122 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 57, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000011642, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000204 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 313184 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:45.123 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 64, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001525372, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000023833 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000006080, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000003040 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 10, 2024-03-12T04:25:45.124 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 7, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 7, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001907923, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000272560 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 3, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 11, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 16, 2024-03-12T04:25:45.125 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 0, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 16, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 13, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 5, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 2, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 2, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 3, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 24, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 15, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 1, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 5, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:45.126 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 4, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 234, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 6, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:45.127 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:45.128 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 12, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 8, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 10, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-03-12T04:25:45.129 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-03-12T04:25:45.130 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 8, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 8, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 2, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-03-12T04:25:45.131 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 4196840, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 7, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.013285186, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001897883 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 20, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 22, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 2, 2024-03-12T04:25:45.132 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 19, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 21, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 2, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 16, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 21, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 5, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 2, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 4, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 2, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 51892, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 207228 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 11, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 10, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 25, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 5, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001387397, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000277479 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:45.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000520526, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000260263 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.135 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.136 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.137 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-03-12T04:25:45.138 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.138 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.138 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.139 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.140 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.141 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:45.142 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-03-12T04:25:45.143 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.144 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 4, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 5, 2024-03-12T04:25:45.145 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 1, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 615, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:45.146 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:45.147 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 256390, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 98, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 88, 2024-03-12T04:25:45.148 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 1, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 83693, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 339, 2024-03-12T04:25:45.149 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 47, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 13202, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 33, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 47, 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-03-12T04:25:45.150 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 47, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.206994842, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004404145 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 47, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 70 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 47, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 0, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 47, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 0, 2024-03-12T04:25:45.151 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 10, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 7, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 10, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 6, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 0, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 20, 2024-03-12T04:25:45.152 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-03-12T04:25:45.153 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 98, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 11, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 8, 2024-03-12T04:25:45.154 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 8, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 16, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 0, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 1 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.155 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 1056, 2024-03-12T04:25:45.156 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1757775, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 1056, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 1056, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1757775, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:45.157 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 47, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 13202, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 47, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 13202, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 47, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 47, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 47, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 47, 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:45.159 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:45.160 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x55cefd7ac6e0": { 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 8, 2024-03-12T04:25:45.161 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 2536, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 8, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 7, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2536, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:45.162 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-12T04:25:45.952 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-12T04:25:45.952 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-12T04:25:45.952 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-12T04:25:45.952 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=12 2024-03-12T04:25:45.952 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-12T04:25:45.952 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:45.952 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:45.952 DEBUG:tasks.check_counter:Getting stats from k 2024-03-12T04:25:45.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-12T04:25:46.193 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T04:25:46.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:46 smithi119 ceph-mon[36119]: pgmap v328: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:46.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:46 smithi136 ceph-mon[35571]: pgmap v328: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:46.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:46 smithi042 ceph-mon[32260]: pgmap v328: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 249, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 234, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1611330, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 133021, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 5, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 1, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.050367286, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.014925914, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.038914202, 2024-03-12T04:25:47.876 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.005661767, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 233, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.017226159, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000073932 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1610496, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 132704 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:47.877 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 700, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 862, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 174058, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 663215, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 7, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.073846848, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.026535302, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.015786674, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.023037421, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 860, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.039761634, 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000046234 2024-03-12T04:25:47.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 447376 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1542, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1544, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 463269, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 479620, 2024-03-12T04:25:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.156113098, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.056291667, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.038985951, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.046628659, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1542, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.063207769, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000040990 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3717, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.109193062, 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000029376 2024-03-12T04:25:47.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 0, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 0, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.882 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 0, 2024-03-12T04:25:47.883 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 1, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-03-12T04:25:47.884 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:47.885 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-03-12T04:25:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-03-12T04:25:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 8, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 0, 2024-03-12T04:25:47.888 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 4196840, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 4196699, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 8 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.889 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 3, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 3, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 1, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 1, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 0, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 49992, 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 215420 2024-03-12T04:25:47.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 0, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 0, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 0, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.891 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.892 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.893 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.894 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-03-12T04:25:47.895 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.895 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.895 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.895 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-03-12T04:25:47.895 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.896 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.897 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:47.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:47.899 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.900 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 4, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 3, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 0, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 0, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 2586, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:47.902 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:47.903 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:47.904 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 190424, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 52, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:47.905 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 8320, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 13, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-03-12T04:25:47.906 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 2237, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 21, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 3, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 2237, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2237, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.401964826, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000626716 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2237, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2238 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.907 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 2237, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 2237, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 1459, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 776, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 0, 2024-03-12T04:25:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 1, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-03-12T04:25:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-03-12T04:25:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 98, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 11, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 5, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 5, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 2, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 0 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-03-12T04:25:47.911 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:47.912 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 2487, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 2009159, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 2487, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2487, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2009159, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.913 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 2237, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 2541, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2237, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2541, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.914 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 2237, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 2237, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2237, 2024-03-12T04:25:47.915 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2237, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:47.916 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x5624bf6a0960": { 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-12T04:25:47.917 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-12T04:25:47.918 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-12T04:25:48.675 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:48 smithi119 ceph-mon[36119]: pgmap v329: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:48.703 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-12T04:25:48.703 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-12T04:25:48.703 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-12T04:25:48.703 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-12T04:25:48.703 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-12T04:25:48.704 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:48.704 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:48.704 DEBUG:tasks.check_counter:Getting stats from c 2024-03-12T04:25:48.704 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-12T04:25:48.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:48 smithi136 ceph-mon[35571]: pgmap v329: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:48.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:48 smithi042 ceph-mon[32260]: pgmap v329: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:48.954 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:25:50.694 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:50 smithi136 ceph-mon[35571]: pgmap v330: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:50.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:50 smithi119 ceph-mon[36119]: pgmap v330: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:50.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:50 smithi042 ceph-mon[32260]: pgmap v330: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:50.844 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-03-12T04:25:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 347, 2024-03-12T04:25:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 347, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1630471, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 536192, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 51, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 1, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.070427179, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.018905706, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.050678270, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.006031054, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 345, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.027149645, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000078694 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 82, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000013186, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000160 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1618704, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 137232 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 116, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 276, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 45021, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 493397, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 10, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.019859829, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.010758031, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.008063980, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000295203, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 274, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.030523671, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000111400 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 183, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000036117, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000197 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 446400 2024-03-12T04:25:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 571, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 673, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 184100, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 384707, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 11, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.066259653, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.026008399, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.036846135, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005078371, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 672, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.040591514, 2024-03-12T04:25:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000060404 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1190, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000294173, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000247 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 4560, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 5600 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 49, 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.002088598, 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000042624 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000009792, 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000004896 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 1, 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-03-12T04:25:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-03-12T04:25:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 1, 2024-03-12T04:25:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 11, 2024-03-12T04:25:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-03-12T04:25:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-03-12T04:25:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 16, 2024-03-12T04:25:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 16, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 13, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 4, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 3, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 10, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 4, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 1, 2024-03-12T04:25:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 5, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 1000, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 1, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-03-12T04:25:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 14, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 9, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 9, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-03-12T04:25:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 6, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 6, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-03-12T04:25:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-03-12T04:25:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 2, 2024-03-12T04:25:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-03-12T04:25:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-03-12T04:25:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-03-12T04:25:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-03-12T04:25:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-03-12T04:25:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-03-12T04:25:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-03-12T04:25:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4196308, 2024-03-12T04:25:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-03-12T04:25:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-03-12T04:25:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 5, 2024-03-12T04:25:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.008522213, 2024-03-12T04:25:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001704442 2024-03-12T04:25:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-03-12T04:25:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-03-12T04:25:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 20, 2024-03-12T04:25:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 22, 2024-03-12T04:25:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 2, 2024-03-12T04:25:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 19, 2024-03-12T04:25:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 22, 2024-03-12T04:25:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 3, 2024-03-12T04:25:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 16, 2024-03-12T04:25:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 20, 2024-03-12T04:25:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 4, 2024-03-12T04:25:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-03-12T04:25:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-03-12T04:25:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-03-12T04:25:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 54208, 2024-03-12T04:25:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 207228 2024-03-12T04:25:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-03-12T04:25:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 1, 2024-03-12T04:25:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 1, 2024-03-12T04:25:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 21, 2024-03-12T04:25:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-03-12T04:25:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-03-12T04:25:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-03-12T04:25:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-03-12T04:25:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-03-12T04:25:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-03-12T04:25:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-03-12T04:25:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 622, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 262531, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 88, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 165, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 82909, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 329, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-03-12T04:25:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 43, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 12486, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 33, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 43, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 43, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.207995306, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004837100 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 43, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 65 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 43, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 0, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 43, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 10, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 0, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 5, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 9, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-03-12T04:25:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 6, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 20, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-03-12T04:25:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-03-12T04:25:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 98, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 11, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 8, 2024-03-12T04:25:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 8, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 14, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 1030, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1749387, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 1030, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1030, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1749387, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 43, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 12486, 2024-03-12T04:25:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 43, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 12486, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 43, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 43, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 43, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 43, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x5636d6eec6e0": { 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 6, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 2004, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 6, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 5, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2004, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:50.898 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-03-12T04:25:51.602 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-12T04:25:51.602 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-12T04:25:51.602 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-12T04:25:51.602 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=14 2024-03-12T04:25:51.602 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-12T04:25:51.602 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:51.603 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:51.603 DEBUG:tasks.check_counter:Getting stats from f 2024-03-12T04:25:51.603 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-12T04:25:51.849 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:25:52.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:52 smithi119 ceph-mon[36119]: pgmap v331: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:52.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:52 smithi136 ceph-mon[35571]: pgmap v331: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:52.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:52 smithi042 ceph-mon[32260]: pgmap v331: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1667, 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1667, 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1451280, 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 517651, 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.156727410, 2024-03-12T04:25:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.050562948, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.040888858, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.049808359, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1666, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.068192326, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000040931 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-12T04:25:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 6, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 169, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1976, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 449662, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.011415235, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.006246481, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.003280249, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000132710, 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 166, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.020192320, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000121640 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448352 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1003, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 988, 2024-03-12T04:25:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1796185, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 367046, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 3, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.117855174, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.040574903, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.049962265, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.029326824, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 987, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.043863492, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000044441 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1610384, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 132240 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4000, 2024-03-12T04:25:53.593 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.109435571, 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000027358 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-03-12T04:25:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 31, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-03-12T04:25:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 31, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 31, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-03-12T04:25:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-03-12T04:25:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-03-12T04:25:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 217, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 5150935, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 5150823, 2024-03-12T04:25:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 217 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 20, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 24, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 4, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 11, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 11, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 0, 2024-03-12T04:25:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 31, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 31, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 53304, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215420 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-03-12T04:25:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.604 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-03-12T04:25:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 5, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 3, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 4, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 3580, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 189433, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 52, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 165, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 89292, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 583, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-03-12T04:25:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 2422, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 26, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 4, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 2422, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2422, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.267971301, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000523522 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2422, 2024-03-12T04:25:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2423 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 2422, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 2422, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 1527, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 892, 2024-03-12T04:25:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 2, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-03-12T04:25:53.623 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-03-12T04:25:53.624 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 98, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 11, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 5, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 5, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-03-12T04:25:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 2672, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 2993299, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 2672, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2672, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2993299, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2422, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 956641, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2422, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 956641, 2024-03-12T04:25:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2422, 2024-03-12T04:25:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 2422, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2422, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2422, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55a459120960": { 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:53.632 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-03-12T04:25:54.439 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-12T04:25:54.439 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-12T04:25:54.439 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-12T04:25:54.439 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-12T04:25:54.440 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-03-12T04:25:54.440 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:54.440 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:54.440 DEBUG:tasks.check_counter:Getting stats from i 2024-03-12T04:25:54.440 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-12T04:25:54.681 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:25:54.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:54 smithi119 ceph-mon[36119]: pgmap v332: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:54.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:54 smithi136 ceph-mon[35571]: pgmap v332: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:54.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:54 smithi042 ceph-mon[32260]: pgmap v332: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:56.604 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:56 smithi136 ceph-mon[35571]: pgmap v333: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 605, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 702, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 192385, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 405662, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 13, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.072311956, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.026070750, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.037814263, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005738280, 2024-03-12T04:25:56.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 701, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.045529009, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000064948 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1228, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000352602, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000287 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 4560, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 5600 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.606 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 125, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 186, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 34561, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 555098, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 52, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.026989476, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.006125817, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.019296928, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000189668, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 183, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.015279820, 2024-03-12T04:25:56.607 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000083496 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 199, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000043427, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000218 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 134560 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 343, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 452, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1650088, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 482760, 2024-03-12T04:25:56.608 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.057485681, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.020748370, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.039236342, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005944461, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 451, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.031816476, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000070546 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 97, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000036175, 2024-03-12T04:25:56.609 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000372 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1626576, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 452256 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 59, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001394588, 2024-03-12T04:25:56.610 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000023637 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000007921, 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000003960 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 5, 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 3, 2024-03-12T04:25:56.611 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001651073, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000550357 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 2, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 11, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 16, 2024-03-12T04:25:56.612 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 16, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 13, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 4, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 2, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 2, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 3, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 17, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 9, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 6, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:56.613 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:56.614 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:56.615 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 2 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 14, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 9, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 10, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-03-12T04:25:56.616 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-03-12T04:25:56.617 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 8, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 8, 2024-03-12T04:25:56.618 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 2, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4196840, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 7, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.012809705, 2024-03-12T04:25:56.619 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001829957 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 20, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 22, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 2, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 19, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 22, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 3, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 16, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 20, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 4, 2024-03-12T04:25:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 2, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 3, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 1, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 52448, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 207228 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 9, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 5, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 25, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 1, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.621 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.622 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-03-12T04:25:56.622 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-03-12T04:25:56.622 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001651073, 2024-03-12T04:25:56.622 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000550357 2024-03-12T04:25:56.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.623 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.624 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.625 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-03-12T04:25:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.627 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.630 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.631 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-03-12T04:25:56.632 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 4, 2024-03-12T04:25:56.633 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 2, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 2, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 2, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 627, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:56.634 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:56.635 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:56.636 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:56.637 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 255457, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 92, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-03-12T04:25:56.638 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 165, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 83709, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 341, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:56.639 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-03-12T04:25:56.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.640 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-03-12T04:25:56.640 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-03-12T04:25:56.640 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 47, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 13263, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 33, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 47, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 47, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.227994869, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004850954 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-03-12T04:25:56.641 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 47, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 70 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 47, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 0, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 47, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 10, 2024-03-12T04:25:56.642 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 7, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 10, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 6, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 20, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:56.643 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-03-12T04:25:56.644 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 98, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 11, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 9, 2024-03-12T04:25:56.645 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 9, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 16, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-03-12T04:25:56.646 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.647 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 1069, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1762825, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 1069, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:56.648 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1069, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1762825, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:56.649 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 47, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 13263, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 47, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 13263, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 47, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 47, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 47, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 47, 2024-03-12T04:25:56.651 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:56.652 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x556d44ca86e0": { 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 8, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 2536, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 8, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 7, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2536, 2024-03-12T04:25:56.654 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:56.655 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:56.655 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:56.655 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:56.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:56.655 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-03-12T04:25:56.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:56 smithi119 ceph-mon[36119]: pgmap v333: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:56.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:56 smithi042 ceph-mon[32260]: pgmap v333: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-12T04:25:57.422 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-12T04:25:57.422 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-12T04:25:57.422 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-03-12T04:25:57.422 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=14 2024-03-12T04:25:57.422 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-12T04:25:57.422 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:25:57.423 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:25:57.423 DEBUG:tasks.check_counter:Getting stats from l 2024-03-12T04:25:57.423 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 7cca013a-e025-11ee-95c8-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-12T04:25:57.663 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T04:25:58.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:25:58 smithi119 ceph-mon[36119]: pgmap v334: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:58.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:25:58 smithi136 ceph-mon[35571]: pgmap v334: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:58.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:25:58 smithi042 ceph-mon[32260]: pgmap v334: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-12T04:25:59.511 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-03-12T04:25:59.511 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1564, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1566, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 469357, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 486405, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.151770215, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.053143155, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.038051134, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.045983513, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1564, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.065565189, 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000041921 2024-03-12T04:25:59.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 259, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 242, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1615966, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 136649, 2024-03-12T04:25:59.513 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.051166187, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.014649619, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.040825720, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005889848, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 242, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.017665991, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000072999 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.514 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1614576, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 135648 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 667, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 834, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 163932, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 657471, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.072673079, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.027311856, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.015956113, 2024-03-12T04:25:59.515 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.022275869, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 831, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.035496553, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000042715 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 450656 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-03-12T04:25:59.516 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3690, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.104987408, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000028451 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.517 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-03-12T04:25:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-03-12T04:25:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 0, 2024-03-12T04:25:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T04:25:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-03-12T04:25:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-03-12T04:25:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 6, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-03-12T04:25:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4196308, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 4196167, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 6 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 3, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 3, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 0, 2024-03-12T04:25:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 1, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 1, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 49200, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215420 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-03-12T04:25:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.531 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-03-12T04:25:59.531 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.531 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.531 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.531 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-03-12T04:25:59.531 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-03-12T04:25:59.532 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-03-12T04:25:59.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-03-12T04:25:59.534 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-03-12T04:25:59.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.536 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-03-12T04:25:59.537 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 3, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 2850, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T04:25:59.538 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T04:25:59.539 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 197473, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 55, 2024-03-12T04:25:59.540 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7936, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 165, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 8320, 2024-03-12T04:25:59.541 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 13, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 2232, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 21, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 3, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 2232, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2232, 2024-03-12T04:25:59.542 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.263971401, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000566295 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2232, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2233 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 2232, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 2232, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 1443, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 787, 2024-03-12T04:25:59.543 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 1, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-03-12T04:25:59.544 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-03-12T04:25:59.545 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 98, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 11, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 6, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 6, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-03-12T04:25:59.546 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-03-12T04:25:59.547 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 2486, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 2009639, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 2486, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2486, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2009639, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:59.549 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2232, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 2009, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2232, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2009, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:59.550 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2232, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 2232, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2232, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2232, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.552 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55db095a8960": { 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-03-12T04:25:59.553 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-03-12T04:25:59.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-03-12T04:25:59.554 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-03-12T04:25:59.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-03-12T04:25:59.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-03-12T04:25:59.554 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-03-12T04:26:00.238 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-12T04:26:00.238 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-12T04:26:00.239 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-12T04:26:00.239 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-12T04:26:00.239 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-12T04:26:00.239 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-12T04:26:00.239 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T04:26:00.239 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-12T04:26:00.249 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-12T04:26:00.249 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:26:00.249 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T04:26:00.270 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:26:00.270 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T04:26:00.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.484+0000 7f5031d0c640 1 -- 172.21.15.42:0/908178213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c106450 msgr2=0x7f502c10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.484+0000 7f5031d0c640 1 --2- 172.21.15.42:0/908178213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c106450 0x7f502c10e020 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5024009cd0 tx=0x7f502402c0f0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.484+0000 7f5031d0c640 1 -- 172.21.15.42:0/908178213 shutdown_connections 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.484+0000 7f5031d0c640 1 --2- 172.21.15.42:0/908178213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c106450 0x7f502c10e020 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.484+0000 7f5031d0c640 1 --2- 172.21.15.42:0/908178213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f502c101d30 0x7f502c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.484+0000 7f5031d0c640 1 --2- 172.21.15.42:0/908178213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c101380 0x7f502c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.484+0000 7f5031d0c640 1 -- 172.21.15.42:0/908178213 >> 172.21.15.42:0/908178213 conn(0x7f502c0fcc20 msgr2=0x7f502c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.485+0000 7f5031d0c640 1 -- 172.21.15.42:0/908178213 shutdown_connections 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.485+0000 7f5031d0c640 1 -- 172.21.15.42:0/908178213 wait complete. 2024-03-12T04:26:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.485+0000 7f5031d0c640 1 Processor -- start 2024-03-12T04:26:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f5031d0c640 1 -- start start 2024-03-12T04:26:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f5031d0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c101380 0x7f502c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f5031d0c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c106450 0x7f502c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f5031d0c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f502c073f00 0x7f502c1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f5031d0c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1a5ef0 con 0x7f502c101380 2024-03-12T04:26:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f5031d0c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1a6060 con 0x7f502c106450 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f5031d0c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1a6220 con 0x7f502c073f00 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c101380 0x7f502c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c101380 0x7f502c073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39492/0 (socket says 172.21.15.42:39492) 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 -- 172.21.15.42:0/1526308072 learned_addr learned my addr 172.21.15.42:0/1526308072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502bfff640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f502c073f00 0x7f502c1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502affd640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c106450 0x7f502c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 -- 172.21.15.42:0/1526308072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f502c073f00 msgr2=0x7f502c1a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f502c073f00 0x7f502c1a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 -- 172.21.15.42:0/1526308072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c106450 msgr2=0x7f502c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c106450 0x7f502c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502b7fe640 1 -- 172.21.15.42:0/1526308072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024009900 con 0x7f502c101380 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.486+0000 7f502affd640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c106450 0x7f502c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.487+0000 7f502b7fe640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c101380 0x7f502c073480 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f501c00d950 tx=0x7f501c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.487+0000 7f5028ff9640 1 -- 172.21.15.42:0/1526308072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c009c30 con 0x7f502c101380 2024-03-12T04:26:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.487+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c1a6500 con 0x7f502c101380 2024-03-12T04:26:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.487+0000 7f5028ff9640 1 -- 172.21.15.42:0/1526308072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f501c009d90 con 0x7f502c101380 2024-03-12T04:26:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.487+0000 7f5028ff9640 1 -- 172.21.15.42:0/1526308072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00f6b0 con 0x7f502c101380 2024-03-12T04:26:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.487+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c1a6a10 con 0x7f502c101380 2024-03-12T04:26:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.488+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c0683c0 con 0x7f502c101380 2024-03-12T04:26:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.489+0000 7f5028ff9640 1 -- 172.21.15.42:0/1526308072 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f501c010aa0 con 0x7f502c101380 2024-03-12T04:26:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.490+0000 7f5028ff9640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f50140a1bd0 0x7f50140a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.490+0000 7f502affd640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f50140a1bd0 0x7f50140a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.490+0000 7f5028ff9640 1 -- 172.21.15.42:0/1526308072 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f501c0cd440 con 0x7f502c101380 2024-03-12T04:26:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.491+0000 7f502affd640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f50140a1bd0 0x7f50140a4090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5020005fd0 tx=0x7f502000b5d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.494+0000 7f5028ff9640 1 -- 172.21.15.42:0/1526308072 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f501c096330 con 0x7f502c101380 2024-03-12T04:26:00.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.687+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f502c109680 con 0x7f502c101380 2024-03-12T04:26:00.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.689+0000 7f5028ff9640 1 -- 172.21.15.42:0/1526308072 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+23076 (secure 0 0 0) 0x7f501c017090 con 0x7f502c101380 2024-03-12T04:26:00.692 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:26:00.692 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:25:15.177682+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1061357309":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/2954927866":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6801/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/3686333836":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/4126408073":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f50140a1bd0 msgr2=0x7f50140a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f50140a1bd0 0x7f50140a4090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5020005fd0 tx=0x7f502000b5d0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c101380 msgr2=0x7f502c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c101380 0x7f502c073480 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f501c00d950 tx=0x7f501c00de10 comp rx=0 tx=0).stop 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 shutdown_connections 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f50140a1bd0 0x7f50140a4090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f502c073f00 0x7f502c1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c106450 0x7f502c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 --2- 172.21.15.42:0/1526308072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f502c101380 0x7f502c073480 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 >> 172.21.15.42:0/1526308072 conn(0x7f502c0fcc20 msgr2=0x7f502c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 shutdown_connections 2024-03-12T04:26:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.691+0000 7f5031d0c640 1 -- 172.21.15.42:0/1526308072 wait complete. 2024-03-12T04:26:00.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:00 smithi119 ceph-mon[36119]: pgmap v335: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:26:00.703 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:00 smithi042 ceph-mon[32260]: pgmap v335: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:26:00.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:26:00.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:26:00.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:00 smithi136 ceph-mon[35571]: pgmap v335: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:26:00.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 -- 172.21.15.42:0/2559390875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 msgr2=0x7f3df410dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2559390875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df410dbc0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009ca0 tx=0x7f3dec02c180 comp rx=0 tx=0).stop 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 -- 172.21.15.42:0/2559390875 shutdown_connections 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2559390875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df410dbc0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2559390875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df4102ce0 0x7f3df410b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2559390875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3df41023c0 0x7f3df41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 -- 172.21.15.42:0/2559390875 >> 172.21.15.42:0/2559390875 conn(0x7f3df4076400 msgr2=0x7f3df4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 -- 172.21.15.42:0/2559390875 shutdown_connections 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.808+0000 7f3dfc440640 1 -- 172.21.15.42:0/2559390875 wait complete. 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 Processor -- start 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 -- start start 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df41023c0 0x7f3df4100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3df4102ce0 0x7f3df4100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df4100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a5a20 con 0x7f3df410b790 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a5b90 con 0x7f3df41023c0 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfc440640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a5d50 con 0x7f3df4102ce0 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfa9dd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df4100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfa9dd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df4100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39506/0 (socket says 172.21.15.42:39506) 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfa9dd640 1 -- 172.21.15.42:0/2515454534 learned_addr learned my addr 172.21.15.42:0/2515454534 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.809+0000 7f3dfa1dc640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df41023c0 0x7f3df4100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3df99db640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3df4102ce0 0x7f3df4100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfa9dd640 1 -- 172.21.15.42:0/2515454534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3df4102ce0 msgr2=0x7f3df4100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfa9dd640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3df4102ce0 0x7f3df4100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfa9dd640 1 -- 172.21.15.42:0/2515454534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df41023c0 msgr2=0x7f3df4100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfa9dd640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df41023c0 0x7f3df4100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfa9dd640 1 -- 172.21.15.42:0/2515454534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec009900 con 0x7f3df410b790 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfa1dc640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df41023c0 0x7f3df4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:26:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfa9dd640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df4100a90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f3dec02c790 tx=0x7f3dec0349e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3deb7fe640 1 -- 172.21.15.42:0/2515454534 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec003e00 con 0x7f3df410b790 2024-03-12T04:26:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3deb7fe640 1 -- 172.21.15.42:0/2515454534 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dec032b30 con 0x7f3df410b790 2024-03-12T04:26:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3deb7fe640 1 -- 172.21.15.42:0/2515454534 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0047c0 con 0x7f3df410b790 2024-03-12T04:26:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df41a5fd0 con 0x7f3df410b790 2024-03-12T04:26:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.810+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41a6450 con 0x7f3df410b790 2024-03-12T04:26:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.811+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df4069f70 con 0x7f3df410b790 2024-03-12T04:26:00.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.812+0000 7f3deb7fe640 1 -- 172.21.15.42:0/2515454534 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3dec03b040 con 0x7f3df410b790 2024-03-12T04:26:00.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.812+0000 7f3deb7fe640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3de00a1bd0 0x7f3de00a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:00.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.813+0000 7f3dfa1dc640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3de00a1bd0 0x7f3de00a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:00.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.813+0000 7f3deb7fe640 1 -- 172.21.15.42:0/2515454534 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3dec0f0270 con 0x7f3df410b790 2024-03-12T04:26:00.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.813+0000 7f3dfa1dc640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3de00a1bd0 0x7f3de00a4090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3df00058c0 tx=0x7f3df0005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:00.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.817+0000 7f3deb7fe640 1 -- 172.21.15.42:0/2515454534 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3dec0b9210 con 0x7f3df410b790 2024-03-12T04:26:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.999+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3df40683a0 con 0x7f3df410b790 2024-03-12T04:26:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:00.999+0000 7f3deb7fe640 1 -- 172.21.15.42:0/2515454534 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3dec0bc8d0 con 0x7f3df410b790 2024-03-12T04:26:01.002 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:26:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3de00a1bd0 msgr2=0x7f3de00a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3de00a1bd0 0x7f3de00a4090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3df00058c0 tx=0x7f3df0005850 comp rx=0 tx=0).stop 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 msgr2=0x7f3df4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df4100a90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f3dec02c790 tx=0x7f3dec0349e0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 shutdown_connections 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3de00a1bd0 0x7f3de00a4090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df410b790 0x7f3df4100a90 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3df4102ce0 0x7f3df4100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 --2- 172.21.15.42:0/2515454534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df41023c0 0x7f3df4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 >> 172.21.15.42:0/2515454534 conn(0x7f3df4076400 msgr2=0x7f3df40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 shutdown_connections 2024-03-12T04:26:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.002+0000 7f3dfc440640 1 -- 172.21.15.42:0/2515454534 wait complete. 2024-03-12T04:26:01.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 -- 172.21.15.42:0/392696252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba01023c0 msgr2=0x7feba01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 --2- 172.21.15.42:0/392696252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba01023c0 0x7feba01027a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7feb90009900 tx=0x7feb9002bcd0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 -- 172.21.15.42:0/392696252 shutdown_connections 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 --2- 172.21.15.42:0/392696252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba010b790 0x7feba010dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 --2- 172.21.15.42:0/392696252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba0102ce0 0x7feba010b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 --2- 172.21.15.42:0/392696252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba01023c0 0x7feba01027a0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 -- 172.21.15.42:0/392696252 >> 172.21.15.42:0/392696252 conn(0x7feba0076400 msgr2=0x7feba0076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 -- 172.21.15.42:0/392696252 shutdown_connections 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.118+0000 7feba613a640 1 -- 172.21.15.42:0/392696252 wait complete. 2024-03-12T04:26:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 Processor -- start 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 -- start start 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba01023c0 0x7feba0197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba0102ce0 0x7feba0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba010b790 0x7feba019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba019c5f0 con 0x7feba0102ce0 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba019c760 con 0x7feba01023c0 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feba613a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba019c920 con 0x7feba010b790 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feb9effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba0102ce0 0x7feba0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feb9effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba0102ce0 0x7feba0197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39532/0 (socket says 172.21.15.42:39532) 2024-03-12T04:26:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feb9effd640 1 -- 172.21.15.42:0/3186028290 learned_addr learned my addr 172.21.15.42:0/3186028290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.119+0000 7feb9f7fe640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba01023c0 0x7feba0197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9ffff640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba010b790 0x7feba019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9effd640 1 -- 172.21.15.42:0/3186028290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba010b790 msgr2=0x7feba019bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9effd640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba010b790 0x7feba019bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9effd640 1 -- 172.21.15.42:0/3186028290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba01023c0 msgr2=0x7feba0197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9effd640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba01023c0 0x7feba0197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9effd640 1 -- 172.21.15.42:0/3186028290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90009560 con 0x7feba0102ce0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9effd640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba0102ce0 0x7feba0197c70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b4a0 tx=0x7feb9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9cff9640 1 -- 172.21.15.42:0/3186028290 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94010610 con 0x7feba0102ce0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba019cc00 con 0x7feba0102ce0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9cff9640 1 -- 172.21.15.42:0/3186028290 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feb94010c20 con 0x7feba0102ce0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feb9cff9640 1 -- 172.21.15.42:0/3186028290 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94017d00 con 0x7feba0102ce0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.120+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba01a4480 con 0x7feba0102ce0 2024-03-12T04:26:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.121+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba0069f90 con 0x7feba0102ce0 2024-03-12T04:26:01.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.122+0000 7feb9cff9640 1 -- 172.21.15.42:0/3186028290 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7feb94010770 con 0x7feba0102ce0 2024-03-12T04:26:01.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.123+0000 7feb9cff9640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7feb880a1ba0 0x7feb880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.123+0000 7feb9f7fe640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7feb880a1ba0 0x7feb880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.123+0000 7feb9cff9640 1 -- 172.21.15.42:0/3186028290 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7feb940cd3e0 con 0x7feba0102ce0 2024-03-12T04:26:01.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.123+0000 7feb9f7fe640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7feb880a1ba0 0x7feb880a4060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb90006010 tx=0x7feb90005df0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.126+0000 7feb9cff9640 1 -- 172.21.15.42:0/3186028290 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb94096380 con 0x7feba0102ce0 2024-03-12T04:26:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.308+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feba0106910 con 0x7feba0102ce0 2024-03-12T04:26:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.309+0000 7feb9cff9640 1 -- 172.21.15.42:0/3186028290 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7feba0106910 con 0x7feba0102ce0 2024-03-12T04:26:01.311 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7feb880a1ba0 msgr2=0x7feb880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7feb880a1ba0 0x7feb880a4060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb90006010 tx=0x7feb90005df0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba0102ce0 msgr2=0x7feba0197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba0102ce0 0x7feba0197c70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b4a0 tx=0x7feb9400b960 comp rx=0 tx=0).stop 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 shutdown_connections 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7feb880a1ba0 0x7feb880a4060 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba010b790 0x7feba019bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba0102ce0 0x7feba0197c70 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 --2- 172.21.15.42:0/3186028290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba01023c0 0x7feba0197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 >> 172.21.15.42:0/3186028290 conn(0x7feba0076400 msgr2=0x7feba00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 shutdown_connections 2024-03-12T04:26:01.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.311+0000 7feba613a640 1 -- 172.21.15.42:0/3186028290 wait complete. 2024-03-12T04:26:01.324 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:26:01.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- 172.21.15.42:0/3738912151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd20106450 msgr2=0x7fbd2010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 --2- 172.21.15.42:0/3738912151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd20106450 0x7fbd2010e020 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009930 tx=0x7fbd1802bc30 comp rx=0 tx=0).stop 2024-03-12T04:26:01.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- 172.21.15.42:0/3738912151 shutdown_connections 2024-03-12T04:26:01.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 --2- 172.21.15.42:0/3738912151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd20106450 0x7fbd2010e020 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 --2- 172.21.15.42:0/3738912151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd20101d30 0x7fbd20105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 --2- 172.21.15.42:0/3738912151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd20101380 0x7fbd20101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- 172.21.15.42:0/3738912151 >> 172.21.15.42:0/3738912151 conn(0x7fbd20078580 msgr2=0x7fbd20078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- 172.21.15.42:0/3738912151 shutdown_connections 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- 172.21.15.42:0/3738912151 wait complete. 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 Processor -- start 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- start start 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd20101380 0x7fbd201976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd20106450 0x7fbd20197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2019bfb0 0x7fbd2019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2019c9a0 con 0x7fbd2019bfb0 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2019cb10 con 0x7fbd20106450 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.428+0000 7fbd27920640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2019cc80 con 0x7fbd20101380 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2019bfb0 0x7fbd2019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2019bfb0 0x7fbd2019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39580/0 (socket says 172.21.15.42:39580) 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 -- 172.21.15.42:0/485422517 learned_addr learned my addr 172.21.15.42:0/485422517 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd256bc640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd20101380 0x7fbd201976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd24ebb640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd20106450 0x7fbd20197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 -- 172.21.15.42:0/485422517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd20101380 msgr2=0x7fbd201976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd20101380 0x7fbd201976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 -- 172.21.15.42:0/485422517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd20106450 msgr2=0x7fbd20197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd20106450 0x7fbd20197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 -- 172.21.15.42:0/485422517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd18009560 con 0x7fbd2019bfb0 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd24ebb640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd20106450 0x7fbd20197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd25ebd640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2019bfb0 0x7fbd2019c460 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fbd18005940 tx=0x7fbd180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd1e7fc640 1 -- 172.21.15.42:0/485422517 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd180044c0 con 0x7fbd2019bfb0 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd201a4260 con 0x7fbd2019bfb0 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd1e7fc640 1 -- 172.21.15.42:0/485422517 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd1802de90 con 0x7fbd2019bfb0 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd1e7fc640 1 -- 172.21.15.42:0/485422517 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd180053c0 con 0x7fbd2019bfb0 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.429+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd201a4710 con 0x7fbd2019bfb0 2024-03-12T04:26:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.430+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd200683c0 con 0x7fbd2019bfb0 2024-03-12T04:26:01.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.432+0000 7fbd1e7fc640 1 -- 172.21.15.42:0/485422517 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fbd18005520 con 0x7fbd2019bfb0 2024-03-12T04:26:01.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.432+0000 7fbd1e7fc640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd080a1ba0 0x7fbd080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.432+0000 7fbd1e7fc640 1 -- 172.21.15.42:0/485422517 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbd180f0110 con 0x7fbd2019bfb0 2024-03-12T04:26:01.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.432+0000 7fbd256bc640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd080a1ba0 0x7fbd080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.433+0000 7fbd256bc640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd080a1ba0 0x7fbd080a4060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005c40 tx=0x7fbd1000a780 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:01.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.436+0000 7fbd1e7fc640 1 -- 172.21.15.42:0/485422517 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd180b9110 con 0x7fbd2019bfb0 2024-03-12T04:26:01.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.618+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd20109770 con 0x7fbd2019bfb0 2024-03-12T04:26:01.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.619+0000 7fbd1e7fc640 1 -- 172.21.15.42:0/485422517 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd20109770 con 0x7fbd2019bfb0 2024-03-12T04:26:01.621 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:26:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd080a1ba0 msgr2=0x7fbd080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd080a1ba0 0x7fbd080a4060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005c40 tx=0x7fbd1000a780 comp rx=0 tx=0).stop 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2019bfb0 msgr2=0x7fbd2019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2019bfb0 0x7fbd2019c460 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fbd18005940 tx=0x7fbd180040c0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 shutdown_connections 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbd080a1ba0 0x7fbd080a4060 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2019bfb0 0x7fbd2019c460 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd20106450 0x7fbd20197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 --2- 172.21.15.42:0/485422517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd20101380 0x7fbd201976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 >> 172.21.15.42:0/485422517 conn(0x7fbd20078580 msgr2=0x7fbd200ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 shutdown_connections 2024-03-12T04:26:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.621+0000 7fbd27920640 1 -- 172.21.15.42:0/485422517 wait complete. 2024-03-12T04:26:01.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:01 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1526308072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:01.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:01 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2515454534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:26:01.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:01 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3186028290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:26:01.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T04:26:01.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:01 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1526308072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:01.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:01 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2515454534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:26:01.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:01 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3186028290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:26:01.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:01 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1526308072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:01.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:01 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2515454534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:26:01.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:01 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3186028290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:26:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 -- 172.21.15.42:0/452800431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 msgr2=0x7f9a8c10dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/452800431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c10dbc0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009ca0 tx=0x7f9a8402c180 comp rx=0 tx=0).stop 2024-03-12T04:26:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 -- 172.21.15.42:0/452800431 shutdown_connections 2024-03-12T04:26:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/452800431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c10dbc0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/452800431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c10b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/452800431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a8c1023c0 0x7f9a8c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 -- 172.21.15.42:0/452800431 >> 172.21.15.42:0/452800431 conn(0x7f9a8c076400 msgr2=0x7f9a8c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 -- 172.21.15.42:0/452800431 shutdown_connections 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.735+0000 7f9a93dbc640 1 -- 172.21.15.42:0/452800431 wait complete. 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 Processor -- start 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 -- start start 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a8c1023c0 0x7f9a8c197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c19c550 con 0x7f9a8c10b790 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c19c6c0 con 0x7f9a8c102ce0 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a93dbc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c19c880 con 0x7f9a8c1023c0 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39600/0 (socket says 172.21.15.42:39600) 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 -- 172.21.15.42:0/2146950415 learned_addr learned my addr 172.21.15.42:0/2146950415 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a91b58640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a8c1023c0 0x7f9a8c197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a91357640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 -- 172.21.15.42:0/2146950415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a8c1023c0 msgr2=0x7f9a8c197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a8c1023c0 0x7f9a8c197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 -- 172.21.15.42:0/2146950415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c102ce0 msgr2=0x7f9a8c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a92359640 1 -- 172.21.15.42:0/2146950415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009900 con 0x7f9a8c10b790 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.736+0000 7f9a91357640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.737+0000 7f9a92359640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c19bf50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9a8402c790 tx=0x7f9a840349e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.737+0000 7f9a82ffd640 1 -- 172.21.15.42:0/2146950415 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84003e00 con 0x7f9a8c10b790 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.737+0000 7f9a82ffd640 1 -- 172.21.15.42:0/2146950415 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a84032b30 con 0x7f9a8c10b790 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.737+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c19cb00 con 0x7f9a8c10b790 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.737+0000 7f9a82ffd640 1 -- 172.21.15.42:0/2146950415 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a840047c0 con 0x7f9a8c10b790 2024-03-12T04:26:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.737+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c1a43e0 con 0x7f9a8c10b790 2024-03-12T04:26:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.738+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c069f70 con 0x7f9a8c10b790 2024-03-12T04:26:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.739+0000 7f9a82ffd640 1 -- 172.21.15.42:0/2146950415 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f9a8403b040 con 0x7f9a8c10b790 2024-03-12T04:26:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.740+0000 7f9a82ffd640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a780a1c20 0x7f9a780a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.740+0000 7f9a91b58640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a780a1c20 0x7f9a780a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:01.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.740+0000 7f9a82ffd640 1 -- 172.21.15.42:0/2146950415 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9a840f00a0 con 0x7f9a8c10b790 2024-03-12T04:26:01.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.740+0000 7f9a91b58640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a780a1c20 0x7f9a780a40e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9a88005fd0 tx=0x7f9a8800b390 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:01.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.744+0000 7f9a82ffd640 1 -- 172.21.15.42:0/2146950415 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a840b9140 con 0x7f9a8c10b790 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.938+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a8c106880 con 0x7f9a8c10b790 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.938+0000 7f9a82ffd640 1 -- 172.21.15.42:0/2146950415 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v98) v1 ==== 81+0+944 (secure 0 0 0) 0x7f9a840bc800 con 0x7f9a8c10b790 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1061357309 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2954927866 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2757555113 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2757555113 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1914532345 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:01.941 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3686333836 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/956219817 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4126408073 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2670488995 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2257992536 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1532115047 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3492564527 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2755197547 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/331646616 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2670488995 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2755197547 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:01.942 INFO:teuthology.orchestra.run.smithi042.stderr:listed 16 entries 2024-03-12T04:26:01.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.940+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a780a1c20 msgr2=0x7f9a780a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.940+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a780a1c20 0x7f9a780a40e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9a88005fd0 tx=0x7f9a8800b390 comp rx=0 tx=0).stop 2024-03-12T04:26:01.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.940+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 msgr2=0x7f9a8c19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:01.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.940+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c19bf50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9a8402c790 tx=0x7f9a840349e0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.940+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 shutdown_connections 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.941+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f9a780a1c20 0x7f9a780a40e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.941+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a8c10b790 0x7f9a8c19bf50 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.941+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.941+0000 7f9a93dbc640 1 --2- 172.21.15.42:0/2146950415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a8c1023c0 0x7f9a8c197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.941+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 >> 172.21.15.42:0/2146950415 conn(0x7f9a8c076400 msgr2=0x7f9a8c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.941+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 shutdown_connections 2024-03-12T04:26:01.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:01.941+0000 7f9a93dbc640 1 -- 172.21.15.42:0/2146950415 wait complete. 2024-03-12T04:26:01.954 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-03-12T04:26:01.954 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T04:26:01.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:26:02.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.109+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1033079779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 msgr2=0x7fbcc810cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.109+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1033079779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc810cf90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009cd0 tx=0x7fbcc002c0f0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1033079779 shutdown_connections 2024-03-12T04:26:02.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1033079779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc810cf90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1033079779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcc8100ca0 0x7fbcc8104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1033079779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc81002f0 0x7fbcc81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1033079779 >> 172.21.15.42:0/1033079779 conn(0x7fbcc8077180 msgr2=0x7fbcc8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1033079779 shutdown_connections 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1033079779 wait complete. 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 Processor -- start 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 -- start start 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcc81002f0 0x7fbcc819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.110+0000 7fbccdbc7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbccdbc7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc81975b0 0x7fbcc8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbccdbc7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc8197fa0 con 0x7fbcc81053c0 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbccdbc7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc8198110 con 0x7fbcc81975b0 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbccdbc7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81982d0 con 0x7fbcc81002f0 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39612/0 (socket says 172.21.15.42:39612) 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 -- 172.21.15.42:0/1569460061 learned_addr learned my addr 172.21.15.42:0/1569460061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcc7fff640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc81975b0 0x7fbcc8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcc77fe640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcc81002f0 0x7fbcc819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 -- 172.21.15.42:0/1569460061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcc81002f0 msgr2=0x7fbcc819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcc81002f0 0x7fbcc819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 -- 172.21.15.42:0/1569460061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc81975b0 msgr2=0x7fbcc8197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc81975b0 0x7fbcc8197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 -- 172.21.15.42:0/1569460061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc0009900 con 0x7fbcc81053c0 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcc7fff640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc81975b0 0x7fbcc8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:26:02.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcbeffd640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc819d940 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d740 tx=0x7fbcb000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:02.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcc57fa640 1 -- 172.21.15.42:0/1569460061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0009c30 con 0x7fbcc81053c0 2024-03-12T04:26:02.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc81a42c0 con 0x7fbcc81053c0 2024-03-12T04:26:02.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcc57fa640 1 -- 172.21.15.42:0/1569460061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbcb000de50 con 0x7fbcc81053c0 2024-03-12T04:26:02.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbcc57fa640 1 -- 172.21.15.42:0/1569460061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0004e40 con 0x7fbcc81053c0 2024-03-12T04:26:02.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.111+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc81a47d0 con 0x7fbcc81053c0 2024-03-12T04:26:02.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.112+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc8067f70 con 0x7fbcc81053c0 2024-03-12T04:26:02.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.114+0000 7fbcc57fa640 1 -- 172.21.15.42:0/1569460061 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fbcb0004fa0 con 0x7fbcc81053c0 2024-03-12T04:26:02.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.114+0000 7fbcc57fa640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbcb40a1ba0 0x7fbcb40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.114+0000 7fbcc77fe640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbcb40a1ba0 0x7fbcb40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.114+0000 7fbcc57fa640 1 -- 172.21.15.42:0/1569460061 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbcb00cd6b0 con 0x7fbcc81053c0 2024-03-12T04:26:02.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.115+0000 7fbcc77fe640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbcb40a1ba0 0x7fbcb40a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009890 tx=0x7fbcb8008040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:02.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.118+0000 7fbcc57fa640 1 -- 172.21.15.42:0/1569460061 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcb00965a0 con 0x7fbcc81053c0 2024-03-12T04:26:02.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.303+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcc8109600 con 0x7fbcc81053c0 2024-03-12T04:26:02.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.305+0000 7fbcc57fa640 1 -- 172.21.15.42:0/1569460061 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+23076 (secure 0 0 0) 0x7fbcb0017090 con 0x7fbcc81053c0 2024-03-12T04:26:02.308 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:26:02.308 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:25:15.177682+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1061357309":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/2954927866":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6801/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/3686333836":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/4126408073":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:26:02.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbcb40a1ba0 msgr2=0x7fbcb40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbcb40a1ba0 0x7fbcb40a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009890 tx=0x7fbcb8008040 comp rx=0 tx=0).stop 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 msgr2=0x7fbcc819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc819d940 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d740 tx=0x7fbcb000dc00 comp rx=0 tx=0).stop 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 shutdown_connections 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fbcb40a1ba0 0x7fbcb40a4060 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc81975b0 0x7fbcc8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc81053c0 0x7fbcc819d940 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 --2- 172.21.15.42:0/1569460061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcc81002f0 0x7fbcc819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 >> 172.21.15.42:0/1569460061 conn(0x7fbcc8077180 msgr2=0x7fbcc8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 shutdown_connections 2024-03-12T04:26:02.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.307+0000 7fbccdbc7640 1 -- 172.21.15.42:0/1569460061 wait complete. 2024-03-12T04:26:02.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T04:08:09.170985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T04:12:52.642281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T04:12:53.381792+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T04:26:02.323 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T04:26:02.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 -- 172.21.15.42:0/1088992102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 msgr2=0x7f52240ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 --2- 172.21.15.42:0/1088992102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f52240ffd40 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5214009930 tx=0x7f521402bc30 comp rx=0 tx=0).stop 2024-03-12T04:26:02.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 -- 172.21.15.42:0/1088992102 shutdown_connections 2024-03-12T04:26:02.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 --2- 172.21.15.42:0/1088992102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5224106570 0x7f522410e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 --2- 172.21.15.42:0/1088992102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5224100310 0x7f5224105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 --2- 172.21.15.42:0/1088992102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f52240ffd40 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 -- 172.21.15.42:0/1088992102 >> 172.21.15.42:0/1088992102 conn(0x7f52240766b0 msgr2=0x7f52240fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 -- 172.21.15.42:0/1088992102 shutdown_connections 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 -- 172.21.15.42:0/1088992102 wait complete. 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 Processor -- start 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.427+0000 7f5229b6c640 1 -- start start 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f5229b6c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f522419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f5229b6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5224106570 0x7f522419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f5229b6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5224197690 0x7f5224197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f5229b6c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224198160 con 0x7f52240ff960 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f5229b6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52241982d0 con 0x7f5224106570 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f5229b6c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224198490 con 0x7f5224197690 2024-03-12T04:26:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f522419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f522419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39622/0 (socket says 172.21.15.42:39622) 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 -- 172.21.15.42:0/642330811 learned_addr learned my addr 172.21.15.42:0/642330811 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f521affd640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5224106570 0x7f522419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f5223fff640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5224197690 0x7f5224197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 -- 172.21.15.42:0/642330811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5224197690 msgr2=0x7f5224197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5224197690 0x7f5224197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 -- 172.21.15.42:0/642330811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5224106570 msgr2=0x7f522419db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5224106570 0x7f522419db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 -- 172.21.15.42:0/642330811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009560 con 0x7f52240ff960 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f521affd640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5224106570 0x7f522419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52237fe640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f522419d5d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5214009230 tx=0x7f52140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.428+0000 7f52217fa640 1 -- 172.21.15.42:0/642330811 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52140044c0 con 0x7f52240ff960 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.429+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52241a43d0 con 0x7f52240ff960 2024-03-12T04:26:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.429+0000 7f52217fa640 1 -- 172.21.15.42:0/642330811 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f521402de90 con 0x7f52240ff960 2024-03-12T04:26:02.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.429+0000 7f52217fa640 1 -- 172.21.15.42:0/642330811 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214005400 con 0x7f52240ff960 2024-03-12T04:26:02.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.429+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52241a48b0 con 0x7f52240ff960 2024-03-12T04:26:02.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.429+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5224102a10 con 0x7f52240ff960 2024-03-12T04:26:02.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.431+0000 7f52217fa640 1 -- 172.21.15.42:0/642330811 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f5214005560 con 0x7f52240ff960 2024-03-12T04:26:02.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.431+0000 7f52217fa640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f52100a1ba0 0x7f52100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.431+0000 7f52217fa640 1 -- 172.21.15.42:0/642330811 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f52140ef8a0 con 0x7f52240ff960 2024-03-12T04:26:02.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.431+0000 7f521affd640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f52100a1ba0 0x7f52100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.432+0000 7f521affd640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f52100a1ba0 0x7f52100a4060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f520c0058c0 tx=0x7f520c005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:02.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.436+0000 7f52217fa640 1 -- 172.21.15.42:0/642330811 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52140b88a0 con 0x7f52240ff960 2024-03-12T04:26:02.608 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:02 smithi136 ceph-mon[35571]: pgmap v336: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:26:02.609 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/485422517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:26:02.609 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2146950415' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T04:26:02.609 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:02 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1569460061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:02.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.620+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52240683c0 con 0x7f52240ff960 2024-03-12T04:26:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.620+0000 7f52217fa640 1 -- 172.21.15.42:0/642330811 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52140b8030 con 0x7f52240ff960 2024-03-12T04:26:02.623 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-03-12T04:26:02.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f52100a1ba0 msgr2=0x7f52100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f52100a1ba0 0x7f52100a4060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f520c0058c0 tx=0x7f520c005850 comp rx=0 tx=0).stop 2024-03-12T04:26:02.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 msgr2=0x7f522419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f522419d5d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5214009230 tx=0x7f52140040c0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 shutdown_connections 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f52100a1ba0 0x7f52100a4060 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5224197690 0x7f5224197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5224106570 0x7f522419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.622+0000 7f5229b6c640 1 --2- 172.21.15.42:0/642330811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52240ff960 0x7f522419d5d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.623+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 >> 172.21.15.42:0/642330811 conn(0x7f52240766b0 msgr2=0x7f52240fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.623+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 shutdown_connections 2024-03-12T04:26:02.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.623+0000 7f5229b6c640 1 -- 172.21.15.42:0/642330811 wait complete. 2024-03-12T04:26:02.635 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:02 smithi042 ceph-mon[32260]: pgmap v336: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:26:02.635 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/485422517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:26:02.635 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2146950415' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T04:26:02.635 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:02 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1569460061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:02.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T04:26:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:02 smithi119 ceph-mon[36119]: pgmap v336: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-12T04:26:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/485422517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:26:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2146950415' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T04:26:02.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:02 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1569460061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 -- 172.21.15.42:0/3927743316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac068f80 msgr2=0x7f2bac105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/3927743316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac068f80 0x7f2bac105d50 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009930 tx=0x7f2b9402bc20 comp rx=0 tx=0).stop 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 -- 172.21.15.42:0/3927743316 shutdown_connections 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/3927743316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bac106430 0x7f2bac10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/3927743316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac068f80 0x7f2bac105d50 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/3927743316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0685d0 0x7f2bac0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 -- 172.21.15.42:0/3927743316 >> 172.21.15.42:0/3927743316 conn(0x7f2bac077180 msgr2=0x7f2bac077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 -- 172.21.15.42:0/3927743316 shutdown_connections 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.739+0000 7f2bb15e2640 1 -- 172.21.15.42:0/3927743316 wait complete. 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 Processor -- start 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 -- start start 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0685d0 0x7f2bac100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac106430 0x7f2bac100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bac100ac0 0x7f2bac1a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac1a5e20 con 0x7f2bac106430 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac1a5f90 con 0x7f2bac0685d0 2024-03-12T04:26:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2bb15e2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac1a6150 con 0x7f2bac100ac0 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2baa7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac106430 0x7f2bac100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2baa7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac106430 0x7f2bac100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39648/0 (socket says 172.21.15.42:39648) 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2baa7fc640 1 -- 172.21.15.42:0/1638845529 learned_addr learned my addr 172.21.15.42:0/1638845529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.740+0000 7f2baaffd640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0685d0 0x7f2bac100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2bab7fe640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bac100ac0 0x7f2bac1a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2baa7fc640 1 -- 172.21.15.42:0/1638845529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bac100ac0 msgr2=0x7f2bac1a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2baa7fc640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bac100ac0 0x7f2bac1a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2baa7fc640 1 -- 172.21.15.42:0/1638845529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0685d0 msgr2=0x7f2bac100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2baa7fc640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0685d0 0x7f2bac100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2baa7fc640 1 -- 172.21.15.42:0/1638845529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009560 con 0x7f2bac106430 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2baaffd640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0685d0 0x7f2bac100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2bab7fe640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bac100ac0 0x7f2bac1a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2baa7fc640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac106430 0x7f2bac100580 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2b94000c00 tx=0x7f2b94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2ba3fff640 1 -- 172.21.15.42:0/1638845529 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9402cb60 con 0x7f2bac106430 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac1a63d0 con 0x7f2bac106430 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2ba3fff640 1 -- 172.21.15.42:0/1638845529 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b94003aa0 con 0x7f2bac106430 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2ba3fff640 1 -- 172.21.15.42:0/1638845529 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94005540 con 0x7f2bac106430 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.741+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac1a68e0 con 0x7f2bac106430 2024-03-12T04:26:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.742+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac067f70 con 0x7f2bac106430 2024-03-12T04:26:02.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.743+0000 7f2ba3fff640 1 -- 172.21.15.42:0/1638845529 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f2b94003c10 con 0x7f2bac106430 2024-03-12T04:26:02.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.744+0000 7f2ba3fff640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2b980a1bd0 0x7f2b980a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:02.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.744+0000 7f2ba3fff640 1 -- 172.21.15.42:0/1638845529 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b940f0500 con 0x7f2bac106430 2024-03-12T04:26:02.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.744+0000 7f2baaffd640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2b980a1bd0 0x7f2b980a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:02.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.744+0000 7f2baaffd640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2b980a1bd0 0x7f2b980a4090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c007e70 tx=0x7f2b9c007850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:02.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.747+0000 7f2ba3fff640 1 -- 172.21.15.42:0/1638845529 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b940b94a0 con 0x7f2bac106430 2024-03-12T04:26:02.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.929+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bac109680 con 0x7f2bac106430 2024-03-12T04:26:02.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.929+0000 7f2ba3fff640 1 -- 172.21.15.42:0/1638845529 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b94043020 con 0x7f2bac106430 2024-03-12T04:26:02.933 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-03-12T04:26:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2b980a1bd0 msgr2=0x7f2b980a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2b980a1bd0 0x7f2b980a4090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c007e70 tx=0x7f2b9c007850 comp rx=0 tx=0).stop 2024-03-12T04:26:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac106430 msgr2=0x7f2bac100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:02.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac106430 0x7f2bac100580 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2b94000c00 tx=0x7f2b94004270 comp rx=0 tx=0).stop 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 shutdown_connections 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f2b980a1bd0 0x7f2b980a4090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bac100ac0 0x7f2bac1a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bac106430 0x7f2bac100580 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 --2- 172.21.15.42:0/1638845529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0685d0 0x7f2bac100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.931+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 >> 172.21.15.42:0/1638845529 conn(0x7f2bac077180 msgr2=0x7f2bac0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.932+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 shutdown_connections 2024-03-12T04:26:02.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:02.932+0000 7f2bb15e2640 1 -- 172.21.15.42:0/1638845529 wait complete. 2024-03-12T04:26:02.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 -- 172.21.15.42:0/3080048372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788105330 msgr2=0x7f978810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 --2- 172.21.15.42:0/3080048372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788105330 0x7f978810cf00 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9770009930 tx=0x7f977002bc30 comp rx=0 tx=0).stop 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 -- 172.21.15.42:0/3080048372 shutdown_connections 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 --2- 172.21.15.42:0/3080048372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788105330 0x7f978810cf00 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 --2- 172.21.15.42:0/3080048372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9788100c10 0x7f9788104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 --2- 172.21.15.42:0/3080048372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9788100260 0x7f9788100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 -- 172.21.15.42:0/3080048372 >> 172.21.15.42:0/3080048372 conn(0x7f9788077180 msgr2=0x7f9788077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 -- 172.21.15.42:0/3080048372 shutdown_connections 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.048+0000 7f978f0c3640 1 -- 172.21.15.42:0/3080048372 wait complete. 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 Processor -- start 2024-03-12T04:26:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 -- start start 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788100260 0x7f97881984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9788105330 0x7f9788198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9788198ff0 0x7f97881a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97881a3c40 con 0x7f9788100260 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97881a3db0 con 0x7f9788105330 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978f0c3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97881a3f70 con 0x7f9788198ff0 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978ce5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788100260 0x7f97881984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978ce5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788100260 0x7f97881984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39660/0 (socket says 172.21.15.42:39660) 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978ce5f640 1 -- 172.21.15.42:0/1110074335 learned_addr learned my addr 172.21.15.42:0/1110074335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f978d660640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9788198ff0 0x7f97881a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.049+0000 7f9787fff640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9788105330 0x7f9788198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978ce5f640 1 -- 172.21.15.42:0/1110074335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9788198ff0 msgr2=0x7f97881a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978ce5f640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9788198ff0 0x7f97881a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978ce5f640 1 -- 172.21.15.42:0/1110074335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9788105330 msgr2=0x7f9788198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978ce5f640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9788105330 0x7f9788198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978ce5f640 1 -- 172.21.15.42:0/1110074335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770009560 con 0x7f9788100260 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f9787fff640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9788105330 0x7f9788198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978ce5f640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788100260 0x7f97881984d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f978000e9f0 tx=0x7f978000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f9785ffb640 1 -- 172.21.15.42:0/1110074335 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780009700 con 0x7f9788100260 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f9785ffb640 1 -- 172.21.15.42:0/1110074335 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f978000cd80 con 0x7f9788100260 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97881a4250 con 0x7f9788100260 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f9785ffb640 1 -- 172.21.15.42:0/1110074335 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780018840 con 0x7f9788100260 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.050+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97881a4760 con 0x7f9788100260 2024-03-12T04:26:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.051+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9788067f70 con 0x7f9788100260 2024-03-12T04:26:03.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.052+0000 7f9785ffb640 1 -- 172.21.15.42:0/1110074335 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f97800189a0 con 0x7f9788100260 2024-03-12T04:26:03.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.053+0000 7f9785ffb640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f97740a1ba0 0x7f97740a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.053+0000 7f9785ffb640 1 -- 172.21.15.42:0/1110074335 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9780014070 con 0x7f9788100260 2024-03-12T04:26:03.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.053+0000 7f9787fff640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f97740a1ba0 0x7f97740a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.053+0000 7f9787fff640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f97740a1ba0 0x7f97740a4060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9788068e90 tx=0x7f9778009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:03.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.056+0000 7f9785ffb640 1 -- 172.21.15.42:0/1110074335 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9780096f20 con 0x7f9788100260 2024-03-12T04:26:03.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.238+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9788108580 con 0x7f9788100260 2024-03-12T04:26:03.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.238+0000 7f9785ffb640 1 -- 172.21.15.42:0/1110074335 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f978009a5e0 con 0x7f9788100260 2024-03-12T04:26:03.241 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-03-12T04:26:03.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f97740a1ba0 msgr2=0x7f97740a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f97740a1ba0 0x7f97740a4060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9788068e90 tx=0x7f9778009500 comp rx=0 tx=0).stop 2024-03-12T04:26:03.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788100260 msgr2=0x7f97881984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788100260 0x7f97881984d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f978000e9f0 tx=0x7f978000eeb0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 shutdown_connections 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f97740a1ba0 0x7f97740a4060 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9788198ff0 0x7f97881a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9788105330 0x7f9788198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 --2- 172.21.15.42:0/1110074335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9788100260 0x7f97881984d0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 >> 172.21.15.42:0/1110074335 conn(0x7f9788077180 msgr2=0x7f97880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 shutdown_connections 2024-03-12T04:26:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.240+0000 7f978f0c3640 1 -- 172.21.15.42:0/1110074335 wait complete. 2024-03-12T04:26:03.254 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 -- 172.21.15.42:0/1770216581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100790 msgr2=0x7f5998100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 --2- 172.21.15.42:0/1770216581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100790 0x7f5998100c10 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5984009900 tx=0x7f598402bc30 comp rx=0 tx=0).stop 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 -- 172.21.15.42:0/1770216581 shutdown_connections 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 --2- 172.21.15.42:0/1770216581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5998101150 0x7f599810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 --2- 172.21.15.42:0/1770216581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100790 0x7f5998100c10 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 --2- 172.21.15.42:0/1770216581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998105070 0x7f5998105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 -- 172.21.15.42:0/1770216581 >> 172.21.15.42:0/1770216581 conn(0x7f5998076640 msgr2=0x7f59980fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 -- 172.21.15.42:0/1770216581 shutdown_connections 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 -- 172.21.15.42:0/1770216581 wait complete. 2024-03-12T04:26:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 Processor -- start 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.356+0000 7f599f00f640 1 -- start start 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599f00f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998100790 0x7f5998197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599f00f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5998101150 0x7f5998197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599f00f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998105070 0x7f599819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599f00f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599819c540 con 0x7f5998105070 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599f00f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599819c6b0 con 0x7f5998101150 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599f00f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599819c870 con 0x7f5998100790 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998105070 0x7f599819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998105070 0x7f599819bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39676/0 (socket says 172.21.15.42:39676) 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 -- 172.21.15.42:0/1042478583 learned_addr learned my addr 172.21.15.42:0/1042478583 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599cdab640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998100790 0x7f5998197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f5997fff640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5998101150 0x7f5998197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 -- 172.21.15.42:0/1042478583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998100790 msgr2=0x7f5998197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998100790 0x7f5998197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 -- 172.21.15.42:0/1042478583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5998101150 msgr2=0x7f5998197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5998101150 0x7f5998197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 -- 172.21.15.42:0/1042478583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984009560 con 0x7f5998105070 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599d5ac640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998105070 0x7f599819bf40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f599000eb10 tx=0x7f599000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f5995ffb640 1 -- 172.21.15.42:0/1042478583 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990004b20 con 0x7f5998105070 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f5995ffb640 1 -- 172.21.15.42:0/1042478583 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5990004c80 con 0x7f5998105070 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599819cb50 con 0x7f5998105070 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.357+0000 7f5995ffb640 1 -- 172.21.15.42:0/1042478583 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990010690 con 0x7f5998105070 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.358+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59981a43d0 con 0x7f5998105070 2024-03-12T04:26:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.359+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59980683c0 con 0x7f5998105070 2024-03-12T04:26:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.359+0000 7f5995ffb640 1 -- 172.21.15.42:0/1042478583 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f5990005020 con 0x7f5998105070 2024-03-12T04:26:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.360+0000 7f5995ffb640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59880a1ba0 0x7f59880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.360+0000 7f599cdab640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59880a1ba0 0x7f59880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.360+0000 7f5995ffb640 1 -- 172.21.15.42:0/1042478583 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5990014070 con 0x7f5998105070 2024-03-12T04:26:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.360+0000 7f599cdab640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59880a1ba0 0x7f59880a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f598c009bd0 tx=0x7f598c005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.364+0000 7f5995ffb640 1 -- 172.21.15.42:0/1042478583 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5990096d30 con 0x7f5998105070 2024-03-12T04:26:03.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.559+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f59981085f0 con 0x7f5998105070 2024-03-12T04:26:03.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.560+0000 7f5995ffb640 1 -- 172.21.15.42:0/1042478583 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v98) v1 ==== 81+0+944 (secure 0 0 0) 0x7f599009a3f0 con 0x7f5998105070 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1061357309 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2954927866 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2757555113 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2757555113 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1914532345 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3686333836 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/956219817 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4126408073 2024-03-13T04:15:11.144062+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2670488995 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2257992536 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1532115047 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3492564527 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2755197547 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/331646616 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2670488995 2024-03-13T04:05:44.816230+0000 2024-03-12T04:26:03.564 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2755197547 2024-03-13T04:05:17.928065+0000 2024-03-12T04:26:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:listed 16 entries 2024-03-12T04:26:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59880a1ba0 msgr2=0x7f59880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59880a1ba0 0x7f59880a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f598c009bd0 tx=0x7f598c005f90 comp rx=0 tx=0).stop 2024-03-12T04:26:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998105070 msgr2=0x7f599819bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998105070 0x7f599819bf40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f599000eb10 tx=0x7f599000c2d0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 shutdown_connections 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f59880a1ba0 0x7f59880a4060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998105070 0x7f599819bf40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5998101150 0x7f5998197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 --2- 172.21.15.42:0/1042478583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998100790 0x7f5998197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 >> 172.21.15.42:0/1042478583 conn(0x7f5998076640 msgr2=0x7f5998103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 shutdown_connections 2024-03-12T04:26:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:03.562+0000 7f599f00f640 1 -- 172.21.15.42:0/1042478583 wait complete. 2024-03-12T04:26:03.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:03 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/642330811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:03 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1638845529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:03 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1110074335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.575 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-03-12T04:26:03.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:26:03.576 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-12T04:26:03.603 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T04:26:03.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[81627]: fuse finished with error 0 and tester_r 0 2024-03-12T04:26:03.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:03 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/642330811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:03 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1638845529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:03 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1110074335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:03 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/642330811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:03 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1638845529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T04:26:03.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:03 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1110074335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T04:26:04.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:04 smithi136 ceph-mon[35571]: pgmap v337: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:26:04.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:04 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1042478583' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T04:26:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:04 smithi042 ceph-mon[32260]: pgmap v337: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:26:04.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:04 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1042478583' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T04:26:04.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:04 smithi119 ceph-mon[36119]: pgmap v337: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-12T04:26:04.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:04 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1042478583' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T04:26:06.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:06 smithi136 ceph-mon[35571]: pgmap v338: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:26:06.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:06 smithi042 ceph-mon[32260]: pgmap v338: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:26:06.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:06 smithi119 ceph-mon[36119]: pgmap v338: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-12T04:26:08.707 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:08 smithi136 ceph-mon[35571]: pgmap v339: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-03-12T04:26:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:08 smithi042 ceph-mon[32260]: pgmap v339: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-03-12T04:26:08.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:08 smithi119 ceph-mon[36119]: pgmap v339: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-03-12T04:26:09.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T04:26:09.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:26:09.705 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-12T04:26:09.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:26:09.737 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-03-12T04:26:09.794 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-12T04:26:09.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:26:09.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T04:26:09.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T04:26:09.872 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-03-12T04:26:09.975 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-12T04:26:09.986 INFO:tasks.cephadm:Teardown begin 2024-03-12T04:26:09.987 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-12T04:26:09.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:26:10.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 -- 172.21.15.42:0/3110888017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 msgr2=0x7f05e41027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 --2- 172.21.15.42:0/3110888017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e41027d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009900 tx=0x7f05d402bc30 comp rx=0 tx=0).stop 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 -- 172.21.15.42:0/3110888017 shutdown_connections 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 --2- 172.21.15.42:0/3110888017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e410b7c0 0x7f05e410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 --2- 172.21.15.42:0/3110888017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e4102d10 0x7f05e410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 --2- 172.21.15.42:0/3110888017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e41027d0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 -- 172.21.15.42:0/3110888017 >> 172.21.15.42:0/3110888017 conn(0x7f05e4077180 msgr2=0x7f05e4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 -- 172.21.15.42:0/3110888017 shutdown_connections 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 -- 172.21.15.42:0/3110888017 wait complete. 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.091+0000 7f05e9d77640 1 Processor -- start 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e9d77640 1 -- start start 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e9d77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e9d77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e4102d10 0x7f05e419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e9d77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e410b7c0 0x7f05e41975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e9d77640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4197c00 con 0x7f05e41023f0 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e9d77640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4197d70 con 0x7f05e4102d10 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e9d77640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4197f30 con 0x7f05e410b7c0 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39688/0 (socket says 172.21.15.42:39688) 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 -- 172.21.15.42:0/2699322518 learned_addr learned my addr 172.21.15.42:0/2699322518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e3fff640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e410b7c0 0x7f05e41975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e2ffd640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e4102d10 0x7f05e419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 -- 172.21.15.42:0/2699322518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e410b7c0 msgr2=0x7f05e41975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e410b7c0 0x7f05e41975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 -- 172.21.15.42:0/2699322518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e4102d10 msgr2=0x7f05e419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e4102d10 0x7f05e419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 -- 172.21.15.42:0/2699322518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d4009560 con 0x7f05e41023f0 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.092+0000 7f05e37fe640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e419d400 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006010 tx=0x7f05d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:10.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.093+0000 7f05e0ff9640 1 -- 172.21.15.42:0/2699322518 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40035f0 con 0x7f05e41023f0 2024-03-12T04:26:10.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.093+0000 7f05e0ff9640 1 -- 172.21.15.42:0/2699322518 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f05d402de90 con 0x7f05e41023f0 2024-03-12T04:26:10.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.093+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e41981b0 con 0x7f05e41023f0 2024-03-12T04:26:10.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.093+0000 7f05e0ff9640 1 -- 172.21.15.42:0/2699322518 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4005440 con 0x7f05e41023f0 2024-03-12T04:26:10.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.093+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e41a44a0 con 0x7f05e41023f0 2024-03-12T04:26:10.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.094+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e4067f70 con 0x7f05e41023f0 2024-03-12T04:26:10.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.095+0000 7f05e0ff9640 1 -- 172.21.15.42:0/2699322518 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f05d4003750 con 0x7f05e41023f0 2024-03-12T04:26:10.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.095+0000 7f05e0ff9640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f05cc0a1ba0 0x7f05cc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.096+0000 7f05e2ffd640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f05cc0a1ba0 0x7f05cc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.096+0000 7f05e0ff9640 1 -- 172.21.15.42:0/2699322518 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f05d40efbb0 con 0x7f05e41023f0 2024-03-12T04:26:10.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.096+0000 7f05e2ffd640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f05cc0a1ba0 0x7f05cc0a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f05e410d650 tx=0x7f05d8008040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:10.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.100+0000 7f05e0ff9640 1 -- 172.21.15.42:0/2699322518 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05d40b8b90 con 0x7f05e41023f0 2024-03-12T04:26:10.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.323+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05e41068b0 con 0x7f05e41023f0 2024-03-12T04:26:10.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.324+0000 7f05e0ff9640 1 -- 172.21.15.42:0/2699322518 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7f05d40bc250 con 0x7f05e41023f0 2024-03-12T04:26:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:26:10.327 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:26:10.327 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:26:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f05cc0a1ba0 msgr2=0x7f05cc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f05cc0a1ba0 0x7f05cc0a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f05e410d650 tx=0x7f05d8008040 comp rx=0 tx=0).stop 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 msgr2=0x7f05e419d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e419d400 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006010 tx=0x7f05d4004270 comp rx=0 tx=0).stop 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 shutdown_connections 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f05cc0a1ba0 0x7f05cc0a4060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e410b7c0 0x7f05e41975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e4102d10 0x7f05e419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 --2- 172.21.15.42:0/2699322518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05e41023f0 0x7f05e419d400 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 >> 172.21.15.42:0/2699322518 conn(0x7f05e4077180 msgr2=0x7f05e40fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 shutdown_connections 2024-03-12T04:26:10.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.326+0000 7f05e9d77640 1 -- 172.21.15.42:0/2699322518 wait complete. 2024-03-12T04:26:10.340 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 -- 172.21.15.42:0/3868972986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf401002a0 msgr2=0x7faf40105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 --2- 172.21.15.42:0/3868972986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf401002a0 0x7faf40105dd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7faf34009930 tx=0x7faf3402bc30 comp rx=0 tx=0).stop 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 -- 172.21.15.42:0/3868972986 shutdown_connections 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 --2- 172.21.15.42:0/3868972986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf40106500 0x7faf4010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 --2- 172.21.15.42:0/3868972986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf401002a0 0x7faf40105dd0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 --2- 172.21.15.42:0/3868972986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf400ff8f0 0x7faf400ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 -- 172.21.15.42:0/3868972986 >> 172.21.15.42:0/3868972986 conn(0x7faf40076610 msgr2=0x7faf400fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 -- 172.21.15.42:0/3868972986 shutdown_connections 2024-03-12T04:26:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 -- 172.21.15.42:0/3868972986 wait complete. 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.440+0000 7faf4512c640 1 Processor -- start 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf4512c640 1 -- start start 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf4512c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf400ff8f0 0x7faf40100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf4512c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf40106500 0x7faf40100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf4512c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf401012e0 0x7faf401a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf4512c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401a6080 con 0x7faf400ff8f0 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf4512c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401a61f0 con 0x7faf40106500 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf4512c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401a63b0 con 0x7faf401012e0 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf400ff8f0 0x7faf40100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf400ff8f0 0x7faf40100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39700/0 (socket says 172.21.15.42:39700) 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 -- 172.21.15.42:0/2826797036 learned_addr learned my addr 172.21.15.42:0/2826797036 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3f59e640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf401012e0 0x7faf401a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3e59c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf40106500 0x7faf40100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 -- 172.21.15.42:0/2826797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf401012e0 msgr2=0x7faf401a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf401012e0 0x7faf401a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 -- 172.21.15.42:0/2826797036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf40106500 msgr2=0x7faf40100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf40106500 0x7faf40100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 -- 172.21.15.42:0/2826797036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf34009560 con 0x7faf400ff8f0 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3e59c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf40106500 0x7faf40100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.441+0000 7faf3ed9d640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf400ff8f0 0x7faf40100860 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b4a0 tx=0x7faf3800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.442+0000 7faf27fff640 1 -- 172.21.15.42:0/2826797036 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38010610 con 0x7faf400ff8f0 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.442+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf401a6690 con 0x7faf400ff8f0 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.442+0000 7faf27fff640 1 -- 172.21.15.42:0/2826797036 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf38010c20 con 0x7faf400ff8f0 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.442+0000 7faf27fff640 1 -- 172.21.15.42:0/2826797036 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38017d00 con 0x7faf400ff8f0 2024-03-12T04:26:10.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.442+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf401a6bf0 con 0x7faf400ff8f0 2024-03-12T04:26:10.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.443+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf400683c0 con 0x7faf400ff8f0 2024-03-12T04:26:10.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.444+0000 7faf27fff640 1 -- 172.21.15.42:0/2826797036 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7faf38010770 con 0x7faf400ff8f0 2024-03-12T04:26:10.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.444+0000 7faf27fff640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faf280a1bd0 0x7faf280a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.445+0000 7faf3e59c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faf280a1bd0 0x7faf280a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.445+0000 7faf27fff640 1 -- 172.21.15.42:0/2826797036 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7faf380ccd50 con 0x7faf400ff8f0 2024-03-12T04:26:10.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.445+0000 7faf3e59c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faf280a1bd0 0x7faf280a4090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faf3402c170 tx=0x7faf34000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:10.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.449+0000 7faf27fff640 1 -- 172.21.15.42:0/2826797036 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf38095d30 con 0x7faf400ff8f0 2024-03-12T04:26:10.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.673+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf401a6820 con 0x7faf400ff8f0 2024-03-12T04:26:10.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.674+0000 7faf27fff640 1 -- 172.21.15.42:0/2826797036 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11388 (secure 0 0 0) 0x7faf401a6820 con 0x7faf400ff8f0 2024-03-12T04:26:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:26:10.677 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/43691567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":43691567},{"type":"v1","addr":"172.21.15.42:6835","nonce":43691567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3380811405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3380811405},{"type":"v1","addr":"172.21.15.42:6837","nonce":3380811405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T04:12:54.116170+0000","modified":"2024-03-12T04:14:56.225617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24490,"mds_2":24485,"mds_3":24475,"mds_4":14634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/1870620293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":1870620293},{"type":"v1","addr":"172.21.15.42:6839","nonce":1870620293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14634":{"gid":14634,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6841/799836757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":799836757},{"type":"v1","addr":"172.21.15.42:6841","nonce":799836757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/3418533459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3418533459},{"type":"v1","addr":"172.21.15.136:6833","nonce":3418533459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6833/2928023683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2928023683},{"type":"v1","addr":"172.21.15.119:6833","nonce":2928023683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/2794337720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2794337720},{"type":"v1","addr":"172.21.15.119:6835","nonce":2794337720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2718353674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2718353674},{"type":"v1","addr":"172.21.15.136:6835","nonce":2718353674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/23397667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":23397667},{"type":"v1","addr":"172.21.15.136:6837","nonce":23397667}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6837/3839383373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3839383373},{"type":"v1","addr":"172.21.15.119:6837","nonce":3839383373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/3211189908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211189908},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211189908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/1451526204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1451526204},{"type":"v1","addr":"172.21.15.119:6839","nonce":1451526204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T04:26:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faf280a1bd0 msgr2=0x7faf280a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faf280a1bd0 0x7faf280a4090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faf3402c170 tx=0x7faf34000bc0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf400ff8f0 msgr2=0x7faf40100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf400ff8f0 0x7faf40100860 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b4a0 tx=0x7faf3800b960 comp rx=0 tx=0).stop 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 shutdown_connections 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faf280a1bd0 0x7faf280a4090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf401012e0 0x7faf401a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf40106500 0x7faf40100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 --2- 172.21.15.42:0/2826797036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf400ff8f0 0x7faf40100860 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 >> 172.21.15.42:0/2826797036 conn(0x7faf40076610 msgr2=0x7faf40104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 shutdown_connections 2024-03-12T04:26:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.676+0000 7faf4512c640 1 -- 172.21.15.42:0/2826797036 wait complete. 2024-03-12T04:26:10.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:10 smithi042 ceph-mon[32260]: pgmap v340: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-12T04:26:10.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:10 smithi042 ceph-mon[32260]: osdmap e99: 12 total, 12 up, 12 in 2024-03-12T04:26:10.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:10 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2699322518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:26:10.690 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T04:26:10.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 -- 172.21.15.42:0/1227536586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01013b0 msgr2=0x7faaf0104cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 --2- 172.21.15.42:0/1227536586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01013b0 0x7faaf0104cc0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7faae0009930 tx=0x7faae002bc30 comp rx=0 tx=0).stop 2024-03-12T04:26:10.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 -- 172.21.15.42:0/1227536586 shutdown_connections 2024-03-12T04:26:10.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 --2- 172.21.15.42:0/1227536586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf01053f0 0x7faaf010cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 --2- 172.21.15.42:0/1227536586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01013b0 0x7faaf0104cc0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 --2- 172.21.15.42:0/1227536586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf0100a00 0x7faaf0100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 -- 172.21.15.42:0/1227536586 >> 172.21.15.42:0/1227536586 conn(0x7faaf0076610 msgr2=0x7faaf00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 -- 172.21.15.42:0/1227536586 shutdown_connections 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 -- 172.21.15.42:0/1227536586 wait complete. 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.791+0000 7faaf770f640 1 Processor -- start 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf770f640 1 -- start start 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf770f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf0100a00 0x7faaf019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf770f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf01053f0 0x7faaf019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf770f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01976d0 0x7faaf0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf770f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf01981a0 con 0x7faaf01976d0 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf770f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0198310 con 0x7faaf0100a00 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf770f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf01984d0 con 0x7faaf01053f0 2024-03-12T04:26:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01976d0 0x7faaf0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01976d0 0x7faaf0197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39722/0 (socket says 172.21.15.42:39722) 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 -- 172.21.15.42:0/556332643 learned_addr learned my addr 172.21.15.42:0/556332643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf4caa640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf01053f0 0x7faaf019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf54ab640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf0100a00 0x7faaf019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 -- 172.21.15.42:0/556332643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf01053f0 msgr2=0x7faaf019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf01053f0 0x7faaf019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 -- 172.21.15.42:0/556332643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf0100a00 msgr2=0x7faaf019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf0100a00 0x7faaf019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 -- 172.21.15.42:0/556332643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009560 con 0x7faaf01976d0 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf54ab640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf0100a00 0x7faaf019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faaf5cac640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01976d0 0x7faaf0197b50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7faae800b970 tx=0x7faae800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.792+0000 7faae67fc640 1 -- 172.21.15.42:0/556332643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae800cb40 con 0x7faaf01976d0 2024-03-12T04:26:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.793+0000 7faae67fc640 1 -- 172.21.15.42:0/556332643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faae800cca0 con 0x7faaf01976d0 2024-03-12T04:26:10.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.793+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf01a4470 con 0x7faaf01976d0 2024-03-12T04:26:10.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.793+0000 7faae67fc640 1 -- 172.21.15.42:0/556332643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80076e0 con 0x7faaf01976d0 2024-03-12T04:26:10.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.793+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf01a49a0 con 0x7faaf01976d0 2024-03-12T04:26:10.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.794+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf00683c0 con 0x7faaf01976d0 2024-03-12T04:26:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.795+0000 7faae67fc640 1 -- 172.21.15.42:0/556332643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7faae8014440 con 0x7faaf01976d0 2024-03-12T04:26:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.795+0000 7faae67fc640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faadc0a1c20 0x7faadc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.796+0000 7faaf54ab640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faadc0a1c20 0x7faadc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.796+0000 7faae67fc640 1 -- 172.21.15.42:0/556332643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7faae80ccf90 con 0x7faaf01976d0 2024-03-12T04:26:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.796+0000 7faaf54ab640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faadc0a1c20 0x7faadc0a40e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faaec0058c0 tx=0x7faaec005d30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:10.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.800+0000 7faae67fc640 1 -- 172.21.15.42:0/556332643 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faae8095f70 con 0x7faaf01976d0 2024-03-12T04:26:10.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:10 smithi119 ceph-mon[36119]: pgmap v340: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-12T04:26:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:10 smithi119 ceph-mon[36119]: osdmap e99: 12 total, 12 up, 12 in 2024-03-12T04:26:10.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:10 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2699322518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:26:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:10 smithi136 ceph-mon[35571]: pgmap v340: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-12T04:26:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:10 smithi136 ceph-mon[35571]: osdmap e99: 12 total, 12 up, 12 in 2024-03-12T04:26:10.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:10 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2699322518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:26:10.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.982+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaf01101b0 con 0x7faaf01976d0 2024-03-12T04:26:10.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.983+0000 7faae67fc640 1 -- 172.21.15.42:0/556332643 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+23143 (secure 0 0 0) 0x7faae8019090 con 0x7faaf01976d0 2024-03-12T04:26:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-03-12T04:26:10.987 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":99,"fsid":"7cca013a-e025-11ee-95c8-87774f69a715","created":"2024-03-12T04:04:45.008136+0000","modified":"2024-03-12T04:26:09.454606+0000","last_up_change":"2024-03-12T04:12:00.566723+0000","last_in_change":"2024-03-12T04:11:39.454453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T04:08:09.170985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T04:12:52.642281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T04:12:53.381792+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66e5b58b-4170-4df0-a9b2-5abe5a61884c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6803","nonce":1137485788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6805","nonce":1137485788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6809","nonce":1137485788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1137485788},{"type":"v1","addr":"172.21.15.42:6807","nonce":1137485788}]},"public_addr":"172.21.15.42:6803/1137485788","cluster_addr":"172.21.15.42:6805/1137485788","heartbeat_back_addr":"172.21.15.42:6809/1137485788","heartbeat_front_addr":"172.21.15.42:6807/1137485788","state":["exists","up"]},{"osd":1,"uuid":"7e1a8d54-070c-49ef-a2a6-f2fe66c292ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6801","nonce":3462838566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6803","nonce":3462838566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6807","nonce":3462838566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3462838566},{"type":"v1","addr":"172.21.15.119:6805","nonce":3462838566}]},"public_addr":"172.21.15.119:6801/3462838566","cluster_addr":"172.21.15.119:6803/3462838566","heartbeat_back_addr":"172.21.15.119:6807/3462838566","heartbeat_front_addr":"172.21.15.119:6805/3462838566","state":["exists","up"]},{"osd":2,"uuid":"fb3ebf6a-d807-4ead-89b0-bd81dcf650e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6801","nonce":2837276479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6803","nonce":2837276479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6807","nonce":2837276479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2837276479},{"type":"v1","addr":"172.21.15.136:6805","nonce":2837276479}]},"public_addr":"172.21.15.136:6801/2837276479","cluster_addr":"172.21.15.136:6803/2837276479","heartbeat_back_addr":"172.21.15.136:6807/2837276479","heartbeat_front_addr":"172.21.15.136:6805/2837276479","state":["exists","up"]},{"osd":3,"uuid":"bcfd2c44-146b-4495-92e4-ad2e3c85d16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6811","nonce":3878628201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6813","nonce":3878628201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6817","nonce":3878628201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3878628201},{"type":"v1","addr":"172.21.15.42:6815","nonce":3878628201}]},"public_addr":"172.21.15.42:6811/3878628201","cluster_addr":"172.21.15.42:6813/3878628201","heartbeat_back_addr":"172.21.15.42:6817/3878628201","heartbeat_front_addr":"172.21.15.42:6815/3878628201","state":["exists","up"]},{"osd":4,"uuid":"97a77fed-dfbe-45a5-8048-e1424bfee89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6809","nonce":239664950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6811","nonce":239664950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6815","nonce":239664950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":239664950},{"type":"v1","addr":"172.21.15.119:6813","nonce":239664950}]},"public_addr":"172.21.15.119:6809/239664950","cluster_addr":"172.21.15.119:6811/239664950","heartbeat_back_addr":"172.21.15.119:6815/239664950","heartbeat_front_addr":"172.21.15.119:6813/239664950","state":["exists","up"]},{"osd":5,"uuid":"68dc50d1-ecd2-4abc-af46-3ad102d339be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6809","nonce":2791487312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6811","nonce":2791487312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6815","nonce":2791487312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2791487312},{"type":"v1","addr":"172.21.15.136:6813","nonce":2791487312}]},"public_addr":"172.21.15.136:6809/2791487312","cluster_addr":"172.21.15.136:6811/2791487312","heartbeat_back_addr":"172.21.15.136:6815/2791487312","heartbeat_front_addr":"172.21.15.136:6813/2791487312","state":["exists","up"]},{"osd":6,"uuid":"99e7959d-3257-4c03-b302-c98052f009e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6819","nonce":515757893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6821","nonce":515757893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6825","nonce":515757893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":515757893},{"type":"v1","addr":"172.21.15.42:6823","nonce":515757893}]},"public_addr":"172.21.15.42:6819/515757893","cluster_addr":"172.21.15.42:6821/515757893","heartbeat_back_addr":"172.21.15.42:6825/515757893","heartbeat_front_addr":"172.21.15.42:6823/515757893","state":["exists","up"]},{"osd":7,"uuid":"c430bbfa-80fb-485a-a186-ee581a9a8196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6817","nonce":1789109628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6819","nonce":1789109628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6823","nonce":1789109628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1789109628},{"type":"v1","addr":"172.21.15.119:6821","nonce":1789109628}]},"public_addr":"172.21.15.119:6817/1789109628","cluster_addr":"172.21.15.119:6819/1789109628","heartbeat_back_addr":"172.21.15.119:6823/1789109628","heartbeat_front_addr":"172.21.15.119:6821/1789109628","state":["exists","up"]},{"osd":8,"uuid":"c1b72976-753c-4cee-9be5-867ebd0479ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6817","nonce":509882015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6819","nonce":509882015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6823","nonce":509882015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":509882015},{"type":"v1","addr":"172.21.15.136:6821","nonce":509882015}]},"public_addr":"172.21.15.136:6817/509882015","cluster_addr":"172.21.15.136:6819/509882015","heartbeat_back_addr":"172.21.15.136:6823/509882015","heartbeat_front_addr":"172.21.15.136:6821/509882015","state":["exists","up"]},{"osd":9,"uuid":"4d19ae72-048c-4680-bf7d-2b2f2bf353f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6827","nonce":1422185869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6829","nonce":1422185869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6833","nonce":1422185869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1422185869},{"type":"v1","addr":"172.21.15.42:6831","nonce":1422185869}]},"public_addr":"172.21.15.42:6827/1422185869","cluster_addr":"172.21.15.42:6829/1422185869","heartbeat_back_addr":"172.21.15.42:6833/1422185869","heartbeat_front_addr":"172.21.15.42:6831/1422185869","state":["exists","up"]},{"osd":10,"uuid":"f3e7f4b5-f77f-42f8-8bc4-84f5f6fbf2d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6825","nonce":3868326698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6827","nonce":3868326698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6831","nonce":3868326698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3868326698},{"type":"v1","addr":"172.21.15.119:6829","nonce":3868326698}]},"public_addr":"172.21.15.119:6825/3868326698","cluster_addr":"172.21.15.119:6827/3868326698","heartbeat_back_addr":"172.21.15.119:6831/3868326698","heartbeat_front_addr":"172.21.15.119:6829/3868326698","state":["exists","up"]},{"osd":11,"uuid":"bf7c32db-7617-4fe4-ab70-4bf8efcb0869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6825","nonce":2421468523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6827","nonce":2421468523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6831","nonce":2421468523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2421468523},{"type":"v1","addr":"172.21.15.136:6829","nonce":2421468523}]},"public_addr":"172.21.15.136:6825/2421468523","cluster_addr":"172.21.15.136:6827/2421468523","heartbeat_back_addr":"172.21.15.136:6831/2421468523","heartbeat_front_addr":"172.21.15.136:6829/2421468523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:18.821528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:07:42.871289+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:06.617782+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:32.195442+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:08:57.018485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:22.080221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:09:48.154883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:13.742473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:10:39.458815+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:06.530706+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:31.686856+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T04:11:58.282934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1061357309":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/2954927866":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6801/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2757555113":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/1914532345":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/3686333836":"2024-03-13T04:15:11.144062+0000","172.21.15.42:0/956219817":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/4126408073":"2024-03-13T04:15:11.144062+0000","172.21.15.42:6800/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/2257992536":"2024-03-13T04:05:44.816230+0000","172.21.15.42:0/1532115047":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/3492564527":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6800/2755197547":"2024-03-13T04:05:17.928065+0000","172.21.15.42:0/331646616":"2024-03-13T04:05:17.928065+0000","172.21.15.42:6801/2670488995":"2024-03-13T04:05:44.816230+0000","172.21.15.42:6801/2755197547":"2024-03-13T04:05:17.928065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T04:26:10.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faadc0a1c20 msgr2=0x7faadc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faadc0a1c20 0x7faadc0a40e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faaec0058c0 tx=0x7faaec005d30 comp rx=0 tx=0).stop 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01976d0 msgr2=0x7faaf0197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01976d0 0x7faaf0197b50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7faae800b970 tx=0x7faae800be30 comp rx=0 tx=0).stop 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 shutdown_connections 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7faadc0a1c20 0x7faadc0a40e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaf01976d0 0x7faaf0197b50 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf01053f0 0x7faaf019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 --2- 172.21.15.42:0/556332643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf0100a00 0x7faaf019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.985+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 >> 172.21.15.42:0/556332643 conn(0x7faaf0076610 msgr2=0x7faaf00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.986+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 shutdown_connections 2024-03-12T04:26:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:10.986+0000 7faaf770f640 1 -- 172.21.15.42:0/556332643 wait complete. 2024-03-12T04:26:11.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-12T04:26:11.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.103+0000 7f55c600f640 1 -- 172.21.15.42:0/1894624625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 msgr2=0x7f55c010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.103+0000 7f55c600f640 1 --2- 172.21.15.42:0/1894624625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c010cf00 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009cd0 tx=0x7f55b802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.103+0000 7f55c600f640 1 -- 172.21.15.42:0/1894624625 shutdown_connections 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.103+0000 7f55c600f640 1 --2- 172.21.15.42:0/1894624625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c010cf00 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.103+0000 7f55c600f640 1 --2- 172.21.15.42:0/1894624625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0100c10 0x7f55c0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.103+0000 7f55c600f640 1 --2- 172.21.15.42:0/1894624625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c0100260 0x7f55c0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.103+0000 7f55c600f640 1 -- 172.21.15.42:0/1894624625 >> 172.21.15.42:0/1894624625 conn(0x7f55c0077180 msgr2=0x7f55c0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 -- 172.21.15.42:0/1894624625 shutdown_connections 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 -- 172.21.15.42:0/1894624625 wait complete. 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 Processor -- start 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 -- start start 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c0100260 0x7f55c0196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c01969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c019ad50 0x7f55c019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c019b740 con 0x7f55c0105330 2024-03-12T04:26:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c019b8b0 con 0x7f55c0100260 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.104+0000 7f55c600f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c019ba20 con 0x7f55c019ad50 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c01969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c01969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39740/0 (socket says 172.21.15.42:39740) 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 -- 172.21.15.42:0/1932972509 learned_addr learned my addr 172.21.15.42:0/1932972509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55bf7fe640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c0100260 0x7f55c0196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 -- 172.21.15.42:0/1932972509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c019ad50 msgr2=0x7f55c019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c019ad50 0x7f55c019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 -- 172.21.15.42:0/1932972509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c0100260 msgr2=0x7f55c0196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c0100260 0x7f55c0196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 -- 172.21.15.42:0/1932972509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009900 con 0x7f55c0105330 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55beffd640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c01969d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c970 tx=0x7f55b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55bcff9640 1 -- 172.21.15.42:0/1932972509 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4004440 con 0x7f55c0105330 2024-03-12T04:26:11.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55bcff9640 1 -- 172.21.15.42:0/1932972509 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f55b4007d10 con 0x7f55c0105330 2024-03-12T04:26:11.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c01a3000 con 0x7f55c0105330 2024-03-12T04:26:11.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55bcff9640 1 -- 172.21.15.42:0/1932972509 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b400f450 con 0x7f55c0105330 2024-03-12T04:26:11.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.105+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c01a34b0 con 0x7f55c0105330 2024-03-12T04:26:11.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.107+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c0067f70 con 0x7f55c0105330 2024-03-12T04:26:11.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.107+0000 7f55bcff9640 1 -- 172.21.15.42:0/1932972509 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f55b4003680 con 0x7f55c0105330 2024-03-12T04:26:11.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.108+0000 7f55bcff9640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55a80a1ba0 0x7f55a80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.108+0000 7f55bf7fe640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55a80a1ba0 0x7f55a80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:11.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.108+0000 7f55bcff9640 1 -- 172.21.15.42:0/1932972509 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f55b40cda40 con 0x7f55c0105330 2024-03-12T04:26:11.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.108+0000 7f55bf7fe640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55a80a1ba0 0x7f55a80a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f55b0005e10 tx=0x7f55b0005da0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:11.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.112+0000 7f55bcff9640 1 -- 172.21.15.42:0/1932972509 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55b4096910 con 0x7f55c0105330 2024-03-12T04:26:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.374+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55c0100c10 con 0x7f55c0105330 2024-03-12T04:26:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.377+0000 7f55bcff9640 1 -- 172.21.15.42:0/1932972509 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55b4099fd0 con 0x7f55c0105330 2024-03-12T04:26:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55a80a1ba0 msgr2=0x7f55a80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55a80a1ba0 0x7f55a80a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f55b0005e10 tx=0x7f55b0005da0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 msgr2=0x7f55c01969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c01969d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c970 tx=0x7f55b400ce30 comp rx=0 tx=0).stop 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 shutdown_connections 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f55a80a1ba0 0x7f55a80a4060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c019ad50 0x7f55c019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0105330 0x7f55c01969d0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 --2- 172.21.15.42:0/1932972509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c0100260 0x7f55c0196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 >> 172.21.15.42:0/1932972509 conn(0x7f55c0077180 msgr2=0x7f55c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 shutdown_connections 2024-03-12T04:26:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.381+0000 7f55c600f640 1 -- 172.21.15.42:0/1932972509 wait complete. 2024-03-12T04:26:11.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-12T04:26:11.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.511+0000 7f3fda849640 1 -- 172.21.15.42:0/2228554015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd4102ce0 msgr2=0x7f3fd410b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.511+0000 7f3fda849640 1 --2- 172.21.15.42:0/2228554015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd4102ce0 0x7f3fd410b1c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009900 tx=0x7f3fc802bcd0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.511+0000 7f3fda849640 1 -- 172.21.15.42:0/2228554015 shutdown_connections 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.511+0000 7f3fda849640 1 --2- 172.21.15.42:0/2228554015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fd410b790 0x7f3fd410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.511+0000 7f3fda849640 1 --2- 172.21.15.42:0/2228554015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd4102ce0 0x7f3fd410b1c0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.511+0000 7f3fda849640 1 --2- 172.21.15.42:0/2228554015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fd41023c0 0x7f3fd41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.511+0000 7f3fda849640 1 -- 172.21.15.42:0/2228554015 >> 172.21.15.42:0/2228554015 conn(0x7f3fd4076400 msgr2=0x7f3fd4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 -- 172.21.15.42:0/2228554015 shutdown_connections 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 -- 172.21.15.42:0/2228554015 wait complete. 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 Processor -- start 2024-03-12T04:26:11.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 -- start start 2024-03-12T04:26:11.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fd41023c0 0x7f3fd4100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fd4102ce0 0x7f3fd4100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd410b790 0x7f3fd4100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41a5aa0 con 0x7f3fd410b790 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41a5c10 con 0x7f3fd41023c0 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.512+0000 7f3fda849640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41a5dd0 con 0x7f3fd4102ce0 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd410b790 0x7f3fd4100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd410b790 0x7f3fd4100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53198/0 (socket says 172.21.15.42:53198) 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 -- 172.21.15.42:0/2098213269 learned_addr learned my addr 172.21.15.42:0/2098213269 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd37fe640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fd4102ce0 0x7f3fd4100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd3fff640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fd41023c0 0x7f3fd4100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 -- 172.21.15.42:0/2098213269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fd4102ce0 msgr2=0x7f3fd4100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fd4102ce0 0x7f3fd4100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 -- 172.21.15.42:0/2098213269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fd41023c0 msgr2=0x7f3fd4100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fd41023c0 0x7f3fd4100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 -- 172.21.15.42:0/2098213269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009560 con 0x7f3fd410b790 2024-03-12T04:26:11.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd3fff640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fd41023c0 0x7f3fd4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T04:26:11.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd8de6640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd410b790 0x7f3fd4100a90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00eb40 tx=0x7f3fcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:11.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd17fa640 1 -- 172.21.15.42:0/2098213269 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc004b20 con 0x7f3fd410b790 2024-03-12T04:26:11.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd17fa640 1 -- 172.21.15.42:0/2098213269 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fcc004c80 con 0x7f3fd410b790 2024-03-12T04:26:11.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fd17fa640 1 -- 172.21.15.42:0/2098213269 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc010690 con 0x7f3fd410b790 2024-03-12T04:26:11.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd41a60b0 con 0x7f3fd410b790 2024-03-12T04:26:11.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.513+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd41a6540 con 0x7f3fd410b790 2024-03-12T04:26:11.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.514+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd4069f70 con 0x7f3fd410b790 2024-03-12T04:26:11.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.516+0000 7f3fd17fa640 1 -- 172.21.15.42:0/2098213269 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f3fcc005000 con 0x7f3fd410b790 2024-03-12T04:26:11.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.516+0000 7f3fd17fa640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3fbc0a1c20 0x7f3fbc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:11.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.517+0000 7f3fd17fa640 1 -- 172.21.15.42:0/2098213269 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8914+0+0 (secure 0 0 0) 0x7f3fcc014070 con 0x7f3fd410b790 2024-03-12T04:26:11.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.517+0000 7f3fd3fff640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3fbc0a1c20 0x7f3fbc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:11.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.517+0000 7f3fd3fff640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3fbc0a1c20 0x7f3fbc0a40e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009c20 tx=0x7f3fc4005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:11.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.520+0000 7f3fd17fa640 1 -- 172.21.15.42:0/2098213269 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fcc096d80 con 0x7f3fd410b790 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2826797036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/556332643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1932972509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1932972509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: osdmap e100: 12 total, 12 up, 12 in 2024-03-12T04:26:11.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mon[36119]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mds[66780]: -- 172.21.15.119:0/201856982 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ef2aa61a0 con 0x561ef2a7cc00 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: starting mds.b at 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k[65836]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k[65836]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mds[65840]: -- 172.21.15.119:0/2414236177 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b8c0541a0 con 0x559b8c02b000 2024-03-12T04:26:11.697 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k[65836]: starting mds.k at 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e[66159]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e[66159]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mds[66163]: -- 172.21.15.119:0/1789835462 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b0ee1c1a0 con 0x561b0edf2c00 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e[66159]: starting mds.e at 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h[66466]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h[66466]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-mds[66470]: -- 172.21.15.119:0/747091818 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5606e1c6e1a0 con 0x5606e1c45000 2024-03-12T04:26:11.698 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:26:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h[66466]: starting mds.h at 2024-03-12T04:26:11.707 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f[65511]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.708 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f[65511]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.708 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f[65511]: starting mds.f at 2024-03-12T04:26:11.708 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l[65191]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.708 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l[65191]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.708 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l[65191]: starting mds.l at 2024-03-12T04:26:11.708 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c[65817]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.708 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c[65817]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c[65817]: starting mds.c at 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2826797036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/556332643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1932972509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1932972509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: osdmap e100: 12 total, 12 up, 12 in 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mon[35571]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-12T04:26:11.709 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i[66121]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.710 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i[66121]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.710 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-mds[66125]: -- 172.21.15.136:0/803823909 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1c93a61a0 con 0x55f1c937cc00 2024-03-12T04:26:11.710 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:26:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i[66121]: starting mds.i at 2024-03-12T04:26:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:11.740+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3fd4106880 con 0x7f3fd410b790 2024-03-12T04:26:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a[32256]: 2024-03-12T04:26:11.375+0000 7fd627630700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T04:26:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2826797036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T04:26:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/556332643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T04:26:11.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1932972509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1932972509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: osdmap e100: 12 total, 12 up, 12 in 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mon[32260]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g[70855]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g[70855]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mds[70859]: -- 172.21.15.42:0/3796910229 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d137ade1a0 con 0x55d137ab4c00 2024-03-12T04:26:11.817 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g[70855]: starting mds.g at 2024-03-12T04:26:11.818 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a[71204]: ignoring --setuser ceph since I am not root 2024-03-12T04:26:11.818 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a[71204]: ignoring --setgroup ceph since I am not root 2024-03-12T04:26:11.818 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-mds[71208]: -- 172.21.15.42:0/1779475735 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d9a8d81a0 con 0x555d9a8aec00 2024-03-12T04:26:11.818 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:26:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a[71204]: starting mds.a at 2024-03-12T04:26:12.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.378+0000 7f3fd17fa640 1 -- 172.21.15.42:0/2098213269 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3fcc09a440 con 0x7f3fd410b790 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3fbc0a1c20 msgr2=0x7f3fbc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3fbc0a1c20 0x7f3fbc0a40e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009c20 tx=0x7f3fc4005f90 comp rx=0 tx=0).stop 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd410b790 msgr2=0x7f3fd4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd410b790 0x7f3fd4100a90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00eb40 tx=0x7f3fcc00c2d0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 shutdown_connections 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7f3fbc0a1c20 0x7f3fbc0a40e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fd410b790 0x7f3fd4100a90 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3fd4102ce0 0x7f3fd4100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 --2- 172.21.15.42:0/2098213269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fd41023c0 0x7f3fd4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 >> 172.21.15.42:0/2098213269 conn(0x7f3fd4076400 msgr2=0x7f3fd40fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:12.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 shutdown_connections 2024-03-12T04:26:12.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.381+0000 7f3fda849640 1 -- 172.21.15.42:0/2098213269 wait complete. 2024-03-12T04:26:12.398 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-12T04:26:12.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.503+0000 7fe0f7925640 1 -- 172.21.15.42:0/877068891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f0108660 msgr2=0x7fe0f0110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.503+0000 7fe0f7925640 1 --2- 172.21.15.42:0/877068891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f0108660 0x7fe0f0110230 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009cd0 tx=0x7fe0e802c0f0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.503+0000 7fe0f7925640 1 -- 172.21.15.42:0/877068891 shutdown_connections 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.503+0000 7fe0f7925640 1 --2- 172.21.15.42:0/877068891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f0108660 0x7fe0f0110230 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.503+0000 7fe0f7925640 1 --2- 172.21.15.42:0/877068891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f0103f40 0x7fe0f0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.503+0000 7fe0f7925640 1 --2- 172.21.15.42:0/877068891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0f0103590 0x7fe0f0103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 -- 172.21.15.42:0/877068891 >> 172.21.15.42:0/877068891 conn(0x7fe0f00fedf0 msgr2=0x7fe0f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 -- 172.21.15.42:0/877068891 shutdown_connections 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 -- 172.21.15.42:0/877068891 wait complete. 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 Processor -- start 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 -- start start 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0f0103590 0x7fe0f019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f0108660 0x7fe0f019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f01997a0 0x7fe0f0199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f019a270 con 0x7fe0f01997a0 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f019a3e0 con 0x7fe0f0103590 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f7925640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f019a5a0 con 0x7fe0f0108660 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.504+0000 7fe0f5ec2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f01997a0 0x7fe0f0199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f01997a0 0x7fe0f0199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53222/0 (socket says 172.21.15.42:53222) 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 -- 172.21.15.42:0/3515893840 learned_addr learned my addr 172.21.15.42:0/3515893840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f56c1640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0f0103590 0x7fe0f019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f4ec0640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f0108660 0x7fe0f019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 -- 172.21.15.42:0/3515893840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f0108660 msgr2=0x7fe0f019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f0108660 0x7fe0f019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 -- 172.21.15.42:0/3515893840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0f0103590 msgr2=0x7fe0f019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0f0103590 0x7fe0f019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 -- 172.21.15.42:0/3515893840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e8009900 con 0x7fe0f01997a0 2024-03-12T04:26:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f5ec2640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f01997a0 0x7fe0f0199c20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8004af0 tx=0x7fe0e8035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0e67fc640 1 -- 172.21.15.42:0/3515893840 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80329e0 con 0x7fe0f01997a0 2024-03-12T04:26:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0e67fc640 1 -- 172.21.15.42:0/3515893840 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe0e8035e90 con 0x7fe0f01997a0 2024-03-12T04:26:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f01a6490 con 0x7fe0f01997a0 2024-03-12T04:26:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0e67fc640 1 -- 172.21.15.42:0/3515893840 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8004680 con 0x7fe0f01997a0 2024-03-12T04:26:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.505+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f01a6910 con 0x7fe0f01997a0 2024-03-12T04:26:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.506+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f0104a60 con 0x7fe0f01997a0 2024-03-12T04:26:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.508+0000 7fe0e67fc640 1 -- 172.21.15.42:0/3515893840 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe0e803b040 con 0x7fe0f01997a0 2024-03-12T04:26:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.508+0000 7fe0e67fc640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0dc0a1ba0 0x7fe0dc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.509+0000 7fe0f56c1640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0dc0a1ba0 0x7fe0dc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.509+0000 7fe0e67fc640 1 -- 172.21.15.42:0/3515893840 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8914+0+0 (secure 0 0 0) 0x7fe0e80f0750 con 0x7fe0f01997a0 2024-03-12T04:26:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.509+0000 7fe0f56c1640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0dc0a1ba0 0x7fe0dc0a4060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00abe0 tx=0x7fe0ec00a2d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:12.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.512+0000 7fe0e67fc640 1 -- 172.21.15.42:0/3515893840 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0e80b9540 con 0x7fe0f01997a0 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: pgmap v342: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2098213269' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: Cluster is now healthy 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/2098213269' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.42:6838/3278918368,v1:172.21.15.42:6840/3278918368] up:boot 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.42:6839/3045028208,v1:172.21.15.42:6841/3045028208] up:boot 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6837/888921309,v1:172.21.15.136:6839/888921309] up:boot 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6832/2031476195,v1:172.21.15.136:6834/2031476195] up:boot 2024-03-12T04:26:12.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6832/725747425,v1:172.21.15.119:6834/725747425] up:boot 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6833/1799033097,v1:172.21.15.136:6835/1799033097] up:boot 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6836/1714799534,v1:172.21.15.119:6838/1714799534] up:boot 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6837/307950040,v1:172.21.15.119:6839/307950040] up:boot 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.136:6836/2016554959,v1:172.21.15.136:6838/2016554959] up:boot 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: mds.? [v2:172.21.15.119:6833/3498138362,v1:172.21.15.119:6835/3498138362] up:boot 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: fsmap 12 up:standby 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:26:12.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:26:12.569 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:12 smithi042 ceph-mon[32260]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:26:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:12.697+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0f010b930 con 0x7fe0f01997a0 2024-03-12T04:26:12.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: pgmap v342: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-03-12T04:26:12.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2098213269' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:26:12.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-12T04:26:12.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-12T04:26:12.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: Cluster is now healthy 2024-03-12T04:26:12.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/2098213269' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.42:6838/3278918368,v1:172.21.15.42:6840/3278918368] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.42:6839/3045028208,v1:172.21.15.42:6841/3045028208] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6837/888921309,v1:172.21.15.136:6839/888921309] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6832/2031476195,v1:172.21.15.136:6834/2031476195] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6832/725747425,v1:172.21.15.119:6834/725747425] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6833/1799033097,v1:172.21.15.136:6835/1799033097] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6836/1714799534,v1:172.21.15.119:6838/1714799534] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6837/307950040,v1:172.21.15.119:6839/307950040] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.136:6836/2016554959,v1:172.21.15.136:6838/2016554959] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: mds.? [v2:172.21.15.119:6833/3498138362,v1:172.21.15.119:6835/3498138362] up:boot 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: fsmap 12 up:standby 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:26:12.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:26:12.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:26:12.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:26:12.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:26:12.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:26:12.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:26:12.948 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:12 smithi119 ceph-mon[36119]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:26:12.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: pgmap v342: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-03-12T04:26:12.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2098213269' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T04:26:12.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-12T04:26:12.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-12T04:26:12.958 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: Cluster is now healthy 2024-03-12T04:26:12.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/2098213269' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-12T04:26:12.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.42:6838/3278918368,v1:172.21.15.42:6840/3278918368] up:boot 2024-03-12T04:26:12.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.42:6839/3045028208,v1:172.21.15.42:6841/3045028208] up:boot 2024-03-12T04:26:12.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6837/888921309,v1:172.21.15.136:6839/888921309] up:boot 2024-03-12T04:26:12.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6832/2031476195,v1:172.21.15.136:6834/2031476195] up:boot 2024-03-12T04:26:12.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6832/725747425,v1:172.21.15.119:6834/725747425] up:boot 2024-03-12T04:26:12.959 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6833/1799033097,v1:172.21.15.136:6835/1799033097] up:boot 2024-03-12T04:26:12.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6836/1714799534,v1:172.21.15.119:6838/1714799534] up:boot 2024-03-12T04:26:12.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6837/307950040,v1:172.21.15.119:6839/307950040] up:boot 2024-03-12T04:26:12.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.136:6836/2016554959,v1:172.21.15.136:6838/2016554959] up:boot 2024-03-12T04:26:12.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: mds.? [v2:172.21.15.119:6833/3498138362,v1:172.21.15.119:6835/3498138362] up:boot 2024-03-12T04:26:12.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: fsmap 12 up:standby 2024-03-12T04:26:12.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T04:26:12.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T04:26:12.961 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T04:26:12.961 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T04:26:12.961 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T04:26:12.961 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T04:26:12.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T04:26:12.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T04:26:12.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T04:26:12.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:12 smithi136 ceph-mon[35571]: from='mgr.15522 172.21.15.42:0/2609360324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T04:26:13.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.473+0000 7fe0e67fc640 1 -- 172.21.15.42:0/3515893840 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe0e80bcc00 con 0x7fe0f01997a0 2024-03-12T04:26:13.476 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-03-12T04:26:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.476+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0dc0a1ba0 msgr2=0x7fe0dc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.476+0000 7fe0f7925640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0dc0a1ba0 0x7fe0dc0a4060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00abe0 tx=0x7fe0ec00a2d0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.476+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f01997a0 msgr2=0x7fe0f0199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.476+0000 7fe0f7925640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f01997a0 0x7fe0f0199c20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8004af0 tx=0x7fe0e8035ca0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 shutdown_connections 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe0dc0a1ba0 0x7fe0dc0a4060 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0f01997a0 0x7fe0f0199c20 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f0108660 0x7fe0f019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 --2- 172.21.15.42:0/3515893840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0f0103590 0x7fe0f019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 >> 172.21.15.42:0/3515893840 conn(0x7fe0f00fedf0 msgr2=0x7fe0f0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 shutdown_connections 2024-03-12T04:26:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.477+0000 7fe0f7925640 1 -- 172.21.15.42:0/3515893840 wait complete. 2024-03-12T04:26:13.494 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 -- 172.21.15.42:0/3686360237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe234102d10 msgr2=0x7fe23410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 --2- 172.21.15.42:0/3686360237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe234102d10 0x7fe23410b1f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009900 tx=0x7fe21c02bcd0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 -- 172.21.15.42:0/3686360237 shutdown_connections 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 --2- 172.21.15.42:0/3686360237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe23410b7c0 0x7fe23410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 --2- 172.21.15.42:0/3686360237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe234102d10 0x7fe23410b1f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 --2- 172.21.15.42:0/3686360237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2341023f0 0x7fe2341027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 -- 172.21.15.42:0/3686360237 >> 172.21.15.42:0/3686360237 conn(0x7fe2340771e0 msgr2=0x7fe2340775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 -- 172.21.15.42:0/3686360237 shutdown_connections 2024-03-12T04:26:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.604+0000 7fe239102640 1 -- 172.21.15.42:0/3686360237 wait complete. 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 Processor -- start 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 -- start start 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2341023f0 0x7fe234197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe234102d10 0x7fe234197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23410b7c0 0x7fe23419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419c600 con 0x7fe23410b7c0 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419c770 con 0x7fe234102d10 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe239102640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419c930 con 0x7fe2341023f0 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe23359e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23410b7c0 0x7fe23419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe23359e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23410b7c0 0x7fe23419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53234/0 (socket says 172.21.15.42:53234) 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe23359e640 1 -- 172.21.15.42:0/1247797165 learned_addr learned my addr 172.21.15.42:0/1247797165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.605+0000 7fe232d9d640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2341023f0 0x7fe234197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe22bfff640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe234102d10 0x7fe234197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe23359e640 1 -- 172.21.15.42:0/1247797165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2341023f0 msgr2=0x7fe234197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe23359e640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2341023f0 0x7fe234197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe23359e640 1 -- 172.21.15.42:0/1247797165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe234102d10 msgr2=0x7fe234197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe23359e640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe234102d10 0x7fe234197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe23359e640 1 -- 172.21.15.42:0/1247797165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c009560 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe23359e640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23410b7c0 0x7fe23419c000 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00eaa0 tx=0x7fe22c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe230d99640 1 -- 172.21.15.42:0/1247797165 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00cdb0 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe230d99640 1 -- 172.21.15.42:0/1247797165 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe22c005020 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23419cc10 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe230d99640 1 -- 172.21.15.42:0/1247797165 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0116b0 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.606+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2341a4490 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.607+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2340683c0 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.608+0000 7fe230d99640 1 -- 172.21.15.42:0/1247797165 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe22c005190 con 0x7fe23410b7c0 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.609+0000 7fe230d99640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2200a1ba0 0x7fe2200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T04:26:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.609+0000 7fe232d9d640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2200a1ba0 0x7fe2200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T04:26:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.609+0000 7fe230d99640 1 -- 172.21.15.42:0/1247797165 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8418+0+0 (secure 0 0 0) 0x7fe22c0cd8b0 con 0x7fe23410b7c0 2024-03-12T04:26:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.609+0000 7fe232d9d640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2200a1ba0 0x7fe2200a4060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe22400bbe0 tx=0x7fe22400b390 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T04:26:13.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.613+0000 7fe230d99640 1 -- 172.21.15.42:0/1247797165 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe22c096830 con 0x7fe23410b7c0 2024-03-12T04:26:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:13.796+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2341068b0 con 0x7fe23410b7c0 2024-03-12T04:26:13.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:13 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3515893840' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T04:26:13.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:13 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3515893840' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T04:26:13.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:13 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3515893840' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T04:26:14.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.474+0000 7fe230d99640 1 -- 172.21.15.42:0/1247797165 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe22c099ef0 con 0x7fe23410b7c0 2024-03-12T04:26:14.477 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-03-12T04:26:14.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2200a1ba0 msgr2=0x7fe2200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:14.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2200a1ba0 0x7fe2200a4060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe22400bbe0 tx=0x7fe22400b390 comp rx=0 tx=0).stop 2024-03-12T04:26:14.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23410b7c0 msgr2=0x7fe23419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T04:26:14.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23410b7c0 0x7fe23419c000 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00eaa0 tx=0x7fe22c00ef60 comp rx=0 tx=0).stop 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 shutdown_connections 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:6800/2974863619,v1:172.21.15.42:6801/2974863619] conn(0x7fe2200a1ba0 0x7fe2200a4060 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23410b7c0 0x7fe23419c000 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe234102d10 0x7fe234197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 --2- 172.21.15.42:0/1247797165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2341023f0 0x7fe234197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 >> 172.21.15.42:0/1247797165 conn(0x7fe2340771e0 msgr2=0x7fe2340fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 shutdown_connections 2024-03-12T04:26:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-03-12T04:26:14.477+0000 7fe239102640 1 -- 172.21.15.42:0/1247797165 wait complete. 2024-03-12T04:26:14.500 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:26:14.546 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:26:14.599 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:26:14.645 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-12T04:26:14.646 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T04:26:14.668 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T04:26:14.688 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T04:26:14.709 INFO:tasks.cephadm:Stopping all daemons... 2024-03-12T04:26:14.709 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-12T04:26:14.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a 2024-03-12T04:26:14.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:14 smithi042 ceph-mon[32260]: pgmap v344: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-03-12T04:26:14.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:14 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/3515893840' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-12T04:26:14.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:14 smithi042 ceph-mon[32260]: osdmap e101: 12 total, 12 up, 12 in 2024-03-12T04:26:14.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:14 smithi042 ceph-mon[32260]: from='client.? 172.21.15.42:0/1247797165' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T04:26:14.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:14 smithi119 ceph-mon[36119]: pgmap v344: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-03-12T04:26:14.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:14 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/3515893840' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-12T04:26:14.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:14 smithi119 ceph-mon[36119]: osdmap e101: 12 total, 12 up, 12 in 2024-03-12T04:26:14.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:14 smithi119 ceph-mon[36119]: from='client.? 172.21.15.42:0/1247797165' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T04:26:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:14 smithi136 ceph-mon[35571]: pgmap v344: 97 pgs: 97 active+clean; 2.2 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-03-12T04:26:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:14 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/3515893840' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-12T04:26:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:14 smithi136 ceph-mon[35571]: osdmap e101: 12 total, 12 up, 12 in 2024-03-12T04:26:14.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:14 smithi136 ceph-mon[35571]: from='client.? 172.21.15.42:0/1247797165' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T04:26:15.026 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:14 smithi042 systemd[1]: Stopping Ceph mon.a for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:15.290 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a[32256]: 2024-03-12T04:26:15.023+0000 7fd62ce3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:15.290 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a[32256]: 2024-03-12T04:26:15.023+0000 7fd62ce3b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-12T04:26:15.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:15 smithi042 podman[99183]: 2024-03-12 04:26:15.288687425 +0000 UTC m=+0.384702235 container died 67e30e1d77b0d260bf52b00c42780cb2061fc24f46d859b5542315ff0428cd3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-12T04:26:15.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:15 smithi042 podman[99183]: 2024-03-12 04:26:15.7151343 +0000 UTC m=+0.811149119 container cleanup 67e30e1d77b0d260bf52b00c42780cb2061fc24f46d859b5542315ff0428cd3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-12T04:26:15.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:15 smithi042 bash[99183]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a 2024-03-12T04:26:16.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service' 2024-03-12T04:26:16.168 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:15 smithi042 podman[99197]: 2024-03-12 04:26:15.899755618 +0000 UTC m=+0.610043612 container remove 67e30e1d77b0d260bf52b00c42780cb2061fc24f46d859b5542315ff0428cd3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-12T04:26:16.168 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:16 smithi042 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-12T04:26:16.168 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:16 smithi042 systemd[1]: Stopped Ceph mon.a for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:26:16.168 INFO:journalctl@ceph.mon.a.smithi042.stdout:Mar 12 04:26:16 smithi042 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.a.service: Consumed 20.695s CPU time. 2024-03-12T04:26:16.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:16.196 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-12T04:26:16.196 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-12T04:26:16.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.b 2024-03-12T04:26:16.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:16 smithi119 systemd[1]: Stopping Ceph mon.b for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:16.838 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-b[36115]: 2024-03-12T04:26:16.500+0000 7f357aec2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:16.838 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-b[36115]: 2024-03-12T04:26:16.500+0000 7f357aec2700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-12T04:26:16.838 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:16 smithi119 podman[72162]: 2024-03-12 04:26:16.552548317 +0000 UTC m=+0.174218650 container died e687522f7f21f9bc97458c7441278fe93e0f2f96cacf0f19a7f80c649c6aa0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397) 2024-03-12T04:26:17.171 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:16 smithi119 podman[72162]: 2024-03-12 04:26:16.926918137 +0000 UTC m=+0.548588479 container cleanup e687522f7f21f9bc97458c7441278fe93e0f2f96cacf0f19a7f80c649c6aa0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:26:17.172 INFO:journalctl@ceph.mon.b.smithi119.stdout:Mar 12 04:26:16 smithi119 bash[72162]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-b 2024-03-12T04:26:17.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.b.service' 2024-03-12T04:26:17.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:17.357 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-12T04:26:17.357 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-12T04:26:17.357 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.c 2024-03-12T04:26:17.681 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:17 smithi136 systemd[1]: Stopping Ceph mon.c for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:17.946 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-c[35564]: 2024-03-12T04:26:17.678+0000 7fe2cc004700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:17.946 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-c[35564]: 2024-03-12T04:26:17.678+0000 7fe2cc004700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-12T04:26:18.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:17 smithi136 podman[71599]: 2024-03-12 04:26:17.944883493 +0000 UTC m=+0.394111688 container died 0b107daf45d65dc70b1cd5bef3b8a8822c6a582a35016271fd06e19eb37f7a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-c, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-12T04:26:18.558 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:18 smithi136 podman[71599]: 2024-03-12 04:26:18.346297566 +0000 UTC m=+0.795525761 container cleanup 0b107daf45d65dc70b1cd5bef3b8a8822c6a582a35016271fd06e19eb37f7a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-c, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-12T04:26:18.558 INFO:journalctl@ceph.mon.c.smithi136.stdout:Mar 12 04:26:18 smithi136 bash[71599]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mon-c 2024-03-12T04:26:18.771 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mon.c.service' 2024-03-12T04:26:18.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:18.821 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-12T04:26:18.821 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-12T04:26:18.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.x 2024-03-12T04:26:19.146 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:26:18 smithi042 systemd[1]: Stopping Ceph mgr.x for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:19.497 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:26:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:26:19.143+0000 7ff7dfa4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:19.497 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:26:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x[32493]: 2024-03-12T04:26:19.143+0000 7ff7dfa4a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-12T04:26:19.497 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:26:19 smithi042 podman[99307]: 2024-03-12 04:26:19.176070264 +0000 UTC m=+0.152259724 container died 58dfb605d35e894405fd599cdf53d3a84a4db3ff0c250d94dd5c611ecbceb75b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:26:19.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:26:19 smithi042 podman[99307]: 2024-03-12 04:26:19.587022802 +0000 UTC m=+0.563212305 container cleanup 58dfb605d35e894405fd599cdf53d3a84a4db3ff0c250d94dd5c611ecbceb75b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5) 2024-03-12T04:26:19.816 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Mar 12 04:26:19 smithi042 bash[99307]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-x 2024-03-12T04:26:19.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.x.service' 2024-03-12T04:26:19.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:19.997 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-12T04:26:19.997 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-12T04:26:19.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.y 2024-03-12T04:26:20.346 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:20 smithi119 systemd[1]: Stopping Ceph mgr.y for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:20.659 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:20 smithi119 podman[72284]: 2024-03-12 04:26:20.344403722 +0000 UTC m=+0.162042399 container died 35294ab596e7fe167187d96baaaa40970c72691e01ca980d0e3929e53edf8566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-12T04:26:20.946 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:20 smithi119 podman[72284]: 2024-03-12 04:26:20.76641941 +0000 UTC m=+0.584058138 container cleanup 35294ab596e7fe167187d96baaaa40970c72691e01ca980d0e3929e53edf8566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-12T04:26:20.946 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:20 smithi119 bash[72284]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y 2024-03-12T04:26:21.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.y.service' 2024-03-12T04:26:21.264 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:20 smithi119 podman[72298]: 2024-03-12 04:26:20.992837741 +0000 UTC m=+0.638885186 container remove 35294ab596e7fe167187d96baaaa40970c72691e01ca980d0e3929e53edf8566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-y, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-12T04:26:21.264 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:20 smithi119 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-12T04:26:21.264 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:21 smithi119 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-12T04:26:21.265 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:21 smithi119 systemd[1]: Stopped Ceph mgr.y for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:26:21.265 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Mar 12 04:26:21 smithi119 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.y.service: Consumed 20.403s CPU time. 2024-03-12T04:26:21.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:21.278 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-12T04:26:21.278 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-12T04:26:21.278 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.z 2024-03-12T04:26:21.707 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:26:21 smithi136 systemd[1]: Stopping Ceph mgr.z for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:22.027 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:26:21 smithi136 podman[71724]: 2024-03-12 04:26:21.714993667 +0000 UTC m=+0.247597009 container died 3682a4875a169f42e3368c4ab2fc151f7530bf4d87592527c4aaa4971a225edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:26:22.352 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:26:22 smithi136 podman[71724]: 2024-03-12 04:26:22.025996396 +0000 UTC m=+0.558599737 container cleanup 3682a4875a169f42e3368c4ab2fc151f7530bf4d87592527c4aaa4971a225edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-12T04:26:22.353 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:26:22 smithi136 bash[71724]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z 2024-03-12T04:26:22.353 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:26:22 smithi136 podman[71738]: 2024-03-12 04:26:22.252620261 +0000 UTC m=+0.537797993 container remove 3682a4875a169f42e3368c4ab2fc151f7530bf4d87592527c4aaa4971a225edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-94a5397) 2024-03-12T04:26:22.353 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Mar 12 04:26:22 smithi136 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-12T04:26:22.370 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mgr.z.service' 2024-03-12T04:26:22.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:22.407 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-12T04:26:22.407 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-12T04:26:22.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.0 2024-03-12T04:26:22.726 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:22 smithi042 systemd[1]: Stopping Ceph osd.0 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:23.066 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0[44527]: 2024-03-12T04:26:22.723+0000 7f3ff52fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:23.067 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0[44527]: 2024-03-12T04:26:22.723+0000 7f3ff52fc700 -1 osd.0 102 *** Got signal Terminated *** 2024-03-12T04:26:23.067 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0[44527]: 2024-03-12T04:26:22.723+0000 7f3ff52fc700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:26:28.453 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:28 smithi042 podman[99428]: 2024-03-12 04:26:28.172502366 +0000 UTC m=+5.575638971 container died bd36700dc75415f63ab3484ec458b3f6143dacf2a1d9c37a67348059f079e662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-12T04:26:28.816 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:28 smithi042 podman[99428]: 2024-03-12 04:26:28.666370399 +0000 UTC m=+6.069507004 container cleanup bd36700dc75415f63ab3484ec458b3f6143dacf2a1d9c37a67348059f079e662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-12T04:26:28.816 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:28 smithi042 bash[99428]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0 2024-03-12T04:26:29.280 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:28 smithi042 podman[99446]: 2024-03-12 04:26:28.926379012 +0000 UTC m=+0.749004581 container remove bd36700dc75415f63ab3484ec458b3f6143dacf2a1d9c37a67348059f079e662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, ceph=True) 2024-03-12T04:26:29.566 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:29 smithi042 podman[99537]: 2024-03-12 04:26:29.181565556 +0000 UTC m=+0.017686110 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:26:29.566 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:29 smithi042 podman[99537]: 2024-03-12 04:26:29.293869019 +0000 UTC m=+0.129989553 container create 66645a8e044192441a7a474b6a548e752683ff27d2075a92728452d88566781b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-94a5397) 2024-03-12T04:26:29.876 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:29 smithi042 podman[99537]: 2024-03-12 04:26:29.572152887 +0000 UTC m=+0.408273424 container init 66645a8e044192441a7a474b6a548e752683ff27d2075a92728452d88566781b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-12T04:26:29.876 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:29 smithi042 podman[99537]: 2024-03-12 04:26:29.575944552 +0000 UTC m=+0.412065093 container start 66645a8e044192441a7a474b6a548e752683ff27d2075a92728452d88566781b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-12T04:26:29.876 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:29 smithi042 podman[99537]: 2024-03-12 04:26:29.671760721 +0000 UTC m=+0.507881262 container attach 66645a8e044192441a7a474b6a548e752683ff27d2075a92728452d88566781b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-12T04:26:30.185 INFO:journalctl@ceph.osd.0.smithi042.stdout:Mar 12 04:26:29 smithi042 podman[99537]: 2024-03-12 04:26:29.874385806 +0000 UTC m=+0.710506347 container died 66645a8e044192441a7a474b6a548e752683ff27d2075a92728452d88566781b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:26:30.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.0.service' 2024-03-12T04:26:30.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:30.579 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-12T04:26:30.580 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-12T04:26:30.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.1 2024-03-12T04:26:30.897 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:30 smithi119 systemd[1]: Stopping Ceph osd.1 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:31.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1[40703]: 2024-03-12T04:26:30.894+0000 7f6732833700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:31.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1[40703]: 2024-03-12T04:26:30.894+0000 7f6732833700 -1 osd.1 102 *** Got signal Terminated *** 2024-03-12T04:26:31.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1[40703]: 2024-03-12T04:26:30.894+0000 7f6732833700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:26:36.664 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:36 smithi119 podman[72408]: 2024-03-12 04:26:36.372069556 +0000 UTC m=+5.597288638 container died ed958a788e4b0d8d4f704b830865f9772879e7b80cf23769a46eebfbeb6b9a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-12T04:26:36.946 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:36 smithi119 podman[72408]: 2024-03-12 04:26:36.753931735 +0000 UTC m=+5.979150816 container cleanup ed958a788e4b0d8d4f704b830865f9772879e7b80cf23769a46eebfbeb6b9a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-12T04:26:36.946 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:36 smithi119 bash[72408]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1 2024-03-12T04:26:37.374 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:37 smithi119 podman[72425]: 2024-03-12 04:26:37.039079215 +0000 UTC m=+0.661348465 container remove ed958a788e4b0d8d4f704b830865f9772879e7b80cf23769a46eebfbeb6b9a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:26:37.696 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:37 smithi119 podman[72516]: 2024-03-12 04:26:37.274812007 +0000 UTC m=+0.018092746 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:26:37.696 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:37 smithi119 podman[72516]: 2024-03-12 04:26:37.373310496 +0000 UTC m=+0.116591224 container create 9b1f705d76446510c8fadc0aad032cf872df68499fbe81efe80dcd97d41dc7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-94a5397) 2024-03-12T04:26:37.696 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:37 smithi119 podman[72516]: 2024-03-12 04:26:37.618105477 +0000 UTC m=+0.361386212 container init 9b1f705d76446510c8fadc0aad032cf872df68499fbe81efe80dcd97d41dc7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1-deactivate, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-12T04:26:37.696 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:37 smithi119 podman[72516]: 2024-03-12 04:26:37.621340244 +0000 UTC m=+0.364620972 container start 9b1f705d76446510c8fadc0aad032cf872df68499fbe81efe80dcd97d41dc7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0) 2024-03-12T04:26:38.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:37 smithi119 podman[72516]: 2024-03-12 04:26:37.728167591 +0000 UTC m=+0.471448353 container attach 9b1f705d76446510c8fadc0aad032cf872df68499fbe81efe80dcd97d41dc7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-12T04:26:38.196 INFO:journalctl@ceph.osd.1.smithi119.stdout:Mar 12 04:26:37 smithi119 podman[72516]: 2024-03-12 04:26:37.907241604 +0000 UTC m=+0.650522325 container died 9b1f705d76446510c8fadc0aad032cf872df68499fbe81efe80dcd97d41dc7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-1-deactivate, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:26:38.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.1.service' 2024-03-12T04:26:38.612 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:38.612 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-12T04:26:38.612 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-12T04:26:38.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.2 2024-03-12T04:26:38.939 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:38 smithi136 systemd[1]: Stopping Ceph osd.2 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:39.207 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2[40714]: 2024-03-12T04:26:38.935+0000 7fa1a1e57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:39.208 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2[40714]: 2024-03-12T04:26:38.935+0000 7fa1a1e57700 -1 osd.2 102 *** Got signal Terminated *** 2024-03-12T04:26:39.208 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2[40714]: 2024-03-12T04:26:38.935+0000 7fa1a1e57700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:26:44.691 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:44 smithi136 podman[71851]: 2024-03-12 04:26:44.387363309 +0000 UTC m=+5.574586697 container died 391789aa2a68b9470192977d825e72fdc05be7cb43ca9bc85899c9a21931c808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-03-12T04:26:44.957 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:44 smithi136 podman[71851]: 2024-03-12 04:26:44.787485977 +0000 UTC m=+5.974709405 container cleanup 391789aa2a68b9470192977d825e72fdc05be7cb43ca9bc85899c9a21931c808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-12T04:26:44.957 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:44 smithi136 bash[71851]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2 2024-03-12T04:26:45.364 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:45 smithi136 podman[71879]: 2024-03-12 04:26:45.067777891 +0000 UTC m=+0.678026457 container remove 391789aa2a68b9470192977d825e72fdc05be7cb43ca9bc85899c9a21931c808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-12T04:26:45.696 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:45 smithi136 podman[71966]: 2024-03-12 04:26:45.265478006 +0000 UTC m=+0.018859061 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:26:45.696 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:45 smithi136 podman[71966]: 2024-03-12 04:26:45.373498313 +0000 UTC m=+0.126879360 container create 4befa4e6001a2e6c565d83969f00de1be0e7521d4683309e66f971c2a80d19d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-12T04:26:45.696 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:45 smithi136 podman[71966]: 2024-03-12 04:26:45.594353075 +0000 UTC m=+0.347734138 container init 4befa4e6001a2e6c565d83969f00de1be0e7521d4683309e66f971c2a80d19d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:26:45.696 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:45 smithi136 podman[71966]: 2024-03-12 04:26:45.597585033 +0000 UTC m=+0.350966081 container start 4befa4e6001a2e6c565d83969f00de1be0e7521d4683309e66f971c2a80d19d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-12T04:26:45.696 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:45 smithi136 podman[71966]: 2024-03-12 04:26:45.694676614 +0000 UTC m=+0.448057673 container attach 4befa4e6001a2e6c565d83969f00de1be0e7521d4683309e66f971c2a80d19d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-03-12T04:26:45.957 INFO:journalctl@ceph.osd.2.smithi136.stdout:Mar 12 04:26:45 smithi136 podman[71966]: 2024-03-12 04:26:45.890306157 +0000 UTC m=+0.643687204 container died 4befa4e6001a2e6c565d83969f00de1be0e7521d4683309e66f971c2a80d19d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-12T04:26:46.574 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.2.service' 2024-03-12T04:26:46.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:46.620 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-12T04:26:46.621 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-12T04:26:46.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.3 2024-03-12T04:26:46.938 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:46 smithi042 systemd[1]: Stopping Ceph osd.3 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:47.209 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:46 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:26:46.935+0000 7f65d39ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:47.209 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:46 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:26:46.935+0000 7f65d39ac700 -1 osd.3 102 *** Got signal Terminated *** 2024-03-12T04:26:47.209 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:46 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:26:46.935+0000 7f65d39ac700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:26:49.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:48.744+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:50.066 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:49 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:26:49.758+0000 7f65cbf98700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.241271+0000 front 2024-03-12T04:26:27.241320+0000 (oldest deadline 2024-03-12T04:26:49.541415+0000) 2024-03-12T04:26:50.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:49.987+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:50.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:49.778+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:50.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:50 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:50.300+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:51.066 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:50 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:26:50.800+0000 7f65cbf98700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.241271+0000 front 2024-03-12T04:26:27.241320+0000 (oldest deadline 2024-03-12T04:26:49.541415+0000) 2024-03-12T04:26:51.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:50 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:50.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:51.209 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:51.028+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:51.210 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:50.782+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:51.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:51 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:51.291+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:51.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:51 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:51.172+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:52.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:51 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:51.893+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:52.205 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:51 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3[51351]: 2024-03-12T04:26:51.820+0000 7f65cbf98700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.241271+0000 front 2024-03-12T04:26:27.241320+0000 (oldest deadline 2024-03-12T04:26:49.541415+0000) 2024-03-12T04:26:52.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:52.039+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:52.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:51.826+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:52.533 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:52 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:52.202+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:52.533 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:52 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:52.266+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:52.816 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:52 smithi042 podman[99645]: 2024-03-12 04:26:52.531350971 +0000 UTC m=+5.721013236 container died 85b54009a77e8b311be4a50717ecf04206af5b9bde8a2bbd86574b78b32637b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.license=GPLv2) 2024-03-12T04:26:53.131 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:52 smithi042 podman[99645]: 2024-03-12 04:26:52.911454762 +0000 UTC m=+6.101117027 container cleanup 85b54009a77e8b311be4a50717ecf04206af5b9bde8a2bbd86574b78b32637b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-12T04:26:53.131 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:52 smithi042 bash[99645]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3 2024-03-12T04:26:53.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:52 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:52.873+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:53.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:53.024+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:53.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:52.783+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:53.542 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:53 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:53.285+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:53.542 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:53 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:53.178+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:53.543 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:53 smithi042 podman[99675]: 2024-03-12 04:26:53.129558287 +0000 UTC m=+0.595837502 container remove 85b54009a77e8b311be4a50717ecf04206af5b9bde8a2bbd86574b78b32637b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:26:53.816 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:53 smithi042 podman[99766]: 2024-03-12 04:26:53.443398516 +0000 UTC m=+0.017621137 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:26:53.816 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:53 smithi042 podman[99766]: 2024-03-12 04:26:53.566306223 +0000 UTC m=+0.140528898 container create d44608719b2c2713333e88aaa583b49db8edb06dae1348899d3b5ca4077d1ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-12T04:26:54.133 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:53 smithi042 podman[99766]: 2024-03-12 04:26:53.833539536 +0000 UTC m=+0.407762160 container init d44608719b2c2713333e88aaa583b49db8edb06dae1348899d3b5ca4077d1ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:26:54.133 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:53 smithi042 podman[99766]: 2024-03-12 04:26:53.836577459 +0000 UTC m=+0.410800071 container start d44608719b2c2713333e88aaa583b49db8edb06dae1348899d3b5ca4077d1ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3-deactivate, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:26:54.133 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:53 smithi042 podman[99766]: 2024-03-12 04:26:53.933659136 +0000 UTC m=+0.507881747 container attach d44608719b2c2713333e88aaa583b49db8edb06dae1348899d3b5ca4077d1ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:26:54.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:53 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:53.916+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:54.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:54.010+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:54.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:53.788+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:54.438 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:54 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:54.181+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:54.439 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:54 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:54.277+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:54.439 INFO:journalctl@ceph.osd.3.smithi042.stdout:Mar 12 04:26:54 smithi042 podman[99766]: 2024-03-12 04:26:54.131352732 +0000 UTC m=+0.705575339 container died d44608719b2c2713333e88aaa583b49db8edb06dae1348899d3b5ca4077d1ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-12T04:26:54.768 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.3.service' 2024-03-12T04:26:54.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:26:54.813 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-12T04:26:54.813 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-12T04:26:54.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.4 2024-03-12T04:26:55.127 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:26:54 smithi119 systemd[1]: Stopping Ceph osd.4 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:26:55.127 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:54 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:54.882+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:55.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:54.997+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:55.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:54.834+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:55.446 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:26:55 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:26:55.124+0000 7fd3e1893700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:26:55.446 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:26:55 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:26:55.124+0000 7fd3e1893700 -1 osd.4 102 *** Got signal Terminated *** 2024-03-12T04:26:55.446 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:26:55 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:26:55.124+0000 7fd3e1893700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:26:55.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:55 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:55.235+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:55.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:55 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:55.174+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:56.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:55 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:55.841+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:56.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:55.949+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:56.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:55.853+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:56.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:56 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:56.241+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:56.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:56 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:56.161+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:56.943 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:56.818+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:57.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:56 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:56.813+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:57.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:56 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:56.941+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:57.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:57 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:57.200+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:57.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:57 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:57.136+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:57.789 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:26:57 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:26:57.700+0000 7fd3dde87700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.042316+0000 front 2024-03-12T04:26:35.042367+0000 (oldest deadline 2024-03-12T04:26:56.742515+0000) 2024-03-12T04:26:57.789 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:26:57 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:26:57.519+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:26:58.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:57 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:57.786+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:58.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:57 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:57.831+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:58.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:57 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:57.950+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:58.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:57 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:57.950+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:26:58.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:58.248+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:58.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:58.249+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:26:58.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:58.159+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:58.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:58.159+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:26:58.826 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:26:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:26:58.705+0000 7fd3dde87700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.042316+0000 front 2024-03-12T04:26:35.042367+0000 (oldest deadline 2024-03-12T04:26:56.742515+0000) 2024-03-12T04:26:58.827 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:26:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:26:58.555+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:26:59.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:58 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:58.824+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:26:59.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:58.783+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:26:59.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:58.936+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:26:59.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:58 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:58.936+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:26:59.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:59.222+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:26:59.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:26:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:26:59.222+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:26:59.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:59.181+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:26:59.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:26:59 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:26:59.181+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:26:59.868 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:26:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4[47633]: 2024-03-12T04:26:59.745+0000 7fd3dde87700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.042316+0000 front 2024-03-12T04:26:35.042367+0000 (oldest deadline 2024-03-12T04:26:56.742515+0000) 2024-03-12T04:26:59.868 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:26:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:26:59.543+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:00.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:26:59 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:26:59.865+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:00.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:26:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:26:59.833+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:00.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:59.945+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:00.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:26:59 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:26:59.945+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:00.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:00.249+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:00.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:00.249+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:00.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:00.207+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:00.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:00 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:00.207+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:00.827 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:00 smithi119 podman[72635]: 2024-03-12 04:27:00.508982116 +0000 UTC m=+5.491810404 container died ca88ac741f7b73dff8b225c6b8868eee778349e67f8698f75543565be16021cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:27:00.828 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:00 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:00.495+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:00.885 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:00 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:00.521+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:01.162 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:00 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:00.881+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:01.163 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:00 smithi119 podman[72635]: 2024-03-12 04:27:00.933678453 +0000 UTC m=+5.916506747 container cleanup ca88ac741f7b73dff8b225c6b8868eee778349e67f8698f75543565be16021cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-12T04:27:01.163 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:00 smithi119 bash[72635]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4 2024-03-12T04:27:01.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:00 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:00.883+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:01.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:00 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:00.883+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:01.208 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:00 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:00.994+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:01.208 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:00 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:00.994+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:01.446 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:01 smithi119 podman[72652]: 2024-03-12 04:27:01.159828909 +0000 UTC m=+0.647624579 container remove ca88ac741f7b73dff8b225c6b8868eee778349e67f8698f75543565be16021cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-12T04:27:01.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:01.211+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:01.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:01.211+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:01.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:01.245+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:01.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:01 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:01.245+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:01.815 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:01 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:01.521+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:01.816 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:01 smithi119 podman[72745]: 2024-03-12 04:27:01.552519614 +0000 UTC m=+0.113553251 container create 9c367c847ab099ca3868aae53ad895577e636347d5040666f53ea5ea9bc9954e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397) 2024-03-12T04:27:01.816 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:01 smithi119 podman[72745]: 2024-03-12 04:27:01.457138869 +0000 UTC m=+0.018172514 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:01.816 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:01 smithi119 podman[72745]: 2024-03-12 04:27:01.813719322 +0000 UTC m=+0.374752966 container init 9c367c847ab099ca3868aae53ad895577e636347d5040666f53ea5ea9bc9954e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4-deactivate, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-03-12T04:27:01.860 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:01 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:01.565+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:02.110 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:01 smithi119 podman[72745]: 2024-03-12 04:27:01.817477346 +0000 UTC m=+0.378510987 container start 9c367c847ab099ca3868aae53ad895577e636347d5040666f53ea5ea9bc9954e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-12T04:27:02.110 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:01 smithi119 podman[72745]: 2024-03-12 04:27:01.905281426 +0000 UTC m=+0.466315087 container attach 9c367c847ab099ca3868aae53ad895577e636347d5040666f53ea5ea9bc9954e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:27:02.110 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:01 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:01.874+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:02.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:01 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:01.858+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:02.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:01 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:01.858+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:02.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:02 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:02.034+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:02.208 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:02 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:02.034+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:02.401 INFO:journalctl@ceph.osd.4.smithi119.stdout:Mar 12 04:27:02 smithi119 podman[72745]: 2024-03-12 04:27:02.107956585 +0000 UTC m=+0.668990224 container died 9c367c847ab099ca3868aae53ad895577e636347d5040666f53ea5ea9bc9954e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-12T04:27:02.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:02 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:02.200+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:02.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:02 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:02.200+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:02.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:02 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:02.290+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:02.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:02 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:02.290+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:02.696 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:02 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:02.550+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:02.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.4.service' 2024-03-12T04:27:02.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:02.776 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-12T04:27:02.776 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-12T04:27:02.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.5 2024-03-12T04:27:02.805 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:02 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:02.547+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:03.066 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:02 smithi136 systemd[1]: Stopping Ceph osd.5 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:03.066 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:02 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:02.844+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:03.066 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:02 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:02.844+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:03.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:02 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:02.886+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:03.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:02 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:02.886+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:03.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:03.063+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:03.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:03.063+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:03.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:03.100+0000 7fe9d4bf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:03.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:03.100+0000 7fe9d4bf0700 -1 osd.5 102 *** Got signal Terminated *** 2024-03-12T04:27:03.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:03.100+0000 7fe9d4bf0700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:27:03.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:03.242+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:03.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:03.242+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:03.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:03.254+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:03.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:03.254+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:03.800 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:03.524+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:03.919 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:03 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:03.549+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:04.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:03 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:03.917+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:04.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:03 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:03.917+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:04.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:04.018+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:04.207 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:04.018+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:04.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:03.797+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:04.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:03 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:03.797+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:04.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:04.275+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:04.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:04.275+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:04.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:04.212+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:04.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:04 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:04.212+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:04.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:04.541+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:04.957 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:04.520+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:04.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:04.760+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:04.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:04.760+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:04.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:04 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:04.760+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:05.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:04.944+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:05.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:04 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:04.944+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:05.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:05.042+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:05.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:05.042+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:05.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:05.232+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:05.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:05.232+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:05.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:05.249+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:05.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:05.249+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:05.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:05 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:05.249+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:05.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:05.583+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:05.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:05.583+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:05.957 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:05.569+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:05.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:05.804+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:05.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:05.804+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:05.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:05 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:05.804+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:06.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:05.980+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:06.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:05.980+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:06.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:05 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:05.980+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:06.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:06.092+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:06.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:06.092+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:06.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:06.218+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:06.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:06.218+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:06.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:06.254+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:06.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:06.254+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:06.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:06 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:06.254+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:06.712 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:06 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:06.539+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:06.712 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:06 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:06.539+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:06.957 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:06.615+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:06.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:06.770+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:06.958 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:06.770+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:06.958 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:06 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:06.770+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:07.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:07.004+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:07.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:07.004+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:07.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:07.004+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:07.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:07.103+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:24.989073+0000 front 2024-03-12T04:26:24.989056+0000 (oldest deadline 2024-03-12T04:26:49.089151+0000) 2024-03-12T04:27:07.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5[47670]: 2024-03-12T04:27:07.103+0000 7fe9cd1dc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:32.590512+0000 front 2024-03-12T04:26:32.590525+0000 (oldest deadline 2024-03-12T04:26:57.890548+0000) 2024-03-12T04:27:07.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:07.247+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:07.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:07.247+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:07.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:07.209+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:07.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:07.209+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:07.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:07.209+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:07.945 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:07.542+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:07.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:07 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:07.542+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:07.957 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:07.632+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:07.957 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:07.632+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:07.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:07.722+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:07.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:07.722+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:07.958 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:07 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:07.722+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:08.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:08.012+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:08.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:08.012+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:08.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:08.012+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:08.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:08.247+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:08.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:08.247+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:08.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:08.247+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:08.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:08.296+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:08.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:08.296+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:08.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:08 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:08.296+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:08.859 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:08.605+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:08.859 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:08.605+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:08.860 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:08 smithi136 podman[72070]: 2024-03-12 04:27:08.537415809 +0000 UTC m=+5.565630861 container stop 93e0d787ef974fc9c1a661c8cc4cad3b639bf43d4e61efbcd1d8a5b20b0e74de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:27:08.860 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:08 smithi136 podman[72070]: 2024-03-12 04:27:08.563009884 +0000 UTC m=+5.591224930 container died 93e0d787ef974fc9c1a661c8cc4cad3b639bf43d4e61efbcd1d8a5b20b0e74de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:27:08.860 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:08.715+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:08.860 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:08.715+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:08.860 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:08 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:08.715+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:08.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:08.591+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:08.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:08 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:08.591+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:09.187 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:08 smithi136 podman[72070]: 2024-03-12 04:27:08.97561853 +0000 UTC m=+6.003833572 container cleanup 93e0d787ef974fc9c1a661c8cc4cad3b639bf43d4e61efbcd1d8a5b20b0e74de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-12T04:27:09.187 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:08 smithi136 bash[72070]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5 2024-03-12T04:27:09.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:09.000+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:09.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:09.000+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:09.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:09.000+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:09.457 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:09 smithi136 podman[72089]: 2024-03-12 04:27:09.185443309 +0000 UTC m=+0.642181722 container remove 93e0d787ef974fc9c1a661c8cc4cad3b639bf43d4e61efbcd1d8a5b20b0e74de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-12T04:27:09.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:09.275+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:09.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:09.275+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:09.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:09.275+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:09.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:09.345+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:09.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:09.345+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:09.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:09 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:09.345+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:09.842 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:09 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:09.625+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:09.842 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:09 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:09.625+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:09.843 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:09 smithi136 podman[72181]: 2024-03-12 04:27:09.569607163 +0000 UTC m=+0.103990908 container create de37e3b73689da578d6e903eefb31f1c141ef3c3a63574ba157147704a994b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-12T04:27:09.843 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:09 smithi136 podman[72181]: 2024-03-12 04:27:09.483370984 +0000 UTC m=+0.017754670 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:09.843 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:09 smithi136 podman[72181]: 2024-03-12 04:27:09.840695386 +0000 UTC m=+0.375079072 container init de37e3b73689da578d6e903eefb31f1c141ef3c3a63574ba157147704a994b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-12T04:27:09.843 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:09 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:09.754+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:09.843 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:09 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:09.754+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:09.843 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:09 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:09.754+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:09.945 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:09.550+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:09.945 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:09.550+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:10.134 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:09 smithi136 podman[72181]: 2024-03-12 04:27:09.844051715 +0000 UTC m=+0.378435404 container start de37e3b73689da578d6e903eefb31f1c141ef3c3a63574ba157147704a994b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-03-12T04:27:10.134 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:09 smithi136 podman[72181]: 2024-03-12 04:27:09.932426061 +0000 UTC m=+0.466809760 container attach de37e3b73689da578d6e903eefb31f1c141ef3c3a63574ba157147704a994b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-12T04:27:10.404 INFO:journalctl@ceph.osd.5.smithi136.stdout:Mar 12 04:27:10 smithi136 podman[72181]: 2024-03-12 04:27:10.132772423 +0000 UTC m=+0.667156112 container died de37e3b73689da578d6e903eefb31f1c141ef3c3a63574ba157147704a994b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-12T04:27:10.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:09.951+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:10.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:09.951+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:10.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:09 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:09.951+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:10.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:10.229+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:10.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:10.229+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:10.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:10.229+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:10.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:10.330+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:10.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:10.330+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:10.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:10 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:10.330+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:10.707 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:10.630+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:10.707 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:10.630+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:10.785 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.5.service' 2024-03-12T04:27:10.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:10.865 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-12T04:27:10.866 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-12T04:27:10.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.6 2024-03-12T04:27:10.935 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:10.594+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:10.935 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:10.594+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:11.176 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:10 smithi042 systemd[1]: Stopping Ceph osd.6 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:11.195 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:10.933+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:11.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:10.933+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:11.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:10 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:10.933+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:11.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:10.778+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:11.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:10.778+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:11.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:10 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:10.778+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:11.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:11.290+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:11.290+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:11.290+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:11.173+0000 7f3633413700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:11.173+0000 7f3633413700 -1 osd.6 102 *** Got signal Terminated *** 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:11.173+0000 7f3633413700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:11.233+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:11.233+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:11.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:11 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:11.233+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:11.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:11.583+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:11.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:11.583+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:11.957 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:11.663+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:11.957 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:11.663+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:11.958 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:11.812+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:11.958 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:11.812+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:11.958 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:11 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:11.812+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:12.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:11.970+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:12.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:11.970+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:12.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:11 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:11.970+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:12.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:12.244+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:12.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:12.244+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:12.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:12.244+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:12.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:12.314+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:12.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:12.314+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:12.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:12.314+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:12.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:12 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:12.596+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:12.946 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:12 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:12.596+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:13.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:12 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:12.710+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:13.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:12 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:12.710+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:13.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:12 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:12.799+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:13.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:12 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:12.799+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:13.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:12 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:12.799+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:13.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:12 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:12.946+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:13.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:12 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:12.946+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:13.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:12 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:12.946+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:13.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:13.221+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:13.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:13.221+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:13.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:13.221+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:13.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:13.309+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:13.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:13.309+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:13.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:13 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:13.309+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:13.911 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:13 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:13.607+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:13.912 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:13 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:13.607+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:14.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:13 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:13.909+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:14.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:13 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:13.909+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:14.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:13 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:13.909+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:14.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:13 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:13.728+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:14.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:13 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:13.728+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:14.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:13 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:13.816+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:14.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:13 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:13.816+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:14.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:13 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:13.816+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:14.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:13 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:13.816+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:14.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:14.254+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:14.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:14.254+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:14.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:14.254+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:14.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:14.265+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:14.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:14.265+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:14.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:14 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:14.265+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:14.902 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:14 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:14.568+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:14.902 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:14 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:14.568+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:15.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:14 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:14.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:15.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:14 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:14.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:15.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:14 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:14.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:15.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:14 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:14.741+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:15.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:14 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:14.741+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:15.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:14 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:14.800+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:15.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:14 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:14.800+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:15.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:14 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:14.800+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:15.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:14 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:14.800+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:15.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:15.287+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:15.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:15.287+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:15.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:15.287+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:15.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:15.231+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:26.731293+0000 front 2024-03-12T04:26:26.731269+0000 (oldest deadline 2024-03-12T04:26:49.631379+0000) 2024-03-12T04:27:15.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:15.231+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.232947+0000 front 2024-03-12T04:26:34.233052+0000 (oldest deadline 2024-03-12T04:26:57.732720+0000) 2024-03-12T04:27:15.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:15.231+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.535096+0000 front 2024-03-12T04:26:41.535282+0000 (oldest deadline 2024-03-12T04:27:05.035266+0000) 2024-03-12T04:27:15.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:15 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6[58066]: 2024-03-12T04:27:15.231+0000 7f362b9ff700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:46.136209+0000 front 2024-03-12T04:26:46.136270+0000 (oldest deadline 2024-03-12T04:27:14.936992+0000) 2024-03-12T04:27:15.896 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:15.523+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:15.896 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:15.523+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:15.896 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:15.523+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:16.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:15.894+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:16.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:15.894+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:16.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:15 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:15.894+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:16.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:15 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:15.749+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:16.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:15 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:15.749+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:16.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:15 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:15.749+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:16.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:15 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:15.796+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:16.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:15 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:15.796+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:16.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:15 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:15.796+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:16.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:15 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:15.796+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:16.560 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:16 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:16.291+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:16.560 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:16 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:16.291+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:16.560 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:16 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:16.291+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:16.560 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:16 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:16.291+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:16.811 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:16 smithi042 podman[99873]: 2024-03-12 04:27:16.558461494 +0000 UTC m=+5.500667559 container died c20940336b3274ad14e61222eb27ed64ceffb1f4159b4a0a332bb94e3b1b3205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-12T04:27:16.925 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:16.564+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:16.925 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:16.564+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:16.926 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:16.564+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:17.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:16.923+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:17.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:16.923+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:17.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:16.923+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:17.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:16 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:16.778+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:17.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:16 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:16.779+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:17.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:16 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:16.779+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:17.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:16 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:16.840+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:17.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:16 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:16.840+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:17.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:16 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:16.840+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:17.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:16 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:16.840+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:17.566 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:17 smithi042 podman[99873]: 2024-03-12 04:27:17.082412785 +0000 UTC m=+6.024618860 container cleanup c20940336b3274ad14e61222eb27ed64ceffb1f4159b4a0a332bb94e3b1b3205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True) 2024-03-12T04:27:17.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:17 smithi042 bash[99873]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6 2024-03-12T04:27:17.567 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:17 smithi042 podman[99890]: 2024-03-12 04:27:17.30052973 +0000 UTC m=+0.737889390 container remove c20940336b3274ad14e61222eb27ed64ceffb1f4159b4a0a332bb94e3b1b3205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:27:17.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:17 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:17.263+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:17.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:17 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:17.263+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:17.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:17 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:17.263+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:17.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:17 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:17.263+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:17.889 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:17 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:17.543+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:17.890 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:17 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:17.543+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:17.890 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:17 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:17.543+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:17.914 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:17 smithi042 podman[99989]: 2024-03-12 04:27:17.548346844 +0000 UTC m=+0.018408303 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:17.914 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:17 smithi042 podman[99989]: 2024-03-12 04:27:17.668322236 +0000 UTC m=+0.138383688 container create 495fcd5cbf0813d02e9de9ad05b9f2e40e073c4d615d9f2a352d4dc2b20f9699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-12T04:27:17.915 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:17 smithi042 podman[99989]: 2024-03-12 04:27:17.912768742 +0000 UTC m=+0.382830207 container init 495fcd5cbf0813d02e9de9ad05b9f2e40e073c4d615d9f2a352d4dc2b20f9699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:27:18.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:17 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:17.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:18.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:17 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:17.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:18.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:17 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:17.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:18.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:17 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:17.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:18.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:17.746+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:18.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:17.746+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:18.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:17.746+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:18.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:17.844+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:18.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:17.844+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:18.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:17.844+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:18.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:17 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:17.844+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:18.222 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:17 smithi042 podman[99989]: 2024-03-12 04:27:17.916616245 +0000 UTC m=+0.386677696 container start 495fcd5cbf0813d02e9de9ad05b9f2e40e073c4d615d9f2a352d4dc2b20f9699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:27:18.222 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:18 smithi042 podman[99989]: 2024-03-12 04:27:18.021269794 +0000 UTC m=+0.491331263 container attach 495fcd5cbf0813d02e9de9ad05b9f2e40e073c4d615d9f2a352d4dc2b20f9699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:27:18.517 INFO:journalctl@ceph.osd.6.smithi042.stdout:Mar 12 04:27:18 smithi042 podman[99989]: 2024-03-12 04:27:18.221085737 +0000 UTC m=+0.691147191 container died 495fcd5cbf0813d02e9de9ad05b9f2e40e073c4d615d9f2a352d4dc2b20f9699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-03-12T04:27:18.518 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:18 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:18.277+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:18.518 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:18 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:18.277+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:18.518 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:18 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:18.277+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:18.518 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:18 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:18.277+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:18.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.6.service' 2024-03-12T04:27:18.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:18.878 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-12T04:27:18.878 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-12T04:27:18.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.7 2024-03-12T04:27:18.888 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:18 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:18.580+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:18.889 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:18 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:18.580+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:18.889 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:18 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:18.580+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:19.189 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:18 smithi119 systemd[1]: Stopping Ceph osd.7 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:19.189 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:18 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:18.886+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:19.189 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:18 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:18.886+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:19.189 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:18 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:18.886+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:19.189 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:18 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:18.886+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:19.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:18 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:18.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:19.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:18 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:18.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:19.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:18 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:18.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:19.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:18 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:18.858+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:19.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:18 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:18.858+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:19.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:18 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:18.858+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:19.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:18 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:18.858+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:19.446 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:19.186+0000 7fed5db20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:19.446 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:19.186+0000 7fed5db20700 -1 osd.7 102 *** Got signal Terminated *** 2024-03-12T04:27:19.446 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:19.186+0000 7fed5db20700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:27:19.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:19.234+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:19.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:19.234+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:19.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:19.234+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:19.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:19 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:19.234+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:19.931 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:19.563+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:19.931 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:19.563+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:19.931 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:19.563+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:20.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:19.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:20.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:19.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:20.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:19.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:20.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:19 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:19.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:20.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:19 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:19.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:20.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:19 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:19.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:20.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:19 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:19.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:20.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:19 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:19.841+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:20.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:19 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:19.841+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:20.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:19 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:19.841+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:20.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:19 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:19.841+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:20.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:20 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:20.232+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:20.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:20 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:20.232+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:20.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:20 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:20.232+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:20.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:20 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:20.232+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:20.945 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:20.563+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:20.945 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:20.563+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:20.945 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:20.563+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:21.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:20.942+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:21.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:20.942+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:21.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:20.942+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:21.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:20.942+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:21.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:20.942+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:21.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:20 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:20.810+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:21.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:20 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:20.810+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:21.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:20 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:20.810+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:21.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:20 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:20.884+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:21.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:20 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:20.884+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:21.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:20 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:20.884+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:21.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:20 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:20.884+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:21.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:21 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:21.186+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:21.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:21 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:21.186+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:21.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:21 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:21.186+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:21.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:21 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:21.186+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:21.929 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:21.537+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:21.929 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:21.537+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:21.930 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:21.537+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:22.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:21.926+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:22.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:21.926+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:22.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:21.926+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:22.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:21.926+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:22.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:21 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:21.926+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:22.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:21 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:21.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:22.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:21 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:21.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:22.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:21 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:21.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:22.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:21 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:21.906+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:22.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:21 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:21.906+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:22.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:21 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:21.906+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:22.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:21 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:21.906+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:22.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:22.183+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:22.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:22.183+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:22.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:22.183+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:22.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:22.183+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:22.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:22 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:22.183+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:22.930 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:22.558+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:22.930 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:22.558+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:22.930 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:22.558+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:22.930 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:22.558+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.807481+0000 front 2024-03-12T04:26:58.807430+0000 (oldest deadline 2024-03-12T04:27:22.307543+0000) 2024-03-12T04:27:23.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:22.927+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:23.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:22.927+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:23.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:22.927+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:23.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:22.927+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:23.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:22 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:22.927+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:23.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:22 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:22.783+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:23.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:22 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:22.783+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:23.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:22 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:22.783+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:23.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:22 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:22.864+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:23.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:22 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:22.864+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:23.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:22 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:22.864+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:23.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:22 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:22.864+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:23.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:23.148+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:23.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:23.148+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:23.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:23.148+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:23.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:23.148+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:23.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:23 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:23.148+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:23.932 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:23.529+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.203159+0000 front 2024-03-12T04:26:33.203060+0000 (oldest deadline 2024-03-12T04:26:57.303142+0000) 2024-03-12T04:27:23.932 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:23.529+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.304848+0000 front 2024-03-12T04:26:41.304859+0000 (oldest deadline 2024-03-12T04:27:04.804825+0000) 2024-03-12T04:27:23.932 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:23.529+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:49.406486+0000 front 2024-03-12T04:26:49.406517+0000 (oldest deadline 2024-03-12T04:27:15.306538+0000) 2024-03-12T04:27:23.932 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7[54335]: 2024-03-12T04:27:23.529+0000 7fed5610c700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.807481+0000 front 2024-03-12T04:26:58.807430+0000 (oldest deadline 2024-03-12T04:27:22.307543+0000) 2024-03-12T04:27:24.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:23.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:24.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:23.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:24.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:23.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:24.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:23.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:24.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:23 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:23.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:24.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:23.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:24.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:23.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:24.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:23.762+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:24.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:23.847+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:24.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:23.847+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:24.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:23.847+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:24.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:23 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:23.847+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:24.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:24 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:24.102+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:24.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:24 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:24.102+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:24.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:24 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:24.102+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:24.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:24 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:24.102+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:24.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:24 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:24.102+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:24.922 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:24 smithi119 podman[72863]: 2024-03-12 04:27:24.618099127 +0000 UTC m=+5.546289529 container died e6db9a800223fd252da06ca3d70c72fa7f63a1c2fbab82af6d7f409d81c61d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-12T04:27:25.196 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:25 smithi119 podman[72863]: 2024-03-12 04:27:25.028026147 +0000 UTC m=+5.956216544 container cleanup e6db9a800223fd252da06ca3d70c72fa7f63a1c2fbab82af6d7f409d81c61d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-12T04:27:25.196 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:25 smithi119 bash[72863]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7 2024-03-12T04:27:25.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:24 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:24.972+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:25.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:24 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:24.972+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:25.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:24 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:24.972+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:25.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:24 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:24.972+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:25.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:24 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:24.972+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:25.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:24.807+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:25.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:24.807+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:25.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:24.807+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:25.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:24.807+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:25.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:24.834+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:25.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:24.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:25.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:24.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:25.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:24.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:25.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:24 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:24.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:25.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:25 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:25.128+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:25.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:25 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:25.128+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:25.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:25 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:25.128+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:25.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:25 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:25.128+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:25.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:25 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:25.128+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:25.640 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:25 smithi119 podman[72880]: 2024-03-12 04:27:25.221178523 +0000 UTC m=+0.605622018 container remove e6db9a800223fd252da06ca3d70c72fa7f63a1c2fbab82af6d7f409d81c61d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397) 2024-03-12T04:27:25.926 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:25 smithi119 podman[72971]: 2024-03-12 04:27:25.542039645 +0000 UTC m=+0.017800906 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:25.926 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:25 smithi119 podman[72971]: 2024-03-12 04:27:25.663685629 +0000 UTC m=+0.139446921 container create cb8301ec96808939e152588fedcab114d23174f985d77abe9158ec5313f60cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-12T04:27:26.196 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:25 smithi119 podman[72971]: 2024-03-12 04:27:25.924877008 +0000 UTC m=+0.400638258 container init cb8301ec96808939e152588fedcab114d23174f985d77abe9158ec5313f60cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-12T04:27:26.196 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:25 smithi119 podman[72971]: 2024-03-12 04:27:25.927984832 +0000 UTC m=+0.403746089 container start cb8301ec96808939e152588fedcab114d23174f985d77abe9158ec5313f60cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-12T04:27:26.196 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:26 smithi119 podman[72971]: 2024-03-12 04:27:26.008298581 +0000 UTC m=+0.484059844 container attach cb8301ec96808939e152588fedcab114d23174f985d77abe9158ec5313f60cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-12T04:27:26.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:25 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:25.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:26.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:25 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:25.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:26.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:25 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:25.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:26.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:25 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:25.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:26.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:25 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:25.929+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:26.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:25.788+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:26.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:25.788+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:26.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:25.788+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:26.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:25.788+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:26.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:25.839+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:26.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:25.839+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:26.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:25.839+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:26.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:25.839+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:26.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:25 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:25.839+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:26.480 INFO:journalctl@ceph.osd.7.smithi119.stdout:Mar 12 04:27:26 smithi119 podman[72971]: 2024-03-12 04:27:26.212372741 +0000 UTC m=+0.688133982 container died cb8301ec96808939e152588fedcab114d23174f985d77abe9158ec5313f60cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-12T04:27:26.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:26 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:26.114+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:26.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:26 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:26.114+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:26.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:26 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:26.114+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:26.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:26 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:26.114+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:26.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:26 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:26.114+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:26.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.7.service' 2024-03-12T04:27:26.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:26.921 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-12T04:27:26.921 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-12T04:27:26.921 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.8 2024-03-12T04:27:27.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:26 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:26.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:27.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:26 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:26.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:27.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:26 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:26.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:27.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:26 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:26.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:27.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:26 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:26.887+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:27.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:26.823+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:27.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:26.823+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:26.823+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:26.823+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:26 smithi136 systemd[1]: Stopping Ceph osd.8 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:26.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:26.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:26.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:26.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:27.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:26 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:26.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:27.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:27 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:27.117+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:27.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:27 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:27.117+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:27.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:27 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:27.117+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:27.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:27 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:27.117+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:27.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:27 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:27.117+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:27.707 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:27.236+0000 7f93d399f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:27.707 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:27.236+0000 7f93d399f700 -1 osd.8 102 *** Got signal Terminated *** 2024-03-12T04:27:27.707 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:27.236+0000 7f93d399f700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:27:28.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:27 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:27.903+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:28.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:27 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:27.903+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:28.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:27 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:27.903+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:28.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:27 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:27.903+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:28.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:27 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:27.903+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:28.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:27.868+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:28.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:27.868+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:28.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:27.868+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:28.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:27.868+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:28.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:27.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:28.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:27.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:28.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:27.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:28.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:27.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:28.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:27 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:27.835+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:28.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:28 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:28.124+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:28.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:28 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:28.124+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:28.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:28 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:28.124+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:28.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:28 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:28.124+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:28.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:28 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:28.124+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:29.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:28.854+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:29.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:28.854+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:29.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:28.854+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:29.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:28.854+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:29.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:28.854+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:29.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:28.869+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:29.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:28.869+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:29.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:28.869+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:29.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:28.869+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:29.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:28.815+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:29.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:28.815+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:29.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:28.815+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:29.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:28.815+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:29.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:28 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:28.815+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:29.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:29 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:29.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:29.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:29 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:29.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:29.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:29 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:29.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:29.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:29 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:29.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:29.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:29 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:29.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:30.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:29 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:29.895+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:30.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:29 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:29.895+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:30.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:29 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:29.895+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:30.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:29 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:29.895+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:30.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:29 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:29.895+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:30.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:29.862+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:30.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:29.862+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:30.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:29.862+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:30.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:29.862+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:30.208 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:29.862+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:06.564700+0000 front 2024-03-12T04:27:06.564587+0000 (oldest deadline 2024-03-12T04:27:29.464719+0000) 2024-03-12T04:27:30.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:29.851+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:30.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:29.851+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:30.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:29.851+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:30.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:29.851+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:30.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:29.851+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:30.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:29 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:29.851+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:30.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:30 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:30.120+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:30.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:30 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:30.120+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:30.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:30 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:30.120+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:30.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:30 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:30.120+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:30.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:30 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:30.120+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:30.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:30 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:30.120+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:31.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:30.891+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:31.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:30.891+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:31.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:30.891+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:31.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:30.891+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:31.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:30 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:30.891+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:31.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:30.892+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:31.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:30.892+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:31.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:30.892+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:30.892+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:30.892+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:06.564700+0000 front 2024-03-12T04:27:06.564587+0000 (oldest deadline 2024-03-12T04:27:29.464719+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:30.852+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:30.852+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:30.852+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:30.852+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:30.852+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:31.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:30 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:30.852+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:31.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:31 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:31.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:31.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:31 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:31.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:31.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:31 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:31.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:31.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:31 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:31.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:31.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:31 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:31.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:31.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:31 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:31.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:32.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:31 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:31.928+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:32.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:31 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:31.928+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:32.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:31 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:31.928+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:32.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:31 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:31.928+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:32.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:31 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:31.928+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:32.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:31 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:31.928+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:32.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:31.933+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.560358+0000 front 2024-03-12T04:26:35.560234+0000 (oldest deadline 2024-03-12T04:27:00.260256+0000) 2024-03-12T04:27:32.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:31.933+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:41.361112+0000 front 2024-03-12T04:26:41.361076+0000 (oldest deadline 2024-03-12T04:27:07.261278+0000) 2024-03-12T04:27:32.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:31.933+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.362470+0000 front 2024-03-12T04:26:51.362452+0000 (oldest deadline 2024-03-12T04:27:14.862349+0000) 2024-03-12T04:27:32.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:31.933+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.963381+0000 front 2024-03-12T04:26:58.963558+0000 (oldest deadline 2024-03-12T04:27:24.263367+0000) 2024-03-12T04:27:32.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8[54189]: 2024-03-12T04:27:31.933+0000 7f93cbf8b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:06.564700+0000 front 2024-03-12T04:27:06.564587+0000 (oldest deadline 2024-03-12T04:27:29.464719+0000) 2024-03-12T04:27:32.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:31.893+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:32.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:31.893+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:32.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:31.893+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:32.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:31.893+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:32.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:31.893+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:32.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:31 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:31.893+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:32.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:32 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:32.141+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:32.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:32 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:32.141+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:32.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:32 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:32.141+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:32.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:32 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:32.141+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:32.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:32 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:32.141+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:32.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:32 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:32.141+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:32.907 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:32 smithi136 podman[72298]: 2024-03-12 04:27:32.625304191 +0000 UTC m=+5.508703122 container died 1d38b98e2522c44a2db0acaa55b0f63984b378f3c0028c0112fa091b2a286625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-12T04:27:32.907 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:32.861+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:32.907 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:32.861+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:32.907 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:32.861+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:32.907 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:32.861+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:32.907 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:32.861+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:32.907 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:32.861+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:33.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 podman[72298]: 2024-03-12 04:27:33.012948756 +0000 UTC m=+5.896347693 container cleanup 1d38b98e2522c44a2db0acaa55b0f63984b378f3c0028c0112fa091b2a286625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-12T04:27:33.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 bash[72298]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8 2024-03-12T04:27:33.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:32 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:32.948+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:33.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:32 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:32.948+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:33.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:32 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:32.948+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:33.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:32 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:32.948+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:33.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:32 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:32.948+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:33.447 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:32 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:32.948+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:33.557 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 podman[72315]: 2024-03-12 04:27:33.24719358 +0000 UTC m=+0.618668680 container remove 1d38b98e2522c44a2db0acaa55b0f63984b378f3c0028c0112fa091b2a286625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-12T04:27:33.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:33.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:33.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:33.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:33.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:33.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:33.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:33.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:33.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:33.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:33.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:33 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:33.133+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:33.894 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 podman[72406]: 2024-03-12 04:27:33.458403404 +0000 UTC m=+0.018010329 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:33.895 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 podman[72406]: 2024-03-12 04:27:33.556181467 +0000 UTC m=+0.115788384 container create 37eb6d393674f9a093a412b965ea210669c2faf1e7a4186c2094527a11403970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS) 2024-03-12T04:27:33.895 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 podman[72406]: 2024-03-12 04:27:33.792135455 +0000 UTC m=+0.351742372 container init 37eb6d393674f9a093a412b965ea210669c2faf1e7a4186c2094527a11403970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-12T04:27:33.895 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 podman[72406]: 2024-03-12 04:27:33.795225413 +0000 UTC m=+0.354832324 container start 37eb6d393674f9a093a412b965ea210669c2faf1e7a4186c2094527a11403970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-12T04:27:33.895 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:33 smithi136 podman[72406]: 2024-03-12 04:27:33.892358723 +0000 UTC m=+0.451965673 container attach 37eb6d393674f9a093a412b965ea210669c2faf1e7a4186c2094527a11403970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-12T04:27:34.207 INFO:journalctl@ceph.osd.8.smithi136.stdout:Mar 12 04:27:34 smithi136 podman[72406]: 2024-03-12 04:27:34.090041705 +0000 UTC m=+0.649648614 container died 37eb6d393674f9a093a412b965ea210669c2faf1e7a4186c2094527a11403970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-12T04:27:34.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:33 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:33.907+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:34.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:33 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:33.907+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:34.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:33 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:33.907+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:34.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:33 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:33.907+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:34.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:33 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:33.907+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:34.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:33 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:33.907+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:34.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:33.954+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:34.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:33.954+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:34.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:33.954+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:34.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:33.954+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:34.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:33.954+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:34.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:33 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:33.954+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:34.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:34 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:34.136+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:34.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:34 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:34.136+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:34.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:34 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:34.136+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:34.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:34 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:34.136+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:34.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:34 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:34.136+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:34.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:34 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:34.136+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:34.773 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.8.service' 2024-03-12T04:27:34.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:34.814 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-12T04:27:34.814 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-12T04:27:34.814 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.9 2024-03-12T04:27:35.093 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:34 smithi042 systemd[1]: Stopping Ceph osd.9 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:35.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:34 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:34.880+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:35.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:34 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:34.880+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:35.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:34 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:34.880+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:35.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:34 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:34.880+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:35.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:34 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:34.880+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:35.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:34 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:34.880+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:35.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:34 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:34.953+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:35.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:34 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:34.953+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:35.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:34 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:34.953+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:35.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:34 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:34.953+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:35.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:34 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:34.953+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:35.446 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:34 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:34.953+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:35.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.091+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.091+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.091+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.091+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.091+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.091+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.118+0000 7fef113a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.118+0000 7fef113a8700 -1 osd.9 102 *** Got signal Terminated *** 2024-03-12T04:27:35.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:35 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:35.118+0000 7fef113a8700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:27:36.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:35 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:35.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:36.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:35 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:35.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:36.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:35 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:35.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:36.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:35 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:35.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:36.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:35 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:35.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:36.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:35 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:35.843+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:36.347 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:35 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:35.969+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:36.347 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:35 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:35.969+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:36.348 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:35 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:35.969+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:36.348 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:35 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:35.969+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:36.348 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:35 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:35.969+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:36.348 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:35 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:35.969+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:36.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:36 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:36.122+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:36.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:36 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:36.122+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:36.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:36 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:36.122+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:36.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:36 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:36.122+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:36.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:36 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:36.122+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:36.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:36 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:36.122+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:37.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:36 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:36.963+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:37.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:36 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:36.963+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:37.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:36 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:36.963+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:37.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:36 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:36.963+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:37.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:36 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:36.963+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:37.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:36 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:36.963+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:37.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:36.865+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:37.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:36.865+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:37.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:36.865+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:37.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:36.865+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:37.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:36.865+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:37.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:36.865+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:37.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:37.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:37.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:37.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:37.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:37.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:37.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:37.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:37.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:37.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:37.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:37 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:37.100+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:38.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:37 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:37.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:38.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:37 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:37.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:38.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:37 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:37.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:38.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:37 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:37.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:38.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:37 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:37.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:38.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:37 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:37.930+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:38.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:37 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:37.902+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:38.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:37 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:37.902+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:38.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:37 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:37.902+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:38.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:37 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:37.902+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:38.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:37 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:37.902+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:38.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:37 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:37.902+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:38.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:38.109+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:38.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:38.109+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:38.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:38.109+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:38.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:38.109+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:38.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:38.109+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:38.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:38 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:38.109+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:39.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:38 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:38.920+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:39.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:38 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:38.920+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:39.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:38 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:38.920+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:39.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:38 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:38.920+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:39.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:38 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:38.920+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:39.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:38 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:38.920+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:39.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:38.882+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:39.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:38.882+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:39.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:38.882+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:39.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:38.882+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:39.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:38.882+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:39.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:38 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:38.882+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:39.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:39.145+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.398735+0000 front 2024-03-12T04:26:27.398830+0000 (oldest deadline 2024-03-12T04:26:50.298918+0000) 2024-03-12T04:27:39.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:39.145+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:34.400100+0000 front 2024-03-12T04:26:34.399967+0000 (oldest deadline 2024-03-12T04:26:57.300026+0000) 2024-03-12T04:27:39.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:39.145+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:43.602399+0000 front 2024-03-12T04:26:43.602233+0000 (oldest deadline 2024-03-12T04:27:07.702239+0000) 2024-03-12T04:27:39.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:39.145+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.203133+0000 front 2024-03-12T04:26:51.203186+0000 (oldest deadline 2024-03-12T04:27:15.903194+0000) 2024-03-12T04:27:39.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:39.145+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:58.204220+0000 front 2024-03-12T04:26:58.204389+0000 (oldest deadline 2024-03-12T04:27:21.704296+0000) 2024-03-12T04:27:39.566 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:39.145+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:07.605245+0000 front 2024-03-12T04:27:07.605274+0000 (oldest deadline 2024-03-12T04:27:29.305134+0000) 2024-03-12T04:27:39.567 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:39 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9[64659]: 2024-03-12T04:27:39.145+0000 7fef0d99c700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.007139+0000 front 2024-03-12T04:27:15.007194+0000 (oldest deadline 2024-03-12T04:27:38.507253+0000) 2024-03-12T04:27:40.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:39 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:39.871+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:40.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:39 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:39.871+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:40.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:39 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:39.871+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:40.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:39 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:39.871+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:40.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:39 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:39.871+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:40.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:39 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:39.871+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:40.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:39 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:39.871+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:40.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:39 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:39.849+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:40.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:39 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:39.849+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:40.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:39 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:39.849+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:40.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:39 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:39.849+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:40.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:39 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:39.849+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:40.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:39 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:39.849+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:40.813 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:40 smithi042 podman[100095]: 2024-03-12 04:27:40.520405796 +0000 UTC m=+5.511243652 container died b5ad832cafb1e84652fef0edfa52b3e08ce22a1b83f94aca8d6dfae43110a8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:27:41.066 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:40 smithi042 podman[100095]: 2024-03-12 04:27:40.910808226 +0000 UTC m=+5.901646089 container cleanup b5ad832cafb1e84652fef0edfa52b3e08ce22a1b83f94aca8d6dfae43110a8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-12T04:27:41.066 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:40 smithi042 bash[100095]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9 2024-03-12T04:27:41.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:40 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:40.885+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:41.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:40 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:40.885+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:41.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:40 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:40.885+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:41.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:40 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:40.885+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:41.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:40 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:40.885+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:41.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:40 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:40.885+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:41.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:40 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:40.885+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:41.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:40.881+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:41.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:40.881+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:41.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:40.881+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:41.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:40.881+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:41.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:40.881+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:41.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:40.881+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:41.481 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:41 smithi042 podman[100112]: 2024-03-12 04:27:41.153702656 +0000 UTC m=+0.625601516 container remove b5ad832cafb1e84652fef0edfa52b3e08ce22a1b83f94aca8d6dfae43110a8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-12T04:27:41.795 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:41 smithi042 podman[100204]: 2024-03-12 04:27:41.479494275 +0000 UTC m=+0.113394351 container create d9884f76458e3575fa40351d959d452d9f21bdfa77c34f7da8ce4eb44cf30789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-12T04:27:41.795 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:41 smithi042 podman[100204]: 2024-03-12 04:27:41.383655576 +0000 UTC m=+0.017555665 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:41.795 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:41 smithi042 podman[100204]: 2024-03-12 04:27:41.706792637 +0000 UTC m=+0.340692727 container init d9884f76458e3575fa40351d959d452d9f21bdfa77c34f7da8ce4eb44cf30789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-12T04:27:41.796 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:41 smithi042 podman[100204]: 2024-03-12 04:27:41.709762495 +0000 UTC m=+0.343662575 container start d9884f76458e3575fa40351d959d452d9f21bdfa77c34f7da8ce4eb44cf30789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-12T04:27:41.796 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:41 smithi042 podman[100204]: 2024-03-12 04:27:41.793501616 +0000 UTC m=+0.427401707 container attach d9884f76458e3575fa40351d959d452d9f21bdfa77c34f7da8ce4eb44cf30789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-12T04:27:42.066 INFO:journalctl@ceph.osd.9.smithi042.stdout:Mar 12 04:27:42 smithi042 podman[100204]: 2024-03-12 04:27:42.006345842 +0000 UTC m=+0.640245919 container died d9884f76458e3575fa40351d959d452d9f21bdfa77c34f7da8ce4eb44cf30789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-12T04:27:42.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:41 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:41.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:42.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:41 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:41.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:42.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:41 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:41.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:42.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:41 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:41.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:42.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:41 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:41.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:42.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:41 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:41.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:42.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:41 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:41.900+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:42.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:41 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:41.869+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:42.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:41 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:41.869+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:42.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:41 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:41.869+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:42.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:41 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:41.869+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:42.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:41 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:41.869+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:42.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:41 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:41.869+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:42.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.9.service' 2024-03-12T04:27:42.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:42.623 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-12T04:27:42.623 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-12T04:27:42.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.10 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 systemd[1]: Stopping Ceph osd.10 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.863+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.863+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.863+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.863+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.863+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.863+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:42.916 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.863+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:43.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.913+0000 7f0b5a500700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:43.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.913+0000 7f0b5a500700 -1 osd.10 102 *** Got signal Terminated *** 2024-03-12T04:27:43.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:42 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:42.913+0000 7f0b5a500700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:27:43.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:42 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:42.910+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:43.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:42 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:42.910+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:43.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:42 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:42.910+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:43.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:42 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:42.910+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:43.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:42 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:42.910+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:43.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:42 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:42.910+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:44.198 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:43.855+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:44.198 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:43.855+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:44.199 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:43.855+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:44.199 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:43.855+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:44.199 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:43.855+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:44.199 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:43.855+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:44.199 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:43 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:43.855+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:44.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:43 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:43.923+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:44.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:43 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:43.923+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:44.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:43 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:43.923+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:44.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:43 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:43.923+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:44.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:43 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:43.923+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:44.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:43 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:43.923+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:45.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:44 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:44.828+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:45.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:44 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:44.828+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:45.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:44 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:44.828+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:45.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:44 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:44.828+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:45.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:44 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:44.828+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:45.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:44 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:44.828+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:45.197 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:44 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:44.828+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:45.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:44.939+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:45.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:44.939+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:45.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:44.939+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:45.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:44.939+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:45.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:44.939+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:45.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:44.939+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:46.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:45 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:45.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:46.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:45 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:45.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:46.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:45 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:45.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:46.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:45 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:45.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:46.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:45 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:45.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:46.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:45 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:45.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:46.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:45 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:45.846+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:46.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:45 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:45.962+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:46.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:45 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:45.962+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:46.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:45 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:45.962+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:46.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:45 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:45.962+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:46.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:45 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:45.962+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:46.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:45 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:45.962+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:47.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:46 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:46.852+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:47.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:46 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:46.852+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:47.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:46 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:46.852+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:47.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:46 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:46.852+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:47.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:46 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:46.852+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:47.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:46 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:46.852+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:47.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:46 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:46.852+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:47.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:46 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:46.992+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:47.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:46 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:46.992+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:47.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:46 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:46.992+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:47.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:46 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:46.992+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:47.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:46 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:46.992+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:47.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:46 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:46.992+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:47.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:46 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:46.992+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:48.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:47 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:47.829+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:27.690899+0000 front 2024-03-12T04:26:27.690768+0000 (oldest deadline 2024-03-12T04:26:50.590673+0000) 2024-03-12T04:27:48.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:47 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:47.829+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:35.192252+0000 front 2024-03-12T04:26:35.192306+0000 (oldest deadline 2024-03-12T04:27:02.192931+0000) 2024-03-12T04:27:48.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:47 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:47.829+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:42.193514+0000 front 2024-03-12T04:26:42.193609+0000 (oldest deadline 2024-03-12T04:27:05.693592+0000) 2024-03-12T04:27:48.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:47 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:47.829+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:51.594450+0000 front 2024-03-12T04:26:51.594475+0000 (oldest deadline 2024-03-12T04:27:17.494392+0000) 2024-03-12T04:27:48.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:47 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:47.829+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:57.995405+0000 front 2024-03-12T04:26:57.995431+0000 (oldest deadline 2024-03-12T04:27:20.895519+0000) 2024-03-12T04:27:48.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:47 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:47.829+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:03.797184+0000 front 2024-03-12T04:27:03.797153+0000 (oldest deadline 2024-03-12T04:27:31.397263+0000) 2024-03-12T04:27:48.196 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:47 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10[61101]: 2024-03-12T04:27:47.829+0000 7f0b52aec700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-03-12T04:27:15.498526+0000 front 2024-03-12T04:27:15.498547+0000 (oldest deadline 2024-03-12T04:27:39.598255+0000) 2024-03-12T04:27:48.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:48.021+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:48.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:48.021+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:48.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:48.021+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:48.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:48.021+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:48.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:48.021+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:48.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:48.021+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:48.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:48 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:48.021+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:48.775 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:48 smithi119 podman[73090]: 2024-03-12 04:27:48.495197196 +0000 UTC m=+5.671863624 container died f33854c069cdcee4b70fa10cf81790d3b0c8d156841f93aea912a3fad3418530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-12T04:27:49.100 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:48 smithi119 podman[73090]: 2024-03-12 04:27:48.863399559 +0000 UTC m=+6.040065986 container cleanup f33854c069cdcee4b70fa10cf81790d3b0c8d156841f93aea912a3fad3418530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:27:49.100 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:48 smithi119 bash[73090]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10 2024-03-12T04:27:49.422 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:49 smithi119 podman[73108]: 2024-03-12 04:27:49.097812358 +0000 UTC m=+0.600081036 container remove f33854c069cdcee4b70fa10cf81790d3b0c8d156841f93aea912a3fad3418530 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-12T04:27:49.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:49.064+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:49.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:49.064+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:49.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:49.064+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:49.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:49.064+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:49.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:49.064+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:49.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:49.064+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:49.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:49 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:49.064+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:49.677 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:49 smithi119 podman[73198]: 2024-03-12 04:27:49.323449494 +0000 UTC m=+0.017556405 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:49.678 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:49 smithi119 podman[73198]: 2024-03-12 04:27:49.426165219 +0000 UTC m=+0.120272164 container create ee6dd0e7e6c2bad5b787cfbc5906ffdb99b68dd3f5ce60927f840005ccfc348a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:27:49.678 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:49 smithi119 podman[73198]: 2024-03-12 04:27:49.676173912 +0000 UTC m=+0.370280826 container init ee6dd0e7e6c2bad5b787cfbc5906ffdb99b68dd3f5ce60927f840005ccfc348a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:27:49.946 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:49 smithi119 podman[73198]: 2024-03-12 04:27:49.67923262 +0000 UTC m=+0.373339524 container start ee6dd0e7e6c2bad5b787cfbc5906ffdb99b68dd3f5ce60927f840005ccfc348a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:27:49.946 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:49 smithi119 podman[73198]: 2024-03-12 04:27:49.759418391 +0000 UTC m=+0.453525318 container attach ee6dd0e7e6c2bad5b787cfbc5906ffdb99b68dd3f5ce60927f840005ccfc348a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10-deactivate, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-12T04:27:50.255 INFO:journalctl@ceph.osd.10.smithi119.stdout:Mar 12 04:27:49 smithi119 podman[73198]: 2024-03-12 04:27:49.965693666 +0000 UTC m=+0.659800571 container died ee6dd0e7e6c2bad5b787cfbc5906ffdb99b68dd3f5ce60927f840005ccfc348a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:27:50.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.087+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:50.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.087+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:50.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.087+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:50.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.087+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:50.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.087+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:50.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.087+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:50.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.087+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:50.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.10.service' 2024-03-12T04:27:50.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:50.577 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-12T04:27:50.577 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-12T04:27:50.577 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.11 2024-03-12T04:27:50.899 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 systemd[1]: Stopping Ceph osd.11 for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:51.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.896+0000 7fb309a14700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:51.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.896+0000 7fb309a14700 -1 osd.11 102 *** Got signal Terminated *** 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:50 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:50.896+0000 7fb309a14700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:51.073+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:51.073+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:51.073+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:51.073+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:51.073+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:51.073+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:51.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:51 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:51.073+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:52.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:52.118+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:52.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:52.118+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:52.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:52.118+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:52.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:52.118+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:52.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:52.118+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:52.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:52.118+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:52.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:52 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:52.118+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:53.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:53.121+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:53.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:53.121+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:53.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:53.121+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:53.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:53.121+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:53.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:53.121+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:53.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:53.121+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:53.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:53 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:53.121+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:54.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:54.119+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:54.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:54.119+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:54.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:54.119+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:54.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:54.119+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:54.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:54.119+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:54.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:54.119+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:54.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:54 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:54.119+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:55.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:55.071+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-03-12T04:26:25.278137+0000 front 2024-03-12T04:26:25.278220+0000 (oldest deadline 2024-03-12T04:26:48.178100+0000) 2024-03-12T04:27:55.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:55.071+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-03-12T04:26:33.979841+0000 front 2024-03-12T04:26:33.980017+0000 (oldest deadline 2024-03-12T04:26:59.879724+0000) 2024-03-12T04:27:55.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:55.071+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-03-12T04:26:39.880008+0000 front 2024-03-12T04:26:39.880085+0000 (oldest deadline 2024-03-12T04:27:04.580219+0000) 2024-03-12T04:27:55.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:55.071+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-03-12T04:26:50.381602+0000 front 2024-03-12T04:26:50.381694+0000 (oldest deadline 2024-03-12T04:27:13.281693+0000) 2024-03-12T04:27:55.457 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:55.071+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-03-12T04:26:59.683615+0000 front 2024-03-12T04:26:59.683497+0000 (oldest deadline 2024-03-12T04:27:24.383526+0000) 2024-03-12T04:27:55.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:55.071+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-03-12T04:27:04.384114+0000 front 2024-03-12T04:27:04.384217+0000 (oldest deadline 2024-03-12T04:27:29.684287+0000) 2024-03-12T04:27:55.458 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:55 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11[60810]: 2024-03-12T04:27:55.071+0000 7fb302000700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-03-12T04:27:22.587131+0000 front 2024-03-12T04:27:22.587041+0000 (oldest deadline 2024-03-12T04:27:46.687016+0000) 2024-03-12T04:27:56.565 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:56 smithi136 podman[72523]: 2024-03-12 04:27:56.303306286 +0000 UTC m=+5.527560918 container died b6843c35c6ec546d9caf1bcc83388e330afbeb132b83d5537e87b674330ba304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397) 2024-03-12T04:27:56.882 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:56 smithi136 podman[72523]: 2024-03-12 04:27:56.646037856 +0000 UTC m=+5.870292494 container cleanup b6843c35c6ec546d9caf1bcc83388e330afbeb132b83d5537e87b674330ba304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-12T04:27:56.882 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:56 smithi136 bash[72523]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11 2024-03-12T04:27:57.207 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:56 smithi136 podman[72540]: 2024-03-12 04:27:56.880655375 +0000 UTC m=+0.576325503 container remove b6843c35c6ec546d9caf1bcc83388e330afbeb132b83d5537e87b674330ba304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-12T04:27:57.618 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:57 smithi136 podman[72628]: 2024-03-12 04:27:57.297963151 +0000 UTC m=+0.112124845 container create e4391a0563f3fec0831a09ea4859fdf45e766ffb0ca3a539fd09d3fca15515d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-12T04:27:57.618 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:57 smithi136 podman[72628]: 2024-03-12 04:27:57.204198504 +0000 UTC m=+0.018360207 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T04:27:57.619 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:57 smithi136 podman[72628]: 2024-03-12 04:27:57.533464391 +0000 UTC m=+0.347626086 container init e4391a0563f3fec0831a09ea4859fdf45e766ffb0ca3a539fd09d3fca15515d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:27:57.619 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:57 smithi136 podman[72628]: 2024-03-12 04:27:57.536553184 +0000 UTC m=+0.350714885 container start e4391a0563f3fec0831a09ea4859fdf45e766ffb0ca3a539fd09d3fca15515d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11-deactivate, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-12T04:27:57.619 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:57 smithi136 podman[72628]: 2024-03-12 04:27:57.616761154 +0000 UTC m=+0.430922863 container attach e4391a0563f3fec0831a09ea4859fdf45e766ffb0ca3a539fd09d3fca15515d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-12T04:27:57.957 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:57 smithi136 podman[72628]: 2024-03-12 04:27:57.827612258 +0000 UTC m=+0.641773954 container died e4391a0563f3fec0831a09ea4859fdf45e766ffb0ca3a539fd09d3fca15515d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:27:58.385 INFO:journalctl@ceph.osd.11.smithi136.stdout:Mar 12 04:27:58 smithi136 podman[72653]: 2024-03-12 04:27:58.369571143 +0000 UTC m=+0.532355256 container remove e4391a0563f3fec0831a09ea4859fdf45e766ffb0ca3a539fd09d3fca15515d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:27:58.391 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@osd.11.service' 2024-03-12T04:27:58.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:27:58.434 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-12T04:27:58.434 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-12T04:27:58.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.a 2024-03-12T04:27:58.752 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:27:58 smithi042 systemd[1]: Stopping Ceph mds.a for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:27:59.066 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:27:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a[71204]: 2024-03-12T04:27:58.749+0000 7f3f2ac93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:27:59.066 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:27:58 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a[71204]: 2024-03-12T04:27:58.749+0000 7f3f2ac93700 -1 mds.a *** got signal Terminated *** 2024-03-12T04:28:03.044 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:28:02 smithi042 podman[100333]: 2024-03-12 04:28:02.75638287 +0000 UTC m=+4.123867414 container died b8e97fa10157d3ab195fbc4ca70dc3c02e840a3cec80b10606e9ee1b70385f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:28:03.316 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:28:03 smithi042 podman[100333]: 2024-03-12 04:28:03.149541237 +0000 UTC m=+4.517025781 container cleanup b8e97fa10157d3ab195fbc4ca70dc3c02e840a3cec80b10606e9ee1b70385f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-12T04:28:03.316 INFO:journalctl@ceph.mds.a.smithi042.stdout:Mar 12 04:28:03 smithi042 bash[100333]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-a 2024-03-12T04:28:03.575 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.a.service' 2024-03-12T04:28:03.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:03.611 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-12T04:28:03.611 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-12T04:28:03.611 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.d 2024-03-12T04:28:04.058 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:28:03 smithi042 systemd[1]: Stopping Ceph mds.d for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:04.058 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:28:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d[70246]: 2024-03-12T04:28:03.919+0000 7f4241c1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:04.058 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:28:03 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d[70246]: 2024-03-12T04:28:03.919+0000 7f4241c1b700 -1 mds.d *** got signal Terminated *** 2024-03-12T04:28:06.316 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:28:06 smithi042 podman[100454]: 2024-03-12 04:28:06.039805095 +0000 UTC m=+2.213549650 container died ca93b1335576c2e0f00f382c69dd42b54cd0fbf10de93a2994f57122490b1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:28:06.772 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:28:06 smithi042 podman[100454]: 2024-03-12 04:28:06.354073191 +0000 UTC m=+2.527817746 container cleanup ca93b1335576c2e0f00f382c69dd42b54cd0fbf10de93a2994f57122490b1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-12T04:28:06.772 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:28:06 smithi042 bash[100454]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d 2024-03-12T04:28:06.772 INFO:journalctl@ceph.mds.d.smithi042.stdout:Mar 12 04:28:06 smithi042 podman[100468]: 2024-03-12 04:28:06.546975258 +0000 UTC m=+0.509575258 container remove ca93b1335576c2e0f00f382c69dd42b54cd0fbf10de93a2994f57122490b1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-12T04:28:06.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.d.service' 2024-03-12T04:28:06.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:06.825 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-12T04:28:06.825 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-12T04:28:06.826 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.g 2024-03-12T04:28:07.066 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:28:06 smithi042 systemd[1]: Stopping Ceph mds.g for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:07.566 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:28:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g[70855]: 2024-03-12T04:28:07.182+0000 7fb377f37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:07.566 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:28:07 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g[70855]: 2024-03-12T04:28:07.182+0000 7fb377f37700 -1 mds.g *** got signal Terminated *** 2024-03-12T04:28:11.469 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:28:11 smithi042 podman[100573]: 2024-03-12 04:28:11.193392538 +0000 UTC m=+4.154195702 container died 01229492569ce6efb85fb855ff686ee218f738c3155ee0b1c6796c96e57586c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:28:11.816 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:28:11 smithi042 podman[100573]: 2024-03-12 04:28:11.558322403 +0000 UTC m=+4.519125566 container cleanup 01229492569ce6efb85fb855ff686ee218f738c3155ee0b1c6796c96e57586c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-12T04:28:11.816 INFO:journalctl@ceph.mds.g.smithi042.stdout:Mar 12 04:28:11 smithi042 bash[100573]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-g 2024-03-12T04:28:11.994 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.g.service' 2024-03-12T04:28:12.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:12.029 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-12T04:28:12.029 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-12T04:28:12.029 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.j 2024-03-12T04:28:12.566 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:28:12 smithi042 systemd[1]: Stopping Ceph mds.j for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:12.566 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:28:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j[69892]: 2024-03-12T04:28:12.336+0000 7f2645a6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:12.566 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:28:12 smithi042 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j[69892]: 2024-03-12T04:28:12.336+0000 7f2645a6b700 -1 mds.j *** got signal Terminated *** 2024-03-12T04:28:15.398 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:28:15 smithi042 podman[100696]: 2024-03-12 04:28:15.136045623 +0000 UTC m=+2.893815179 container died 2e78f253e8940dca465192e29551affd1283e172de4b54b34e4fd019689668e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-12T04:28:15.690 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:28:15 smithi042 podman[100696]: 2024-03-12 04:28:15.479044724 +0000 UTC m=+3.236814279 container cleanup 2e78f253e8940dca465192e29551affd1283e172de4b54b34e4fd019689668e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-12T04:28:15.690 INFO:journalctl@ceph.mds.j.smithi042.stdout:Mar 12 04:28:15 smithi042 bash[100696]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-j 2024-03-12T04:28:15.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.j.service' 2024-03-12T04:28:15.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:15.944 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-12T04:28:15.944 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-12T04:28:15.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.b 2024-03-12T04:28:16.257 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:15 smithi119 systemd[1]: Stopping Ceph mds.b for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:16.602 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: 2024-03-12T04:28:16.254+0000 7f3efb571700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:16.602 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:16 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b[66776]: 2024-03-12T04:28:16.254+0000 7f3efb571700 -1 mds.b *** got signal Terminated *** 2024-03-12T04:28:19.723 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:19 smithi119 podman[73303]: 2024-03-12 04:28:19.459378921 +0000 UTC m=+3.331474176 container died c6dd84ddb419b6e64c2b90ba82f31b3ac6b1884418f446af203c931d18c60447 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-12T04:28:19.982 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:19 smithi119 podman[73303]: 2024-03-12 04:28:19.78759721 +0000 UTC m=+3.659692475 container cleanup c6dd84ddb419b6e64c2b90ba82f31b3ac6b1884418f446af203c931d18c60447 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-12T04:28:19.982 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:19 smithi119 bash[73303]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b 2024-03-12T04:28:20.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.b.service' 2024-03-12T04:28:20.234 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:19 smithi119 podman[73329]: 2024-03-12 04:28:19.980498515 +0000 UTC m=+0.516739105 container remove c6dd84ddb419b6e64c2b90ba82f31b3ac6b1884418f446af203c931d18c60447 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:28:20.234 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:20 smithi119 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.b.service: Deactivated successfully. 2024-03-12T04:28:20.235 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:20 smithi119 systemd[1]: Stopped Ceph mds.b for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:28:20.235 INFO:journalctl@ceph.mds.b.smithi119.stdout:Mar 12 04:28:20 smithi119 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.b.service: Consumed 3.594s CPU time. 2024-03-12T04:28:20.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:20.248 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-12T04:28:20.248 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-12T04:28:20.248 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.e 2024-03-12T04:28:20.576 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:20 smithi119 systemd[1]: Stopping Ceph mds.e for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:20.946 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e[66159]: 2024-03-12T04:28:20.574+0000 7fec94af3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:20.946 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:20 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e[66159]: 2024-03-12T04:28:20.574+0000 7fec94af3700 -1 mds.e *** got signal Terminated *** 2024-03-12T04:28:23.719 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:23 smithi119 podman[73435]: 2024-03-12 04:28:23.456372891 +0000 UTC m=+2.994572305 container died 41cc2f8e28edffbe93d7865fc79586548033f2375e463210b4a5b8fa67a9904d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-12T04:28:23.977 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:23 smithi119 podman[73435]: 2024-03-12 04:28:23.79120752 +0000 UTC m=+3.329406944 container cleanup 41cc2f8e28edffbe93d7865fc79586548033f2375e463210b4a5b8fa67a9904d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-12T04:28:23.978 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:23 smithi119 bash[73435]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e 2024-03-12T04:28:24.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.e.service' 2024-03-12T04:28:24.232 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:23 smithi119 podman[73449]: 2024-03-12 04:28:23.97614659 +0000 UTC m=+0.516839570 container remove 41cc2f8e28edffbe93d7865fc79586548033f2375e463210b4a5b8fa67a9904d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:28:24.232 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:24 smithi119 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.e.service: Deactivated successfully. 2024-03-12T04:28:24.232 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:24 smithi119 systemd[1]: Stopped Ceph mds.e for 7cca013a-e025-11ee-95c8-87774f69a715. 2024-03-12T04:28:24.232 INFO:journalctl@ceph.mds.e.smithi119.stdout:Mar 12 04:28:24 smithi119 systemd[1]: ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.e.service: Consumed 2.122s CPU time. 2024-03-12T04:28:24.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:24.244 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-12T04:28:24.244 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-12T04:28:24.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.h 2024-03-12T04:28:24.572 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:28:24 smithi119 systemd[1]: Stopping Ceph mds.h for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:24.946 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:28:24 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h[66466]: 2024-03-12T04:28:24.570+0000 7f32689f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:24.946 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:28:24 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h[66466]: 2024-03-12T04:28:24.570+0000 7f32689f7700 -1 mds.h *** got signal Terminated *** 2024-03-12T04:28:27.696 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:28:27 smithi119 podman[73555]: 2024-03-12 04:28:27.458789216 +0000 UTC m=+3.001563397 container died 06d5e2c6cabb93e5e21c8884b35f985d7420ce8e97317f80b9745ff3c48f9491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-12T04:28:28.081 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:28:27 smithi119 podman[73555]: 2024-03-12 04:28:27.812300099 +0000 UTC m=+3.355074280 container cleanup 06d5e2c6cabb93e5e21c8884b35f985d7420ce8e97317f80b9745ff3c48f9491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-12T04:28:28.081 INFO:journalctl@ceph.mds.h.smithi119.stdout:Mar 12 04:28:27 smithi119 bash[73555]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-h 2024-03-12T04:28:28.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.h.service' 2024-03-12T04:28:28.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:28.263 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-12T04:28:28.263 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-12T04:28:28.264 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.k 2024-03-12T04:28:28.696 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:28:28 smithi119 systemd[1]: Stopping Ceph mds.k for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:28.696 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:28:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k[65836]: 2024-03-12T04:28:28.582+0000 7f8fca01d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:28.696 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:28:28 smithi119 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k[65836]: 2024-03-12T04:28:28.582+0000 7f8fca01d700 -1 mds.k *** got signal Terminated *** 2024-03-12T04:28:31.753 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:28:31 smithi119 podman[73675]: 2024-03-12 04:28:31.462625536 +0000 UTC m=+2.986893010 container died a47debd23377d0e4fffb38dc819ca8354c238dbbf0a0c319c10815b701c1714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True) 2024-03-12T04:28:31.753 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:28:31 smithi119 podman[73675]: 2024-03-12 04:28:31.751084566 +0000 UTC m=+3.275352039 container cleanup a47debd23377d0e4fffb38dc819ca8354c238dbbf0a0c319c10815b701c1714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-12T04:28:32.084 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:28:31 smithi119 bash[73675]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k 2024-03-12T04:28:32.084 INFO:journalctl@ceph.mds.k.smithi119.stdout:Mar 12 04:28:31 smithi119 podman[73689]: 2024-03-12 04:28:31.97774899 +0000 UTC m=+0.510276461 container remove a47debd23377d0e4fffb38dc819ca8354c238dbbf0a0c319c10815b701c1714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-12T04:28:32.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.k.service' 2024-03-12T04:28:32.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:32.142 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-12T04:28:32.142 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-12T04:28:32.142 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.c 2024-03-12T04:28:32.457 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:28:32 smithi136 systemd[1]: Stopping Ceph mds.c for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:32.720 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:28:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c[65817]: 2024-03-12T04:28:32.567+0000 7f001feda700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:32.721 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:28:32 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c[65817]: 2024-03-12T04:28:32.567+0000 7f001feda700 -1 mds.c *** got signal Terminated *** 2024-03-12T04:28:35.710 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:28:35 smithi136 podman[72734]: 2024-03-12 04:28:35.438997578 +0000 UTC m=+3.106621789 container died 8fde661ac2918e0dac40b85cd278099e8075528917ba02e3d6422aa101035dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-94a5397) 2024-03-12T04:28:36.161 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:28:35 smithi136 podman[72734]: 2024-03-12 04:28:35.958469183 +0000 UTC m=+3.626093395 container cleanup 8fde661ac2918e0dac40b85cd278099e8075528917ba02e3d6422aa101035dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397) 2024-03-12T04:28:36.161 INFO:journalctl@ceph.mds.c.smithi136.stdout:Mar 12 04:28:35 smithi136 bash[72734]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-c 2024-03-12T04:28:36.280 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.c.service' 2024-03-12T04:28:36.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:36.319 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-12T04:28:36.319 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-12T04:28:36.319 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.f 2024-03-12T04:28:36.457 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:28:36 smithi136 systemd[1]: Stopping Ceph mds.f for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:36.957 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:28:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f[65511]: 2024-03-12T04:28:36.645+0000 7fb146375700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:36.957 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:28:36 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f[65511]: 2024-03-12T04:28:36.645+0000 7fb146375700 -1 mds.f *** got signal Terminated *** 2024-03-12T04:28:39.707 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:28:39 smithi136 podman[72868]: 2024-03-12 04:28:39.453869402 +0000 UTC m=+2.920148435 container died 055b1c6b7828cf91a3ae1cd9cc938b43451a8f63ef2b5b3c8b4d3f34ac5a207c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T04:28:40.202 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:28:39 smithi136 podman[72868]: 2024-03-12 04:28:39.797196787 +0000 UTC m=+3.263475774 container cleanup 055b1c6b7828cf91a3ae1cd9cc938b43451a8f63ef2b5b3c8b4d3f34ac5a207c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True) 2024-03-12T04:28:40.202 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:28:39 smithi136 bash[72868]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f 2024-03-12T04:28:40.202 INFO:journalctl@ceph.mds.f.smithi136.stdout:Mar 12 04:28:39 smithi136 podman[72882]: 2024-03-12 04:28:39.998403115 +0000 UTC m=+0.537624337 container remove 055b1c6b7828cf91a3ae1cd9cc938b43451a8f63ef2b5b3c8b4d3f34ac5a207c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-f, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-12T04:28:40.217 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.f.service' 2024-03-12T04:28:40.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:40.252 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-12T04:28:40.253 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-12T04:28:40.253 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.i 2024-03-12T04:28:40.561 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:28:40 smithi136 systemd[1]: Stopping Ceph mds.i for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:40.957 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:28:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i[66121]: 2024-03-12T04:28:40.559+0000 7fed2da34700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:40.957 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:28:40 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i[66121]: 2024-03-12T04:28:40.559+0000 7fed2da34700 -1 mds.i *** got signal Terminated *** 2024-03-12T04:28:43.712 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:28:43 smithi136 podman[72988]: 2024-03-12 04:28:43.459303866 +0000 UTC m=+2.992552182 container died e92a959ee9ecf2b9d0c795d5f3dd1a7ebd1242e7b63f684c17aec5935946aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-12T04:28:44.170 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:28:43 smithi136 podman[72988]: 2024-03-12 04:28:43.925774798 +0000 UTC m=+3.459023110 container cleanup e92a959ee9ecf2b9d0c795d5f3dd1a7ebd1242e7b63f684c17aec5935946aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T04:28:44.170 INFO:journalctl@ceph.mds.i.smithi136.stdout:Mar 12 04:28:43 smithi136 bash[72988]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-i 2024-03-12T04:28:44.339 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.i.service' 2024-03-12T04:28:44.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:44.374 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-12T04:28:44.374 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-12T04:28:44.374 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.l 2024-03-12T04:28:44.706 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:28:44 smithi136 systemd[1]: Stopping Ceph mds.l for 7cca013a-e025-11ee-95c8-87774f69a715... 2024-03-12T04:28:44.957 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:28:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l[65191]: 2024-03-12T04:28:44.704+0000 7fa8e4339700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T04:28:44.957 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:28:44 smithi136 ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l[65191]: 2024-03-12T04:28:44.704+0000 7fa8e4339700 -1 mds.l *** got signal Terminated *** 2024-03-12T04:28:47.707 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:28:47 smithi136 podman[73108]: 2024-03-12 04:28:47.440707836 +0000 UTC m=+2.843094787 container died d76170f81d9f56f205d3578e75dfc9821eed980b1847fb27b8f470dea8cc3008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T04:28:48.066 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:28:47 smithi136 podman[73108]: 2024-03-12 04:28:47.846490445 +0000 UTC m=+3.248877396 container cleanup d76170f81d9f56f205d3578e75dfc9821eed980b1847fb27b8f470dea8cc3008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-12T04:28:48.066 INFO:journalctl@ceph.mds.l.smithi136.stdout:Mar 12 04:28:47 smithi136 bash[73108]: ceph-7cca013a-e025-11ee-95c8-87774f69a715-mds-l 2024-03-12T04:28:48.181 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7cca013a-e025-11ee-95c8-87774f69a715@mds.l.service' 2024-03-12T04:28:48.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T04:28:48.215 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-12T04:28:48.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7cca013a-e025-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T04:28:48.413 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:28:50.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7cca013a-e025-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T04:28:51.051 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:28:53.482 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7cca013a-e025-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T04:28:53.673 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:28:55.972 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:28:56.007 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:28:56.041 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T04:28:56.071 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-12T04:28:56.073 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi042/crash 2024-03-12T04:28:56.074 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash -- . 2024-03-12T04:28:56.114 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T04:28:56.115 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2024-03-12T04:28:56.116 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi119/crash 2024-03-12T04:28:56.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash -- . 2024-03-12T04:28:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:tar: /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T04:28:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:tar: Error is not recoverable: exiting now 2024-03-12T04:28:56.243 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi136/crash 2024-03-12T04:28:56.244 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash -- . 2024-03-12T04:28:56.286 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/7cca013a-e025-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T04:28:56.286 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-03-12T04:28:56.287 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-12T04:28:56.287 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-12T04:28:56.322 INFO:teuthology.orchestra.run.smithi042.stdout:2024-03-12T04:06:44.320523+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-12T04:28:56.323 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-12T04:28:56.324 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[SEC\]' /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-12T04:28:56.398 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]' /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-12T04:28:56.473 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[WRN\]' /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-12T04:28:56.546 INFO:teuthology.orchestra.run.smithi042.stdout:2024-03-12T04:06:44.320523+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-12T04:28:56.547 INFO:tasks.cephadm:Compressing logs... 2024-03-12T04:28:56.547 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-12T04:28:56.589 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-12T04:28:56.591 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-12T04:28:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-12T04:28:56.613 INFO:teuthology.orchestra.run.smithi042.stderr:: No such file or directory 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.6.log 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.a.log 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.g.log 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.j.log 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.3.log 2024-03-12T04:28:56.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.x.log 2024-03-12T04:28:56.621 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-12T04:28:56.621 INFO:teuthology.orchestra.run.smithi136.stderr:: No such file or directory 2024-03-12T04:28:56.621 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.l.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.i.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.l.log: /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.11.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.z.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.11.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.8.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.5.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.f.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-12T04:28:56.622 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.f.log: /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.5.log: 90.2% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:: No such file or directory 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.10.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.b.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.4.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.1.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.4.log: -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.k.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.h.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.7.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.h.log: /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.e.log 2024-03-12T04:28:56.629 INFO:teuthology.orchestra.run.smithi119.stderr: 90.2% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-12T04:28:56.631 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.3.log: 90.1% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-12T04:28:56.631 INFO:teuthology.orchestra.run.smithi042.stderr: 94.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.j.log.gz 2024-03-12T04:28:56.631 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.0.log 2024-03-12T04:28:56.631 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.x.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-12T04:28:56.631 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-12T04:28:56.632 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log 2024-03-12T04:28:56.632 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-12T04:28:56.632 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-12T04:28:56.632 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-12T04:28:56.632 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.9.log 2024-03-12T04:28:56.632 INFO:teuthology.orchestra.run.smithi136.stderr: 91.9% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.z.log.gz 2024-03-12T04:28:56.634 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log 2024-03-12T04:28:56.634 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.2.log 2024-03-12T04:28:56.634 INFO:teuthology.orchestra.run.smithi136.stderr: 83.6% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-12T04:28:56.638 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-12T04:28:56.639 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.y.log 2024-03-12T04:28:56.641 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.d.log 2024-03-12T04:28:56.646 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-12T04:28:56.648 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.c.log 2024-03-12T04:28:56.648 INFO:teuthology.orchestra.run.smithi136.stderr:gzip/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-12T04:28:56.649 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.a.log 2024-03-12T04:28:56.650 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-12T04:28:56.660 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.c.log 2024-03-12T04:28:56.662 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.y.log: 91.8%gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log 2024-03-12T04:28:56.662 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-12T04:28:56.666 INFO:teuthology.orchestra.run.smithi042.stderr: 94.6%/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.a.log: -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.d.log.gz 2024-03-12T04:28:56.669 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-12T04:28:56.669 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.y.log.gz 2024-03-12T04:28:56.676 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.c.log: 94.4% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.l.log.gz 2024-03-12T04:28:56.678 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log: 87.5%gzip -5 --verbose -- /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.b.log 2024-03-12T04:28:56.682 INFO:teuthology.orchestra.run.smithi042.stderr: 94.4% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.g.log.gz 2024-03-12T04:28:56.684 INFO:teuthology.orchestra.run.smithi136.stderr: 91.0% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-12T04:28:56.686 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.k.log.gz 2024-03-12T04:28:56.686 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-12T04:28:56.691 INFO:teuthology.orchestra.run.smithi136.stderr: 92.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.i.log.gz 2024-03-12T04:28:56.693 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-12T04:28:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.h.log.gz 2024-03-12T04:28:56.707 INFO:teuthology.orchestra.run.smithi136.stderr: 92.6% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.c.log.gz 2024-03-12T04:28:56.707 INFO:teuthology.orchestra.run.smithi136.stderr: 94.0% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.f.log.gz 2024-03-12T04:28:56.708 INFO:teuthology.orchestra.run.smithi119.stderr: 94.6% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.e.log.gz 2024-03-12T04:28:56.748 INFO:teuthology.orchestra.run.smithi042.stderr: 92.6% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.a.log.gz 2024-03-12T04:28:56.957 INFO:teuthology.orchestra.run.smithi042.stderr: 89.9% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mgr.x.log.gz 2024-03-12T04:28:56.996 INFO:teuthology.orchestra.run.smithi119.stderr: 92.2% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.b.log.gz 2024-03-12T04:28:57.019 INFO:teuthology.orchestra.run.smithi136.stderr: 92.1% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.c.log.gz 2024-03-12T04:28:57.085 INFO:teuthology.orchestra.run.smithi119.stderr: 93.1% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mds.b.log.gz 2024-03-12T04:28:57.940 INFO:teuthology.orchestra.run.smithi042.stderr: 90.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-mon.a.log.gz 2024-03-12T04:29:05.277 INFO:teuthology.orchestra.run.smithi042.stderr: 90.6% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.9.log.gz 2024-03-12T04:29:06.386 INFO:teuthology.orchestra.run.smithi136.stderr: 90.9% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.8.log.gz 2024-03-12T04:29:06.748 INFO:teuthology.orchestra.run.smithi042.stderr: 90.7% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.6.log.gz 2024-03-12T04:29:07.403 INFO:teuthology.orchestra.run.smithi042.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-12T04:29:07.938 INFO:teuthology.orchestra.run.smithi119.stderr: 90.7% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.7.log.gz 2024-03-12T04:29:09.779 INFO:teuthology.orchestra.run.smithi136.stderr: 90.7% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.11.log.gz 2024-03-12T04:29:10.228 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.10.log.gz 2024-03-12T04:29:11.064 INFO:teuthology.orchestra.run.smithi136.stderr: 90.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.2.log.gz 2024-03-12T04:29:11.147 INFO:teuthology.orchestra.run.smithi119.stderr: 90.5% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.1.log.gz 2024-03-12T04:29:11.512 INFO:teuthology.orchestra.run.smithi042.stderr: 90.7% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.3.log.gz 2024-03-12T04:29:11.774 INFO:teuthology.orchestra.run.smithi119.stderr: 90.6% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.4.log.gz 2024-03-12T04:29:11.776 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-03-12T04:29:11.776 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m15.171s 2024-03-12T04:29:11.776 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m53.830s 2024-03-12T04:29:11.777 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m1.637s 2024-03-12T04:29:16.185 INFO:teuthology.orchestra.run.smithi136.stderr: 90.6% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.5.log.gz 2024-03-12T04:29:16.187 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-03-12T04:29:16.187 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m19.581s 2024-03-12T04:29:16.187 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m55.770s 2024-03-12T04:29:16.187 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m1.673s 2024-03-12T04:29:18.490 INFO:teuthology.orchestra.run.smithi042.stderr: 90.4% -- replaced with /var/log/ceph/7cca013a-e025-11ee-95c8-87774f69a715/ceph-osd.0.log.gz 2024-03-12T04:29:18.491 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-03-12T04:29:18.492 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m21.891s 2024-03-12T04:29:18.492 INFO:teuthology.orchestra.run.smithi042.stderr:user 1m6.150s 2024-03-12T04:29:18.492 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m1.897s 2024-03-12T04:29:18.492 INFO:tasks.cephadm:Archiving logs... 2024-03-12T04:29:18.492 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi042/log 2024-03-12T04:29:18.493 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T04:29:28.306 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi119/log 2024-03-12T04:29:28.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T04:29:37.919 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi136/log 2024-03-12T04:29:37.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T04:29:47.861 INFO:tasks.cephadm:Removing cluster... 2024-03-12T04:29:47.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7cca013a-e025-11ee-95c8-87774f69a715 --force 2024-03-12T04:29:48.061 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:29:48.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7cca013a-e025-11ee-95c8-87774f69a715 --force 2024-03-12T04:29:48.576 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:29:48.902 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7cca013a-e025-11ee-95c8-87774f69a715 --force 2024-03-12T04:29:49.094 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 7cca013a-e025-11ee-95c8-87774f69a715 2024-03-12T04:29:49.431 INFO:tasks.cephadm:Removing cephadm ... 2024-03-12T04:29:49.431 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T04:29:49.450 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T04:29:49.467 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T04:29:49.481 INFO:tasks.cephadm:Teardown complete 2024-03-12T04:29:49.482 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-12T04:29:49.495 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-12T04:29:49.496 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T04:29:49.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T04:29:49.509 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T04:29:49.607 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-12T04:29:49.617 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-12T04:29:49.617 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T04:29:49.620 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T04:29:49.622 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T04:29:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-03-12T04:29:49.634 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-03-12T04:29:49.635 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-03-12T04:29:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T04:29:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-03-12T04:29:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 -258us[ -258us] +/- 131ms 2024-03-12T04:29:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 -1889us[-1868us] +/- 69ms 2024-03-12T04:29:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +1939us[+1939us] +/- 67ms 2024-03-12T04:29:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:29:49.639 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T04:29:49.640 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-03-12T04:29:49.640 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 -227us[ -227us] +/- 131ms 2024-03-12T04:29:49.640 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -1874us[-1898us] +/- 69ms 2024-03-12T04:29:49.640 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1982us[+1958us] +/- 67ms 2024-03-12T04:29:49.640 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:29:49.641 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T04:29:49.641 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-03-12T04:29:49.641 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 -204us[ -204us] +/- 131ms 2024-03-12T04:29:49.641 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -1847us[-1855us] +/- 69ms 2024-03-12T04:29:49.641 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +1992us[+1983us] +/- 67ms 2024-03-12T04:29:49.642 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:29:49.642 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-12T04:29:49.651 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-12T04:29:49.652 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-12T04:29:49.720 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T04:29:49.811 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T04:29:49.884 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T04:29:49.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T04:29:49.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:29:49.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T04:29:50.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:29:50.017 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T04:29:50.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:29:50.048 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-12T04:29:50.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-12T04:29:50.078 INFO:teuthology.task.internal:Duration was 2095.966434 seconds 2024-03-12T04:29:50.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-12T04:29:50.100 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-12T04:29:50.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T04:29:50.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T04:29:50.106 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T04:29:50.146 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T04:29:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T04:29:50.149 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T04:29:50.487 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-12T04:29:50.487 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-03-12T04:29:50.487 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T04:29:50.507 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-03-12T04:29:50.507 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T04:29:50.555 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-03-12T04:29:50.555 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T04:29:50.575 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-12T04:29:50.576 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T04:29:50.578 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T04:29:50.598 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T04:29:50.645 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-12T04:29:50.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T04:29:50.647 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T04:29:50.666 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T04:29:51.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-12T04:29:51.053 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-12T04:29:51.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T04:29:51.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T04:29:51.106 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T04:29:51.133 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-12T04:29:51.143 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T04:29:51.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T04:29:51.149 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T04:29:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-03-12T04:29:51.172 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-03-12T04:29:51.198 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-03-12T04:29:51.222 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T04:29:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:29:51.238 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T04:29:51.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:29:51.252 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T04:29:51.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:29:51.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-12T04:29:51.287 INFO:teuthology.task.internal:Transferring archived files... 2024-03-12T04:29:51.287 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi042 2024-03-12T04:29:51.288 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T04:29:51.331 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi119 2024-03-12T04:29:51.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T04:29:51.367 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348/remote/smithi136 2024-03-12T04:29:51.368 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T04:29:51.406 INFO:teuthology.task.internal:Removing archive directory... 2024-03-12T04:29:51.406 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T04:29:51.408 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T04:29:51.411 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T04:29:51.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-12T04:29:51.467 INFO:teuthology.task.internal:Not uploading archives. 2024-03-12T04:29:51.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-12T04:29:51.478 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-12T04:29:51.479 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T04:29:51.481 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T04:29:51.483 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T04:29:51.493 INFO:teuthology.orchestra.run.smithi042.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 04:29 /home/ubuntu/cephtest 2024-03-12T04:29:51.495 INFO:teuthology.orchestra.run.smithi119.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 04:29 /home/ubuntu/cephtest 2024-03-12T04:29:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 04:29 /home/ubuntu/cephtest 2024-03-12T04:29:51.510 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-12T04:29:51.521 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-12T04:29:51.544 INFO:teuthology.nuke:Checking targets against current locks 2024-03-12T04:29:51.570 DEBUG:teuthology.nuke:shortname: smithi042 2024-03-12T04:29:51.571 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T04:29:51.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 03:42:37.212831', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJwPTkt6lpgayKwmzGh6G+CKpCR7nFE6xi/tjhd2VFmzYQm/EjQK+BoOEBczi14EEvC216yVNmz8BVh3Mj88a5k='} 2024-03-12T04:29:51.600 DEBUG:teuthology.nuke:shortname: smithi119 2024-03-12T04:29:51.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T04:29:51.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 03:42:37.210894', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5KBhfoTIyfHd2ltYBKNkMkHxZM2TzqfsHQjC6DZhgyDMU2qf7iepU6kSc1Oz4Kn8oOeGNJR42yvHLG4aOjNXc='} 2024-03-12T04:29:51.630 DEBUG:teuthology.nuke:shortname: smithi136 2024-03-12T04:29:51.630 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T04:29:51.656 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 03:42:37.214475', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3BJyFff1cqpwLxYCcbthQ1lcefv47aApahzvQRmI9yl7fjCBZWRzo0rZpHzbnIZSR78GU2uI5gkl8cbCEbAOc='} 2024-03-12T04:29:51.663 INFO:teuthology.orchestra.console.smithi042:Power off 2024-03-12T04:29:51.663 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T04:29:51.687 DEBUG:teuthology.orchestra.console.smithi042:power off output: Chassis Power Control: Down/Off 2024-03-12T04:29:51.687 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:51.702 INFO:teuthology.orchestra.console.smithi119:Power off 2024-03-12T04:29:51.703 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T04:29:51.717 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-03-12T04:29:51.725 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2024-03-12T04:29:51.725 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:51.745 INFO:teuthology.orchestra.console.smithi136:Power off 2024-03-12T04:29:51.745 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T04:29:51.754 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-03-12T04:29:51.766 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2024-03-12T04:29:51.767 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:51.789 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-12T04:29:55.717 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:55.755 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:55.790 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:55.830 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-03-12T04:29:55.865 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-03-12T04:29:55.899 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-12T04:29:59.832 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:59.866 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:59.901 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:29:59.943 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-03-12T04:29:59.975 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-03-12T04:30:00.010 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-12T04:30:00.044 INFO:teuthology.orchestra.console.smithi042:Power off completed 2024-03-12T04:30:03.978 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:30:04.010 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T04:30:04.090 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-03-12T04:30:04.121 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-03-12T04:30:04.191 INFO:teuthology.orchestra.console.smithi119:Power off completed 2024-03-12T04:30:04.222 INFO:teuthology.orchestra.console.smithi136:Power off completed 2024-03-12T04:30:04.327 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} duration: 2095.966434240341 failure_reason: '"2024-03-12T04:06:44.320523+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-03-12T04:30:04.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:30:04.423 INFO:teuthology.run:FAIL